Efficient Web Accessibility Testing — Tom Tester's report

Three pages from the Yr Meteorological Institute weather reporting site were selected for analysis. Failures already identified for one type of page are not reported for other types of pages, so the results are cumulative and unique. Apart from clarity of presentation, the rationale behind this is that once a single instance of a failure is identified, the developer would audit the entire site code for similar failures. In addition, the main purpose of this exercise is to highlight the usefulness — or lack thereof — of selected automated accessibility checkers in finding failures.

The results of analysis performed are consistent with the detailed individual assessment of selected tools. TotalValidator is focused mostly in parser-type failures which are easiest to find and code for in automated software, whereas Wave and Firefox Accessibility Evaluator are only of very limited use, but still valuable as tools due to their ability to identify problems hard to find for a human developer, such as contrast ration failures, instances of duplicated text, or broken references.

Out of the total of 64 failures identified during analysis, the human evaluator found 59, or 92.2%, TotalValidator found 9, or 14%, Wave found 11, or 17.2%, while the FAE extension found 12, or 18.8%.

Yr Meteorological Institute — Front page

The matrix of failures for the front page

For each accessibility failure, the table indicates whether it was found by a human evaluator, TotalValidator, Wave, and FAE, respectively.
Failure Human TotalValidator Wave FAE
The page has no real outline, or well-defined sections. There are no level-1 headings, only subheadings, even though clearly there are distinct content groups present yes no yes yes
Subheadings are empty (containing only a non-breaking space,  ) yes no yes yes
The strong emphasis element is misused for presentational purposes (bold text) — as a child of a subheadings, as a child of the table caption, and as a definition list term substitute, also on location weather pages yes no no no
Text search form controls are not marked up as such (input type=search) — instead, plain text inputs are used yes no no no
Not all distinct content is grouped in meaningful containers (such as main, article, or section). Generic layers (div) are used instead yes no no no
Subheading content is linked, whereas the long text node sufficient for a separate paragraph should follow the heading as a real paragraph, with the relevant parts linked instead yes no no no
Images with an empty alt attribute are present yes yes yes yes
Elements with an empty title are present (title="") yes yes no no
Anchors with an empty title (see above) have child image elements (img). Titles relevant for the anchors are used for the child image instead (Til forsiden på yr.no) yes no no no
The site logo is not a descendant of a header element yes no no no
The language selector contains empty ins elements, used for insertion of icons through CSS. This is the task for CSS generated content, whereas the ins element should be used for marking up document changes instead. Empty span elements are misused for the same purpose in the location weather pages yes no no yes
The language chooser is implemented through JavaScript, but it will be inoperable in visual browsers with scripting on and enhanced security, due to cross-domain sharing of scripts (see XSS). The same applies to Yr Tools on location weather pages yes no no no
Regardless of the language chosen (including if the default Norwegian Bokmål is retained), the page offers mixed-language content that is not marked up with the lang attribute. This is due to inadequate and prone-to-failure server-side language detection. This failure applies equally to text browsers and to visual, script-enabled browsers, also on location weather pages yes no no no
Form search auto-suggestion is implemented with JavaScript. While the search form is usable without scripting, unskipable auto-suggestion results are still injected into the page, in a wrong language yes no no no
The auto-suggestion code contains fake anchors with an empty href attribute yes no no no
The auto-suggestion result block is a list of locations. It is marked up with a layout table that has no ARIA roles assigned to them yes no no no
Auto-suggestion result tables have no description, whether in the form of captions, summaries, a details element, or descriptive text in any way associated with them yes yes no yes
Ornamental icons used for auto-suggestion result presentation are hardcoded, whereas CSS should have been used instead yes no no no
The auto-suggestion result block includes raw, encoded XML markup: <Type> i <kommune>, <område> that has no place in an HTML document yes no no no
Definition lists are not marked up as such. Instead, the strong element is misused instead of the definition term, dt, and a generic span element is used to imitate the definition description element, dd yes no no no
Abbreviations such as f.eks. are not marked up with the abbr element yes no no no
Empty caption elements are present (containing only a non-breaking space, &nbsp; yes no no no
Redundant title text is used, identical to the raw anchor text content yes no yes no
Although equipped with a descriptive title, numerical variable values such as temperatures are marked up with a generic span wrapper element, instead of the var element yes no no no
Table cells with img children elements have identical, duplicated title text as said image elements, and also as the alt content, resulting in the same text being read three times (for example: fog, fog, fog) in screen readers yes no yes no
In data tables, there are other cells containing image children, where the descriptive title is erroneously applied to the cell instead of the content image yes no no no
Spacer shims are used for layout yes no yes yes
The map is implemented as an unskipable collection of 50+ repetitive content — absolutely positioned generic layers and numerous images with no alt attributes yes no no no
The weather progress animation has a motion control, implemented through a button with no text content, only a title yes no yes no
Lists are used for navigation without description, preceding heading, or meaningful markup (e.g. the nav element) yes no no no
The figure element is not used for diagrams with long description. The same applies to images which do have a long description provided. On location weather pages, the dl element is misused for this purpose. The same applies to webcam images and their descriptions on location weather pages, and to meteogram diagrams for detailed weather pages yes no no no
If an image has a long, multiple-lines description provided, it is injected into the title attribute instead of regular content, to save space, making this content unavailable to sighted users unless they navigate there with a device yes no yes no
News snippets utilize the level-3 subheading as the only content, misusing the heading for a section that does not have content following the heading yes no no no
The time element is not used to mark up dates and times. Instead, the emphasis element, em, is misused for this purpose yes no no no
Inline frames do not have an associated title yes yes no yes
Inline frames do not have fallback content yes no no no
The page footer is not marked up as such (the footer element), and its contents are unskipable — instead, multiple generic presentational layers are employed yes no no no
The unmarked footer contains a hidden navigation list not accessible to visual browser users yes no no no
The noscript element is used for advertisements, with its contents incorrectly encoded, mangling up the elements into a stream of meaningless characters: 
 <img 
src="http://nrk.tns-cs.net/
j0=,,,;+,cp=nrk/
yr+url=yr.no/
forsida.aspx;;;"
yes no no no
Absolute values are used for font size (pixels) yes no no no
Visual presentation on anchors relies solely on color, with underlines removed via CSS yes no no no
Color values in CSS are coded incorrectly, in hexagonal form without the pound sign, resulting in the color declaration being dropped, which can lead to color mismatch for users yes no no no
Aural styles are not used; in particular, the speak: none declaration is not used in combination with the display: none declaration, wherever the latter is required yes no no no
Meta refresh with a timeout is used no yes yes no
Multiple links have the same text content no yes yes yes
Insufficient contrast ratio on the page (below 7:1) no no yes yes

Yr Meteorological Institute — Location

The matrix of failures for the location weather page

For each accessibility failure, the table indicates whether it was found by a human evaluator, TotalValidator, Wave, and FAE, respectively.
Failure Human TotalValidator Wave FAE
The strong emphasis element is misused for presentational purposes (bold text) — as a child of a level-1 heading yes no no no
The breadcrumb navigation list is unskipable yes no no no
The contrast ratio for the breadcrumb navigation is below minimum acceptable level (3:1). Elsewhere, very light gray text color is used on white background for presentational purposes (to diminish the importance of the rain data table footer), creating severe contrast issues yes no no yes
Meaningful markup (the nav element) is not used for the breadcrumb navigation control yes no no no
Fake JavaScript links are employed for content update information — unclickable in a visual browser yes no no no
The content updates are marked up with a table without a caption element. Given the presence of a long summary, the details and summary elements should have been used as descendants of the caption element yes no no no
Abbreviations are either not marked up as such (such as PDF) with abbr or marked inconsistently (e.g. KL for klokken) yes no no no
If the abbr element is used, the title description is provided more than once yes no no no
Absolute (pixel) widths are used for the 'tabbed' weather report type navigation. The values used are custom-tailored to the designer-chosen narrow font type, whereas users of other common fonts will face cut-off cell content yes no no no
The usage of absolute widths on cells in combination with hover also results in the content jumping up and down, creating unnecessary, distracting motion yes no no no
There are complex tables present with non-linear structure — i.e. where the number of cells in a row or column does not always correspond to the maximum number of rows or columns in a table due to the presence of colspan or rowspan attributes. Cells in these complex tables have no headers, or id attributes used to associate cells with their headers yes no no yes
An empty (and obsolete) summary attribute is used on a table, instead of a caption yes yes no no
Rain data provides location information in the title attribute, but not in the content yes no no no
The em emphasis element is misused for the purposes of distinguishing the authorship data, resulting in an unnecessary tone change, whereas the cite element should have been used instead yes no no no
The list of nearby places is not marked up as a list. Instead, a stream of anchors is used for this purpose, creating an unskipable link soup yes no no no
Ambiguous/insufficient text is used for anchors (e.g. read more) yes no no yes
Elements with a duplicate id attribute are used, a source of confusion for screen readers no yes no no
There are multiple instances of links with identical text no yes no no

Yr Meteorological Institute — Hour-by-hour forecast

No new issues have been found.