Efficient Web Accessibility Testing — WebAIM Wave

Version

Versions tested: various; latest: 0.1.1.4

Mode of operation

Wave is available in three different forms:

The Chrome/Chromium extension is recommended by the authors as more featured than the Firefox toolbar (in particular, the color contrast diagnostic tool is available). Since web development requires testing in all major browsers, developers preferring Firefox should have no qualms about using the extension. Said extension does not download resources on its own, thus saving bandwidth for the tested servers, which makes it preferable to using the web service. Unlike several automated or semi-automated checkers, this extension is under active development.

Batch mode
Wave does not have a batch mode, requiring manual activation for every document tested. In that, it is a semi-automated tool, which is not very suitable or convenient for validating conformance of existing large sites, testsuites, and other multi-document projects of sufficient scale.
Single-document mode — online
Wave is useful for on-the-fly development of new documents, provided they are already online. There is no way to automatically start Wave for a newly loaded page. There are also no keyboard shortcuts. In that, Wave requires a mouse to use. See also the next section for more on this.
Single-document mode — offline
Wave doesn't work offline — the extension is disabled, while for obvious reasons the service cannot be used.

Error reporting

The primary mode of operation for Wave is through a sidebar, where an HTML document with the report for the currently tested page is created. As mentioned in the previous section, the mouse is required for operation. However, due to how extensions work in Google Chrome/Chromium, extension icons are displayed on the right side of the main address toolbar, while the sidebar is created on the left hand side of the window. This requires a lot of hand movement, which is very straining if a large number of documents has to be tested.

The report page is "tabbed", with the summary on the first, default tab. While this allows to ascertain how many errors or alerts (warnings) are present in the document, it would have been more useful if the error tab had been auto-activated if there was more error than one. One could also question the decision to split the report into tabs. As it is, result inspection is more labor-intensive than it has to be.

It is interesting to note that Wave doesn't use background or color for the documents it generates. Users with user stylesheets will be confronted with a messy page, though it is easily fixable. A larger problem involves style conflicts between author styles for the document being tested and Wave injections.

Wave uses icons (without titles) to visually "mark up" the location of the problem, in the vein of a WYSIWYG program. This is a poor design decision, as it requires the user to learn the meaning of the icons and study the sidebar-supplied legend — what is commonly described as a "mystery meat" problem. The icons break the visual layout of all but the simplest web pages, very often making it very difficult to find the problem, the analysis of which requires source inspection anyway. A much better solution would have been to inject problem descriptions into the line-numbered source of the document, and present that instead (much in the vein of TotalValidator).

Another issue, similar to the (mis-)placement of icon flags is the layout breakage due to layer injection, the styles of which can conflict with the page styles. In particular, Wave uses a generic div element for its informational layers, and therefore author styles for that element will be in conflict with those of Wave. One example is a site where this element is animated, and assigned a substantial width and height as percentage of the viewport. The informational layers of Wave will obscure the content of the page, and then scroll off the viewport, making it impossible to read them. This method of style injection can be used if and only if all possible styles that can be applied to the injected element are reset first. Even then, if the page uses structural selectors to style the content, such layers will break this structure, messing up the display.

CSS

Wave does not parse stylesheets at all. If a stylesheet is loaded directly, the extension will silently fail — clicking the icon does not open the sidebar. Even if there is no CSS support as such, the sidebar should open and say so, presenting a note to that effect.

SVG

Wave does not parse scalable vector graphics at all. If a vector graphic is loaded directly, the extension will silently fail — clicking the icon does not open the sidebar. Even if there is no SVG support as such, the sidebar should open and say so, presenting a note to that effect.

On the other hand, Wave does not issue bogus report for inline SVG, or islands, when vector graphics are directly used in the HTML document, and not loaded as an external resource. This eliminates false positives.

XML

The failure to parse SVG is actually part of a bigger problem. Wave does not parse XML documents at all. The extension silently fails — clicking the icon does not open the sidebar. Even if there is no XML support as such, the sidebar should open and say so, presenting a note to that effect. Examples of document types resulting in a failure include:

Server headers — language

The most prodigious source of false positives in Wave is due to its failure to check and process server headers, which are the most efficient way to declare encoding, language, content type, navigation structure of auxiliary documents, and even stylesheets. An example to achieve this is the Apache server configuration:

AddLanguage no .html

This can be overriden anywhere down in the URL hierachy, or from the top configuration file, for example through the <Files> directive. Server headers are trivially checked from within the graphical browser — here examples from Firefox/Pentadactyl:

server response, Norwegian version server response, English version
Server responses in a graphical browser

The same information can be trivially obtained from the command line, using a text browser:

% lynx -head -dump //unicus.no/ | grep Language
Content-Language: no
% lynx -head -dump //unicus.no/en/ | grep Language
Content-Language: en

Wave does not check and process server headers, and thus will complain that the language is not set for the document, with the following error:

Document language missing

Using the lang attribute is the only way to avoid the false positive (note that xml:lang should not be used in HTML pages, and Wave cannot parse documents delivered as XML — see the previous section). This attribute makes sense for standalone documents (such as testcases), and even then only for those which are shared through other means. Recommending this attribute is poor advice even for small servers, let alone huge sites with thousands of pages. Documents servers with a valid server header should never fail validation.

Failure to check and parse server headers is a critical flaw in automated accessibility checkers:

Wave does not check, parse, or process server headers, but at least it doesn't issue bogus content type error warnings for other server headers, such as Content-Script-Type and Content-Style-Type.

Server headers — links

Stylesheets can be added to documents via link elements, but links need not reside within the documents themselves — they can be set on the server. An example for this is the Apache server configuration:

<Files ~ "index\.(html)$">
Header add Link '</css/define.css>; rel="stylesheet"'
</Files>

This directive injects a stylesheet link to all documents with the html extension. It may be used as a global stylesheet for the entire site, as a base stylesheet to be overridden by document-specific stylesheets, or as a way to neatly define CSS Variables, on its own a useful accessibility feature of cascading stylesheets. It is unfortunately not mentioned in the WCAG 2.0 standard, which it post-dates.

Wave is unaware of such links. Non-sensical or otherwise erroneous headers sent by misconfigured server will also not be caught. The same applies to real color contrast errors which may be present in such stylesheets.

Server headers — site navigation links

Other link types can also be specified on the server, such as the sitemap, copyright notice, newsfeed, and many more:

<Files ~ "index\.(html)$">
Header add Link '</index/>; rel=index'
Header add Link '</legal/>; rel=copyright'
Header add Link '</atom.xml>; rel=alternate; type=application/atom+xml;
	title="site updates newsfeed"'
</Files>

These links affect accessibility in more ways than one. Some applications populate a separate navigation bar with these links, where they may appear as text and/or icon buttons. Others may add it to the text menu. Some more advanced search engine robots process server headers and may present information extracted thus in the site overview. It is also recommended that newsfeeds are appended to the documents through such a link.

Wave is unaware of such links. Non-sensical or otherwise erroneous headers sent by misconfigured server will also not be caught. Documents linking to newsfeeds within the body of the document, but neither through a head link nor a server header link, are not flagged.

Color contrast

Like Google Accessibility Developer Tools, Wave offers a built-in color contrast checker, based on the same codebase. Section 1.4.3 of the WCAG specification mandates that depending on the font size, sufficiently high contrast ratio of the text and background must be provided in order to make the text readable. Quite sensibly, the specification is less stringent in the case of very large text, in which case the letter shapes are much more discernible regardless of color. The highest contrast is that of black text on white background, or vice versa, while the lowest contrast is that of white text on white background — or more generally, when the foreground and background colors are identical. WCAG specifies two thresholds, depending on the text size. In our case, the minimum value is 4.5 to 1.

The Wave contrast checker incorrectly reports very low contrast in certain cases:

a {
  color: hsl(232,10%,95%);
  background: linear-gradient(hsl(232,40%,22%),hsl(232,40%,38%));
}

This pair of foreground and background colors satisfies the WCAG minimum contrast requirement, with plenty of room to spare. In fact, Wave would report a failure even the highest contrast possible was used:

a {
  color: white;
  background: linear-gradient(black,black);
}

a {
  color: white;
  background: black linear-gradient(black,black);
}

White on black is reported as wrong not because the constituent colors are wrong, but because Wave does not have support for the linear-gradient or radial-gradient properties. This failure is not due to an incorrect heuristic as such — the colors are processed correctly, but the incomplete implementation of the CSS specification results in the entire declaration being dropped, which in turn implies that Wave is left with an incorrect pair of colors during contrast evaluation.

Objects

Wave incorrectly interprets all object elements as plugins, issuing a warning (alert), rather than an error. While some objects with parameters may be used as plugin containers, this element is a generic element, which can also be used for inline text frames and, in particular, image objects:

<object data="/path/to/image">
	...object fallback...
</object>

The type attribute is reduntant as long as the relevant Content-Type header is set, and in the case of common images, it always is. The difference is only in browser processing speed, depending on implementation. Regardless, Wave will always flag objects as plugins, which in the case of images is wrong, becoming a source of false positives. This is also true for otherwise empty objects.

Alerts: Plugin

What It Means: An unidentified plugin is present.

Why It Matters: Plugins allow the introduction of non-HTML content, media players, etc. Because of limitations in non-HTML content, these often introduce accessibility issues.

How to Fix It: Provide an HTML alternative or ensure the plugin content is accessible. Provide a link to download any required software.

The Algorithm... in English: An <object> or <embed> element is present that is not identified as Flash, Quicktime, RealPlayer, or Windows Media Player.

Standards and Guidelines: Section 508 (m)

ARIA for elements with events

ARIA is not used almost anywhere on the net. Wave will correctly fail documents which use the role attribute, but do not fix the underlying accessibility problem.

<div role="button" onclick="alert('Button Pressed')">button</div>

With the advent of HTML5, most ARIA workarounds have been obsoleted. Rather than using duct tape to hold the document together, authors should use relevant semantics.

An event handler is present that may not be accessible.

Why It Matters: The JavaScript events in use do not appear to be accessible to both mouse and keyboard users. To be fully accessible, critical JavaScript interaction should be device independent.

How to Fix It: Ensure that critical functionality and content is accessible by using a device independent event handler (which responds to both keyboard and mouse) or by using both a mouse dependent and a keyboard dependent event handler.

The Algorithm... in English: One of the following are present: an onmouseover event but not an onfocus event an onclick event on something other than a link, form control, or element with a tabindex value of 0 ondblclicktill doesn't work if go back nav, unless html page reloaded

In that, Wave is useful for flagging documents which, although using ARIA, still require work to be not only comformant, but truly accessible. A pig with lipstick on is still a pig.

Suspicious link text

Continuing with useful features: if the anchor text is bogus, Wave will correctly flag it as "suspicious":

<a href="/">click me</a>
<a href="/">here</a>

Multiple caption elements

On the downside, Wave does not flag multiple caption elements as erroneous:

<table>
<caption>sufficient description here</caption>
<caption>invalid caption element</caption>

Table layout

Wave will sometimes flag the presence of tables which it suspects of being used for layout. This is not reported as an error, or even a warning (alert), but listed as a feature. It is important to keep this in mind, since only the former two are considered issues which require intervention.

Colspan detection

If there is a data table with a table header cell with a colspan attribute, the value of which matches the maximum number of cells in this table, then Wave will issue a recommendation to use a table caption instead. This is smart, but not smart enough, because there can only be one caption per table, and it is preposterous to assume that the page author abused the attribute. Not all instances of colspan matching the cell number max are synonymous with a caption, and most will have no relation to it, because the caption element is used to provide a table contents description, even including long-winded details and summary element descendants. While such a warning flag may catch one instance of a full table spanner of a table header cell, such spanners may be used as headers for table sections, marked by tbody elements. Wave should instead recommend using table sections if there are none, rather than flagging max-column colspan as the source of the problem.

Noscript

Any noscript element will be flagged as an error.

Content within <noscript> is presented if JavaScript is disabled. Because nearly all users (including users of screen readers and other assistive technologies) have JavaScript enabled, <noscript> cannot be used to provide an accessible version of inaccessible scripted content.

How to Fix: Ensure that scripted content is accessible. The <noscript> content will be presented to very few users, but must be accessible if used.

This prohibition stands in contrast with the WCAG guidelines (vide: noscript test). This is a very useful warning, but not an error.

Freeze

Wave processing normally kicks in after the page has finished loading, which for large pages may take a while. However, Wave can "freeze" on some simple documents (never finish processing them):

<form>
<input type="radio" name="ownerDocument">
</form>

The above snippet is enough to freeze Wave in an otherwise minimal document. Wave has problems with unqouted values and empty forms, or form elements, and if there is no freeze, then Wave may generate the report, but fail to populate it. A sample crasher testcase is enclosed.

Summary

Major issues with Wave are:

The benefits of using Wave are: