Accessibility Testing Methods – User testing vs automated testing

In terms of accessibility, testing is crucial to make sure that a website is as usable as possible for everyone. Regardless of the method you use, it’s best to plan testing into a project from the start, and continue it throughout, rather than waiting to do it later and end up spending twice as long fixing an issue that testing has highlighted.

There are two types of accessibility testing to consider; user testing and automated testing.

Automated testing 

This is achieved using an online or downloadable piece of software that scans through the code of the website and determines how technically-sound the website is, based on the code and the standards set out in the W3C’s Web Content Accessibility Guidelines. Using pre-scripted tests, their function is to compare the actual results with the expected results of an accessible website. In doing so, it can spot errors that are buried in the code.

Automated testing is great for when you have changed just one aspect of a site and want to test that this part functions correctly. It is also great for eliminating the risk of human error, as testers can get tester-fatigue if they have seen the site before and know how it works, as they can navigate it better than a new user would be able to due to experience – creating false positive in tests.

User testing 

This is where human testers use a website or digital service and report back on how they found it. Here at HeX, we use Shaw Trust Accessibility Service’s genuine disabled testers to go through the wireframes, initial build and the final build of all of our websites to ensure that everyone will be able to access it using screen readers, keyboard tabbing, voice commands, etc.

The most advantageous part of this method is getting the human element of the testing; the real-life feedback that is received. A piece of software is great for spotting technical errors, but a person with a disability is the best person to inform you as to where a problem lies in the user interface and design.

Specific user-journey testing is also available with this method, meaning a tester can run through a specific scenario for example: looking for and buying a product using the website or navigating to the contact page and filling out a form. This allows for the interface, the experience and the technicalities to be tested effectively.

Which testing method is best? 

One gives you the freedom and the time to carry out other tasks, whilst the other provides real-life results. But truthfully, one method of testing isn’t better than another and, when used in tandem, they provide the recipe for a better and more usable website.

Despite the broad coverage of automated testing, user testing should never be underestimated in relation to accessibility. As much as automated programmes work in a sense that they highlight things like missing Alt tags on images, no link description, etc, they can never accurately tell you that a website’s colour contrasts are too low for a low vision user.

At HeX, we use three different testing methods to be confident of a good result; an automated test to run through errors on the site, a technical manual review where one of our expert team members run through the site checking the code and our Shaw Trust user testers who fully run through the website. If you’d like to find out more about our accessible websites and user testing, get in touch with us <link to new contact us pages>