Accessibility Testing Methods – User testing vs automated testing
Accessibility testing is crucial to make sure that a website is as usable as possible for everyone. Regardless of the method you use, it’s best to plan testing into a project from the start. Rather than waiting to test later, continue to test throughout the project. Leaving it until the end could mean spending twice as long fixing an issue that has been highlighted.
There are two types of accessibility testing to consider; user testing and automated testing.
Automated testing is achieved using an online or downloadable piece of software. It scans through the code of the website and determines how technically-sound the website is. Usually assessing it’s conformance with the standards set out in the W3C’s Web Content Accessibility Guidelines. Using pre-scripted tests, their function is to compare the actual results with the expected results of an accessible website. It is able to spot errors that are buried in the code.
Automated testing is great for when you have changed one aspect of a site and want to test that it functions correctly. It is also great for removing the risk of human error. Testers can get tester-fatigue if they have seen the site before and know how it works. They become able to navigate a website better than a new user would be able to due to experience. This can create false positive in tests.
This is where human testers use a website or digital service and report on how they found it. At HeX, we use Shaw Trust Accessibility Service’s genuine disabled testers to go through the wireframes, initial build and the final build of all of our websites. This ensures that everyone will be able to access it using assistive technology. Testing methods include screen readers, keyboard tabbing, voice commands, and more.
The best part of this method is getting the human aspect of the testing. The real-life feedback that is received can be very valuable. A piece of software is great for spotting technical errors. However, a person with a disability is the best person to inform you as to where a problem lies in the user interface and design.
Specific user-journey testing is also available with this method. This means a tester can run through a specific scenario, for example:
- Looking for and buying a product using the website.
- Navigating to the contact page and filling out a form.
- Creating an account on a digital service.
- Booking a table, flight or hotel room.
- Selecting a service and finding out more about it.
User journey testing allows for effective assessment of the interface, the experience and the technicalities.
Which accessibility testing method is best?
One gives you the freedom and the time to carry out other tasks, whilst the other provides real-life results. One method of testing isn’t better than another. When used in tandem, they provide the recipe for a better and more usable website.
Despite the broad coverage of automated testing, disabled user testing should never be underestimated. Automated programmes can highlight things like:
- Missing Alt tags on images.
- No link descriptions.
- Inconsistent heading structure.
However, they can never accurately tell you things a user tester will be able to. Such as:
- A website’s colour contrasts being too low for a low vision user.
- The user interface not being logical
- The content structure being complex and difficult to scan for a user with processing difficulties.
At HeX, we use three different testing methods to be confident of a good result. We start with an automated test to run through errors on the site. We then perform a technical manual review where one of our expert team members run through the site checking the code. Finally, our Shaw Trust user tester runs through the website fully. If you’d like to find out more about our accessible websites and user testing, get in touch with us