--- title: Why Automated Accessibility Testing Isn’t Enough url: "https://b13.com/knowledge/why-automated-accessibility-testing-isnt-enough" description: "Automated testing only catches around 20–30% of WCAG issues. Learn why manual testing is essential for true accessibility and an inclusive user experience!" image: "https://b13.com/fileadmin/_processed_/a/8/csm_ManualTesting_Headerbild_90cf656100.png" date: 2025-02-25 modified: 2026-08-19 lastUpdated: 2026-08-19 --- # Why Automated Accessibility Testing Isn’t Enough [ Accessibility ](https://b13.com/knowledge/accessibility) [ SEO ](https://b13.com/knowledge/seo) Why Automated Accessibility Testing Isn’t Enough ================================================== ![](https://b13.com/fileadmin/_processed_/9/7/csm_marco_206efb483a.jpg)Marco Heine 28 February 2025 [ RSS Feed ](https://b13.com/rss.xml) ![Illustration of a manual and automated website testing process. A human hand holds a magnifying glass, highlighting various information symbols, while a robotic hand also uses a magnifying glass. In the background, gears and a technical grid pattern emphasize the technological aspect of the topic. The image symbolizes the contrast between manual and automated accessibility testing.](https://b13.com/fileadmin/_processed_/a/8/csm_ManualTesting_Headerbild_0c26b71bbf.webp) When we do accessibility audits at b13 our main focus is on **manual testing**. But we also have incorporated **automated testing** tools, like [Axe](https://www.deque.com/axe/ "https://www.deque.com/axe/") or [Wave](https://wave.webaim.org/ "https://wave.webaim.org/") into our testing workflow. These tools are great. They help us to find common failures of the WCAG success criteria quickly and get a feel for the Accessibility of the page. Automated tools for accessibility testing ------------------------------------------- Axe or Wave are great in finding certain issues. They tell you about: - empty links and buttons, - missing alt-text, - if elements are not focusable, - if form labels are missing, - if color contrast is not enough. In short: they find the most **common issues** almost 96% of all websites have, according to the [WebAIM Million report](https://webaim.org/projects/million/ "https://webaim.org/projects/million/"). If time and budget are valid concerns for your accessibility testing and you want to do something quickly about your accessibility issues, making leverage of automated testing tools is a way to go. So is that enough to fix all the accessibility issues you have on your website? If you use these automated testing tools and fix the most common issues that’s a great way to handle this and you won’t need to test manually anymore, right? Sorry to destroy your hope at this point, but no. **Automated testing tools only catch between 20 and 30%** of all WCAG failures, depending on who you ask. Gen Herres [summarized a study by Deque](https://easya11yguide.com/tips/how-much-can-automated-accessibility-tools-do/ "https://easya11yguide.com/tips/how-much-can-automated-accessibility-tools-do/"). Deque is the developer company behind Axe and the study prevails that Axe is able to **test 15 out of the 50 WCAG success criteria**, or about 30%. Melwyn Joseph [summarizes several studies](https://www.webyes.com/blogs/can-automated-tools-make-website-fully-wcag-compliant/ "https://www.webyes.com/blogs/can-automated-tools-make-website-fully-wcag-compliant/") and draws a similar conclusion and states: > Automated tools can’t detect all WCAG violations, making manual testing essential to address the nuanced issues they miss. To achieve full WCAG compliance, a combination of automated scans and manual checks is necessary“ Melwyn Joseph Adrian Roselli [conducted a study himself](https://adrianroselli.com/2023/01/comparing-manual-and-free-automated-wcag-reviews.html "https://adrianroselli.com/2023/01/comparing-manual-and-free-automated-wcag-reviews.html") where he tested all applicable success criteria manually and used automated tools. He comes out with even **lower numbers**, but also shares a valuable lesson: > This does not mean you should avoid automated tools. Like any tool, it means they have a place in your toolbox. Adrian Roselli What automated tools can’t test properly ------------------------------------------ Why can’t we get higher numbers in the range, automated tools are able to test for? The big problem with tools like Axe or Wave are: **they lack the context of a page and the user**. They can only test the code of the current page, not the changes in viewport, orientation or size. Here is an exemplary list of WCAG success criteria, automated tools are not able to test properly. ### 1.1.1 Non-text Content - Automated tools are great in telling you if images are missing alt text, as stated in the beginning. But the thing about alt text is, it’s **not a simple boolean comparison**: having an alt text or not. It’s more complex. Alt text needs to incorporate **context and emotion**. - Automated tools can’t figure out if the present alt text is meaningful. They can’t figure it if an image is decorative and therefore empty alt text is sufficient. ### 1.2.2 Captions - Captions are important for deaf people if they watch a video. Automated tools are not able to determine if a video has captions and if this captions are actually meaning- and helpful to deaf people. ### 1.3.1 Info and Relationships - This success criterion is about the proper usage of HTML Elements. It’s about characterizing headings as headings, tables as tables, paragraphs as paragraphs, lists as lists, block quotes as block quotes and so on. Automated tools are not able to detect if specific information should rather be presented as a table or as a text block. Or if a text block should rather be shown as an ordered list. Humans can, because the can evaluate the context of the content. ### 1.4.3 Contrast (Minimum) - Automated tools are able to determine if the color contrast on an element is insufficient. - But they still have huge problems to figure this out for semi-transparent and gradient backgrounds. ### 2.1.1 Keyboard - Automated tools may be able to tell if elements are focusable (2.4.7) and if the focus is visible to users. - But they can’t tell if a whole website is fully usable with a Keyboard. ### 2.4.2 Page Titled - Automated tools are able to test if a website has a page title. - But this success criterion is not only about having one, it’s about having a meaningful title. ### 2.4.3 Focus Order - The way users with a screen reader read content can differ from how sighted people do it. Automated tools cannot detect the correct focus order because they lack the context. ### 3.2.1 On Focus, 3.2.2 On Input - These success criteria are about unannounced context switching. If you focus an element, or change the setting of a component, the context should not changed unless clearly advised about the behavior before. This can lead to huge cognitive load and confusion for users. Something automated tools can’t test at all. ### 3.3.3 Error Suggestion - If a user fills out a form and there’s an error message, it should be clear to the user and understandable in a way that they are able to correct their input. How are automated tools able to test this? It’s again about context and these tools can’t check if error messages are helpful or not. Only humans can. ### 4.1.2 Name, Role, Value - Automated tools can check if components have WAI-ARIA roles and properties. This makes it easier for screen reader users to understand them. - But they are not able to fully simulate how an actual screen reader is used. So while adding the correct WAI-ARIA Role to a custom element satisfies the Axe report, it’s still possible that Screen reader users cannot operate this part of the website. `This is just a small sample of the success critera automated tools can’t test for. Unfortunately the list could go on and on. The important thing that remains is, if something is accessible when it’s meaningful or needs the correct context to test it, automated tools are not suited for that task, only manual testing is. ` What about Accessibility Overlays? ------------------------------------ If automated tools are not enough for accessibility testing, maybe accessibility overlays can help. You’ve probably heard about them, a lot of websites use them nowadays. There’s often a floating button at the bottom of the page, which by click opens the overlay. An accessibility overlay gives the users options to **fix accessibility issues for themselves**. For example they give users the possibility to change the page contrast, enlarge the text size, generate alt text and so on. Overlays also apply some fixes automatically, based on AI. The bold claim here is: if you use an overlay, you don’t need to test for accessibility. Common overlays are **Eye-able** or **accessiBee**. While this seems to be an easy win for your accessibility testing, overlays are not the solution to the problem. As Daniela Kubesch states it: “They are a band-aid on a broken bone”. She wrote her master thesis about this topic: [The Impact of Web Accessibility Overlays on the Usability and User Experience for People with Permanent Visual Impairments](https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1877583&dswid=-9561 "https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1877583&dswid=-9561") and is an expert in this field. In her scientific work, she found out that overlays can detect WCAG failures and resolve them. But resolving them often leads to **new errors**. She also found out that there are a lot of issues overlays can’t fix. And while overlays fix errors for some users, they create new ones for others. Her resumee is clear: > Don’t use accessibility overlays in their current state. They are no solution. Daniela Kubesch Another big pain point about Accessibility Overlays is, that they are a redundant solution. They offer problem solving strategies, for problems which are **already solved**, either by browsers or by the users. Browsers offer a huge variaty to enhance a website, they often can even do way more to make a website accessible. One of the simplest example is, that every browser lets the user zoom in and out. On the other hand users already have software or plugins installed, which helps them use a website in a way they need it, which makes overlays unnecessary. In some cases overlays even interfere with these user settings, overwrite them and make a website less accessible. Finally some Overlays even claim to be 100% WCAG compliant, which in comparison no automated tool does. These already led to lawsuits, the tool accessiBee is required [to pay 1 million dollars](https://www.lflegal.com/2025/01/ftc-accessibe-million-dollar-fine/ "https://www.lflegal.com/2025/01/ftc-accessibe-million-dollar-fine/") for these false claims. Conclusion ------------ Automated testing is a valuable tool—it’s fast, cost-effective, and helps detect common accessibility issues. However, fixing accessibility is not a quick fix. As shown above, automated tests cannot catch every **WCAG** failure, nor can they ensure a website is fully accessible. That’s why an effective **accessibility testing strategy** combines both **automated and manual testing**. These two approaches work best together: - **Automated testing alone** is quick but misses many critical issues. - **Manual testing alone** is thorough but costly and time-intensive. By combining both methods, **a more accurate testing result** can be achieved andthe **most** WCAG violations can be identified. But we need to remember: **WCAG compliance is just the baseline**. Meeting the guidelines does not automatically mean a website is fully accessible for all users. Our approach -------------- This is why we incorporate **both** approaches. If you’re interested in learning more about our accessibility testing solutions, check out what we have to offer! [ More about accessibility ](https://b13.com/blog/accessibility) #### Sources - [Can Automated Tools Make Your Website Fully WCAG Compliant?](https://www.webyes.com/blogs/can-automated-tools-make-website-fully-wcag-compliant "https://www.webyes.com/blogs/can-automated-tools-make-website-fully-wcag-compliant") - by Melwyn Joseph - [Automated and manual accessibility testing work best together](https://blog.pope.tech/2025/01/09/automated-and-manual-accessibility-testing-work-best-together/ "https://blog.pope.tech/2025/01/09/automated-and-manual-accessibility-testing-work-best-together/") - by Whitney Lewis - [How much can automated accessibility tools do?](https://easya11yguide.com/tips/how-much-can-automated-accessibility-tools-do/ "https://easya11yguide.com/tips/how-much-can-automated-accessibility-tools-do/") - by Gen Herres - [Accessibility-Overlays wissenschaftlich beleuchtet—technica11y mit Daniela Kubesch](https://www.youtube.com/live/Atc5v64gqdE "https://www.youtube.com/live/Atc5v64gqdE") - [Comparing Manual and Free Automated WCAG Reviews](https://adrianroselli.com/2023/01/comparing-manual-and-free-automated-wcag-reviews.html "https://adrianroselli.com/2023/01/comparing-manual-and-free-automated-wcag-reviews.html") - by Adrian Roselli - [Fixing digital accessibility issues with the press of a button: too good to be true?](https://www.elevenways.be/en/articles/overlays-digital-accessibility-fix-with-a-press-of-a-button "https://www.elevenways.be/en/articles/overlays-digital-accessibility-fix-with-a-press-of-a-button") - by Roel Van Gils - [4 Reasons An Overlay Widget Will Not Solve Your Accessibility Woes](https://litzdigital.com/blog/4-reasons-an-overlay-widget-will-not-solve-your-accessibility-woes/ "https://litzdigital.com/blog/4-reasons-an-overlay-widget-will-not-solve-your-accessibility-woes/") - by Matt Litzinger - [The WebAIM Million](https://webaim.org/projects/million/ "https://webaim.org/projects/million/") - ### Written by: ![Bild von Marco Heine](https://b13.com/fileadmin/_processed_/9/7/csm_marco_de08856ad5.webp) Marco Heine Development [ more from Marco Heine ](https://b13.com/team/marco-heine) Related Articles ------------------ - ![A cartoon character resembling a shield gives a thumbs up in front of a computer interface with various menu options.](https://b13.com/fileadmin/_processed_/e/6/csm_Header_cda1f80173.webp) ### Meet T3ppy: A New Era for Editorial Work in TYPO3 01 April 2026 | Florian “Flix” Keitgen T3ppy and the AiM extension bring AI directly into the TYPO3 backend—enhancing SEO, content quality, and workflows with centralized control. [ Read more: Meet T3ppy: A New Era for Editorial Work in TYPO3 ](https://b13.com/knowledge/meet-t3ppy-a-new-era-for-editorial-work-in-typo3) - ![A laptop is connected to multiple file folders and browser windows, illustrating data management and organization concepts.](https://b13.com/fileadmin/_processed_/8/9/csm_SubdirectoriesSubdomains_Sharing_be4a05761c.webp) ### Subdirectories vs. Subdomains: Which is Better for SEO? 23 October 2025 | David Steeb Subdirectories, Subdomains, and Better SEO Rankings. The answer to which one should I use? is always it depends. Subdomains and subdirectories interact differently with SEO… [ Read more: Subdirectories vs. Subdomains: Which is Better for SEO? ](https://b13.com/knowledge/subdirectories-vs-subdomains) - ![Two robotic arms are positioned around a magnifying glass, focusing on the letter "g" in a stylized font. The background features a grid pattern with repeated "Ag" symbols.](https://b13.com/fileadmin/_processed_/1/8/csm_FontGuidelines_Headerbild_6f3f5a0baa.webp) ### Designing with Type: A Guide to UI Font Size Guidelines 03 June 2025 | David Steeb Discover the importance of UI font size guidelines for creating accessible and visually appealing UI designs. Learn the best practices for typography in UI, including consistency,… [ Read more: Designing with Type: A Guide to UI Font Size Guidelines ](https://b13.com/knowledge/designing-with-type-a-guide-to-ui-font-size-guidelines) - ![A stylized pink brain is depicted with a search bar overlay, featuring icons for voice and camera search. The background includes yellow gears, suggesting themes of knowledge and technology.](https://b13.com/fileadmin/_processed_/b/c/csm_GoogleSGE_Headerbild_5cf7414daa.webp) ### Maximizing Website SEO with Structured Data in Google’s Search Generative Experience 28 March 2025 | David Steeb Explore structured data’s impact on SEO in Google’s SGE. Learn to boost your website’s visibility with practical TYPO3 implementation tips. [ Read more: Maximizing Website SEO with Structured Data in Google’s Search Generative Experience ](https://b13.com/knowledge/maximizing-website-seo-structured-data-googles-search-generative-experience) - ![A laptop with a yellow padlock on the screen, surrounded by red icons representing an eye, ear, microphone, hand, and brain, symbolizing security and privacy concerns in digital environments.](https://b13.com/fileadmin/_processed_/d/a/csm_AccessibilityIsKey_Headerbild_a91f1c635a.webp) ### Web Accessibility: Designing Beyond Compliance to Create Inclusive Digital Experiences for All Users 25 March 2025 | Marco Heine Complacency, lack of time, and insufficient knowledge often push accessibility to the back burner. But small changes made today can make your website more inclusive for everyone.… [ Read more: Web Accessibility: Designing Beyond Compliance to Create Inclusive Digital Experiences for All Users ](https://b13.com/knowledge/web-accessibility-designing-beyond-compliance-to-create-inclusive-digital-experiences-for-all-users) - ![Illustration of movers moving boxes](https://b13.com/fileadmin/_processed_/8/1/csm_MovingPagesRedirects_Headerbild_1ed3894ad6.webp) ### Navigating Page Movements and Redirects in TYPO3: A Comprehensive Guide 10 March 2025 | David Steeb Explore our comprehensive guide on managing redirects and page movements in TYPO3. Learn how to maintain SEO and enhance user experience with seamless navigation adjustment. [ Read more: Navigating Page Movements and Redirects in TYPO3: A Comprehensive Guide ](https://b13.com/knowledge/navigating-page-movements-and-redirects-in-typo3-a-comprehensive-guide) - ![](https://b13.com/fileadmin/_processed_/a/5/csm_CodeAccessibility_Headerbild_d1ed666070.webp) ### Guide to coding for web accessibility standards 07 March 2025 | David Steeb Writing code is fundamentally an exercise in crafting a strong foundation, one that should incorporate inclusivity as much as scalability. Compliance standards are a great place to… [ Read more: Guide to coding for web accessibility standards ](https://b13.com/knowledge/guide-to-coding-for-web-accessibility-standards) - ![Two individuals with different hair colors hold framed pictures, each depicting a smaller version of themselves. The background features radial lines in warm colors, enhancing the focus on the subjects.](https://b13.com/fileadmin/_processed_/3/5/csm_AvoidingDuplicateContent_Headerbild_6887956ad1.webp) ### Addressing Duplicate Content in TYPO3 11 December 2024 | Benni Mack Discover how to efficiently tackle duplicate content in TYPO3 with our expert guide. Learn about unique strategies and essential tools to boost your site’s SEO today. [ Read more: Addressing Duplicate Content in TYPO3 ](https://b13.com/knowledge/avoiding-duplicate-content-on-your-websites) - ![A pathway leads through stylized clouds, flanked by a magnifying glass and gears, with icons representing text and documents in the background, suggesting themes of exploration and technology.](https://b13.com/fileadmin/_processed_/8/0/csm_accessibility-guide_0acd1932f3.webp) ### A Guide to Accessibility Considerations in UX Design 12 April 2024 | Franzi Töpler People with disabilities constitute one of the most significant segments of online users, but it is an often-overlooked population in the digital world. Increasing accessibility to… [ Read more: A Guide to Accessibility Considerations in UX Design ](https://b13.com/knowledge/a-guide-to-accessibility-considerations-in-ux-design) - ![Two hands are adjusting colored tokens labeled "a/a" on a digital interface, with a search bar prominently displayed. The background features a purple color with subtle patterns.](https://b13.com/fileadmin/_processed_/0/b/csm_URLtrailingSlash_Headerbild_52019d4501.webp) ### To Slash or not to Slash? The Significance of Trailing Slashes in URLs 22 March 2024 | Benni Mack Discover the significance of trailing slashes in URLs and how they impact SEO and user experience. Learn best practices and TYPO3 solutions with b13’s expert guide. [ Read more: To Slash or not to Slash? The Significance of Trailing Slashes in URLs ](https://b13.com/knowledge/to-slash-or-not-to-slash-the-significance-of-trailing-slashes-in-urls)