Awesome Automation Strategies for Accessibility Testing with Crystal Preston-Watson

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way o the test Illinovation Testing Podcast Guiding testers with our nation awesomeness from ancient realms to modern days, they lead the way. Oh, the testicle of nature Testing Podcast.

Accessibility is more than just a checkbox. It's become a civil right and a human need. And today we'll explore how proper automation can elevate your accessibility testing and take it to a whole nother level. In today's episode, Crystal Preston Watson, a Senior Digital Accessibility Analyst at Salesforce, joins us and this session was taken from Our Automation Guild 2022 online event where Crystal took us on an insightful journey addressing high level questions and critical issues related to accessibility testing, specifically in automation. To learn more about different types of accessibility testing, you definitely want to listen in. And speaking of accessibility testing, I also want to share an automated accessibility tool from this episode sponsor, BrowserStack as you know and as we go over in this session, accessibility testing is huge. Nowadays to help include all your users of your software, Accessibility testing is also becoming more and more important to help you comply with all the new legal standards that are coming out like the Americans with Disability act or the web Content Accessibility Guidelines, which helps reduce your risk of lawsuits via software. However, ensuring your website is accessible can be a tough task. Juggling multiple tools for accessibility testing is time consuming, inefficient and unscalable, often requiring QA or devs to rerun automated scans across various elements and states just to identify any potential issues. And having to collect all this data from all these tests is very cumbersome. And gaining access to devices with screen readers enabled can also be a huge burden. That's why I want to introduce BrowserStacks accessibility testing. With BrowserStack, you can seamlessly test, report and monitor the accessibility health of your web applications, all from a single platform, identifying both basic and complex accessibility issues at lightning speed. For example, they have a workflow scanner that you can use to scan user workflows in one go, pinpointing issues like missing alt text or insufficient color contrast without the hassle of multiple scans and for more intricate problems. Their assisted tests guide you through identifying advanced accessibility concerns with simple auto generated questions. Plus, with instant access to real screen readers on real devices like Voiceover, NVDA and Talkback, you can experience your site just as if your users with disabilities do, ensuring an exclusive digital experience Browser Stack's accessibility testing empowers you to create a website that is not only compliant, but also welcomes and delights all your users. So head on over to testskill Me Estac for more details or click on the link down below to learn more.

Hello everyone and welcome to Automation and Accessibility Mutual, Not Exclusive. I'm Crystal Preston Watson, Senior Digital Accessibility Analyst at Salesforce. I want to tell you some of the things you won't find in this session. This is not an accessibility 101 a foundation in accessibility is necessary for accessibility testing, but today I focus on higher level questions and issues teams need to consider before beginning accessibility automation. That also means I won't be coding or demoing tools. However, I will give you links to items and resources that I discussed today. If you feel that you don't have a solid understanding of the basics of digital accessibility, I encourage you to come back to this video and after doing a bit more learning, I'll be here waiting. Well, not me. This is the digital version of me. I don't live in this video let's get started by going over the three types of accessibility testing. Manual accessibility testing is when teams test applications manually for accessibility issues that may cause problems for users with disabilities. This testing is conducted using browser and plugin tools like Wave, Lighthouse or Axe. Also assistive technology like screen readers and switches. Currently on the screen I have a slide that has several logos for automated tools like the ones I just mentioned briefly and Screen readers, Jaws and ndva. Also an image of two Jelly Bean switch devices. Automation Accessibility testing is my focus today. You might be wondering why I listed some semi automated tools under manual instead of automation. Well, while many of those tools can be fully automated, their browser extensions are also widely utilized in manual testing. This slide showcases three Axe applitools and Expresso. Deque's open source Axe Core Library powers a considerable portion of free and paid accessibility tools and automation frameworks available. Currently, applitools Contrast Advisor validates minimum color contrast ratios for text and graphics and can be integrated into existing automation frameworks. Expresso is an Android UI testing framework that has integrated accessibility checks. User or user acceptance testing gets overlooked by companies when it comes to accessibility. Testing with people with disabilities is necessary to understand how your application is or site performs in the real world and finds issues and bugs beyond accessibility conformance. This testing needs to be done with more than one person or a group with a singular disability within your target audience. Also, make sure that it's paid. People's time is worth money, and given that this testing can find very costly bugs, it's worth more than free software. Part 1 Are you ready to automate I regularly ask those who tell me that their testing team wants to automate their accessibility testing if they are ready. I don't ask this expecting an answer, but to get them to reflect on their current accessibility initiative. There's a vast difference between wanting to automate and being ready and able to automate. Accessibility testing teams tend to focus on technical readiness, but with accessibility, two significant considerations must be addressed before a framework is selected or code written to make sure automation is successful. Let's take a look at those now Experience with Accessibility for many testing teams, accessibility is not an intentional undertaking. As my grandfather liked to say, they fall ass backward into it. A demand by company leadership in response to customers concerns, complaints or even ADA lawsuits, there's scrambling to inventory accessibility in the product. There's no time for preparation or ramp up for training or education. The issue compounds with the realization that accessibility has a needed prominence. On manual testing. There's pressure to automate ASAP to make testing quicker or more effective. You can't see me on screen, but I did make air quotes when saying effective. Don't get me wrong, I am on board with test automation. When done right, it increases productivity, quality and requires a smaller time investment. But the magic words are when done right. That doesn't happen with teams that do not have a solid foundation in digital accessibility that struggle to interpret and test against the web content accessibility guidelines success criteria, let alone understand how disabled people use assistive technology devices to to navigate their products. In a presentation called after the Audit, I suggest actions that teams can take to develop accessibility skills and experience to integrate into their testing processes. Here are a few of those actions that your team can undertake. 1. Establish a baseline of knowledge it's essential to create a shared language and understanding when discussing accessibility and disability. 2. Communicate why and how to go through testing. Encouraging teams to learn through courses, attend conferences and providing team training on core fundamentals. 2. Play to people's strengths, that is finding people for the tasks that fit their experience and skill. If someone is apprehensive about testing using screen readers, well, you want them to have the skill they can work on keyboard testing and teaching that to others while they scale up with other assistive technology. 3. Shifting accessibility to the left it's not unusual for testing teams to become the sole champion of accessibility within a company. For accessibility to be a genuine priority and done with quality, it has to be the concern of the whole organization. This means promoting and informing stakeholders and leadership that accessibility cannot be tested away and needs to be a company wide priority. This last action leads me to the second consideration for being automation ready. Capacity and Support As I previously mentioned, testing often becomes the sole champion of accessibility within a company. Unfortunately, it is considered an afterthought or a nice to have instead of a necessity. This is reminiscent of some opinions about QA and testing, especially when it comes to manual and exploratory testing. I was at QA for nine years and I'm all too familiar with the siloing and devaluing of testing a qa. It's not shocking to see both viewed the same and why testing teams become the de facto owner of accessibility. But given the individuals, situations and circumstances that are affected by inaccessibility in applications and sites, this is not something that can be left up to Tech Deck or a one off Crunch Sprint. Here are three items you should evaluate to see if you have the capacity and support to begin automation. Everyone is on board on board and taking up appropriate roles when it comes to accessibility. This is why it needs to be a company wide priority with backing and training and education for everyone. If the company accessibility initiative boils down to testing, it's a failed initiative and no automation will make up for that ongoing manual testing. I'll get into the particulars later, but manual testing cannot be automated away from accessibility. Automation and manual here are genuinely mutual. If you have the headcount for both, that's great, but if you don't, then manual is the one that needs to continue Keeping Regression in mind Regression and accessibility testing can be problematic mainly when bugs and issues are found due to audits. As third party vendors usually conduct them. It's important to make sure bugs fixed due to audits or other avenues make it into your regression testing. It's not unusual to find the same bugs with every new audit because regression test cases never make it into the in house testing team's regression suites. If you aren't frozen in existential dread after considering what goes into being automation ready, congratulations. You probably are ready. But don't break out that leetcode just yet. What should you and should not automate A point that needs to be driven home regarding accessibility automation is that it will not find everything. Most accessibility experts agree that between 20 and 40% of accessibility issues are found with automation testing. Combining manual and automation is a balanced and holistic approach to accessibility testing. Teams might be reluctant to add manual testing for accessibility, especially if they are moving to automation in other areas, but manual testing is unavoidable for catching accessibility issues that automation cannot detect. Let's look at areas where you can enhance your accessibility Testing with automation and the best places for Manual Unit and Integration Unit integration tests are perfect for accessibility automation. This is where you can detect low lying bugs that take up vital time during manual testing and watch for regression. This also allows you to incorporate accessibility into your continuous integration builds. Some of the things you would want to focus on in unit integration would be and I want to credit Marcy Sultan for this list because she's where I got this from. Unit Component Specific Behavior interaction and focus APIs, ARIA states integration Real world browser testing, Document and page level rules and Color contrast UI I don't usually encourage UI automation for accessibility. UI tests are known to be flaky due to frequent adjustments and changes. My personal experience and opinion that time can be spent creating unit integration tasks or best for a person to conduct them manually. But if you want to have UI automation, these are the scenarios that tend to work best. Multi page workflows and form flows Manual an automation framework cannot make subjective interpretations whether something is or isn't accessible. Also, when it comes to assistive technology devices and software, it's either impossible or not practical to automate their operations. The heart of accessibility is the human experience. This is why manual testing is priceless and a necessity for accessibility. Here are some of the areas that you would want to focus on. Keyboard interactions Assistive Technology Devices Caption Transcript Alt Text Quality and error identification part 3 so are you ready to automate? At the beginning of this video I said that there was a difference between wanting to automate and being ready to automate your accessibility testing. So I ask you, given all that I've discussed, your team's experience with accessibility, your capacity, support from within your company, are you ready to automate accessibility? Again, I'm not expecting an answer here. This is the digital version of me. I want to thank the Automation Guild for giving me the opportunity to speak to you today and if you take anything away from this video, let it be this. Accessibility is more than design, code and testing. It's about humans and their experiences. Accessibility is also a human and civil right. If you have any questions about information I presented today or you would like links to some of the resources and items that I've talked about, you can reach me via Twitter copicengineer, LinkedIn. Just type in Crystal Preston Watson and my website crystalprestonwatson.com and I will see you next time. The real version of me, the digital version One of us will see you again hopefully.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a515 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation and testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Lighting tester showing the secrets to behold.