Accessibility Testing Using Cypress with Pragati Sharma

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Progati all about accessibility testing using Cypress. Brigatti is a senior quality consultant working at ThoughtWorks. She's an expert in all kinds of automation for Web Web Services, REST, REST, APIs, GraphQL and non functional testing like performance and accessibility, which is what we'll be talking all about today. This session was actually taken from last year's Automation Guild to give you a flavor of the types of actual automation you discover at our events. So it's going to be the audio version of the session. If you actually want a video recording of this to actually see what Bugatti is talking about, all you need to do is go to the show notes@testguild.com forward/a376 actually watch it there as well. I'm also doing a little experiment. I know sometimes people learn by viewing. I know some people learn by watching, other people learn by doing, but some people actually learn by listening. So it's just an experiment to see if I can take a a session that I had in Automation Guild and just gave you the audio version if you got value from it. So I'd be really interested to know if this is something you want to hear more about in upcoming episodes to have more like audio tutorials rather than just an interview. So let me know and you could drop me a line anytime@joe USGuild.com and by the way, registration for Automation Guild 2022 is almost ready to learn more about all the types of topics you need to know to accelerate your career and create less flaky automation in 2022. Head on over to automationguild.com and register for the 6th annual 5 day online event taking place February 7th to the 11th. So you don't want to miss this episode. You don't want to miss this event. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment. For every framework, browser, OS, mobile device and API, get a free trial. Visit testguild.comsourcelabs and click on the exclusive Sponsors section to try it for free today. Check it out.

Hi everyone, thanks for joining in for this session. So in this session we are going to talk all about accessibility, but before getting started Let me just give you a brief introduction about myself. Hi everybody, I'm Prabhati Sharma and I hail from Delhi, which is the capital of India. I've been working in the corporate industry for the past 4.5 years. Now a little more than that as a quality engineer, quality consultant. Currently I'm working with ThoughtWorks as a senior quality consultant. So in my entire tenure I have had the chance and the opportunity to work on different facets and different types of testing and automation. So I've done web automation, I have done Web Services automation, BE REST, APIs, GraphQL, and then non functional automation as well. That comprises of performance and off fleet accessibility.

So I'll be very honest. So when I started working on accessibility, I clearly understood that it is an underdog because we do not talk much about it, but it is so very important for us as engineers, as a development team for the company, for the organization and for the clients and customers. So I guess it's the right time for us to pay heed and attention to accessibility testing as well. And this is what this session is all about.

So let me first get you well versed with what is accessibility and why do we need it? So starting off with the whys, whenever we talk about something being accessible, it simply means that it is easily understandable and easily obtainable. So if I say that I want to make a platform which is accessible, it would simply mean that everybody around the world should be able to access it with or without disabilities. Also, 20% of the world population is disabled. So that makes a huge part of the entire user base. And we definitely need to be cognizant about their needs as well. Let me just give you a very relatable example of how accessibility is not only targeted to a specific group, but on a holistic level. It increases the user experience by multifold. So you would have seen wherever we have staircases, we have ramp adjacent to that. So the primary motive of that ramp is for people who travel via wheelchairs. But whenever you go to subways, you would see that more than 50% of the population is using ramp instead of the staircases for going up and going down. So why is that happening? So the ramp, which was actually made to solve an accessibility pain point, essentially increased the user experience and people were more happier using the ramp instead of the stairs, because using stairs is a lot of effort for them.

So in a similar fashion, let's jump onto the other aspect of it, which is why do we need accessibility?

Again, for this I would give you a real World example. So there is a very famous streaming company. I would not be taking the name, but in 2016 a heavy lawsuit was actually filed against this company because it did not have the closed caption. So this lawsuit was filed by the national association of Deaf because their community and the people who could not hear were actually not able to absorb the content on that streaming website.

And because of this, the company had to actually incur a cost of $750,000 and they were made to sign a contract with set within two years. They need to make 100% of the content with closed captioning by incorporating closed captioning to their content. So in this example, Title 3 of ADA, which is Americans with Disability act was violated because of which this entire lawsuit happened at the first place. So this was something which is very specific to America. But then different nations have different federal laws. Supposedly UK it has its own federal law, it goes by the name of Discrimination Disability Act. Then in India we have another law which is Rights of Persons with Disability Act. So now wherever you are and wherever you're deploying your product, you need to make sure that you are actually abiding to the laws of that particular nationality. And this is what makes accessibility all the more important for us to abide to. As quality analysts, we need to make sure that our products are accessible by one and all. And at the same time we need to make sure that we are able to give a faster feedback for any kind of accessibility violations so that the team is able to handle that in a timely fashion. And at the end, it's very important for us to automate this entire process so that the overall regression becomes way more easier for us.

So let's just type right in.

Let's try and understand what the problem statement is. So now what we have in hand is a web application and we need to perform functional automation for that. And we need to club that with some accessibility automation for this we would be using Cypress as our web automation framework. And Cypress actually provides us handful of advantages, like one of them being that it runs in the same loop as your application, unlike other testing tools which sit outside your browser, which means that you would definitely be encountering lot less synchronization issues while you use Cypress. Another thing to notice is that it provides you the native access to the DOM elements. So to all the document object models, like different objects, window objects, document objects, HTML divs, all of them would be easily accessible using Cypress. Also it shows you the hidden shadow elements or shadow DOM elements, they would also be accessible using Cypress.

Then Cypress can be actually thought of as a one stop solution for all your automation needs because it also has its own assertions. Now coming onto the other thing which is the accessibility engine called Axe. The beauty of Axe is that you can actually integrate that with n number of different testing tools, name it and you have it. So it can beautifully get integrated with Cypress, Selenium, Testcafe, rest assured. So it simply means that whatever automation suite you have in place just on the top of it, Axe as the accessibility engine. Also whatever modern day browsers we usually use for testing purposes like Chrome, Firefox, ie, Safari, so all of them can be tested using Axe. Now moving ahead. So the application that we would be using is a todo app. So this is basically an app which gives you the flexibility to create your to do list. So now if I want to say that sanitize your hands.

Wash your hands, that's my to do list. And as soon as you complete that, you can strike these things off. So now this is what we're going to automate today. Let me refresh this for this. Like I mentioned we would be using Axe. Let me give you a manual demo of what we are trying to achieve here. For that we would be needing.

A plugin called Axe Coconut. So this is a Chrome extension. You can simply go and add this. After adding this go to your web application that you want to test for accessibility issues.

And here go to Wax Coconut. Click on Analyze and you'll figure out the various accessibility vulnerabilities that your application has. Now this application has four. Let me walk you through few of them and then we can get started with the Hands on automation. So this one shows that there is a lack of sufficient color contrast. Now think of a person who has a poor vision for that person, this background and foreground seems to get merged and he or she is not able to read that out properly. So now this is an issue that is to be solved. For that you'll have to change the color of your foreground so that there is sufficient contrast.

So that it doesn't become incomprehensible. Now if you see that there are certain tags, so now this is a category of color which is getting violated. And these are different guidelines. So now this WCAG standards and these are like different versioning of standards as well. So now you'll have to go ahead and ask your product owners as to what have you complied to. And the testing that we do should be according to that because you have to prioritize your issues according to which standard are you complying with? Let me talk about some other issue. Now, this issue says that every element should have a lang attribute. So lang is the language attribute. And this is a category that is mentioned as well. Another thing that we should understand is why is this issue important? This issue is important because people who cannot see, who are visually impaired, so they would not be able to see it, but they would be able to hear. For that, they use tools called screen readers. So there are various screen readers. Like in Mac there is an inbuilt screen reader called Voiceover. Then there are other screen readers like nvda, Jaws. So you can actually make use of those screen readers and you can navigate through your application and figure out what every element says by hearing it out. Now, if you do not provide a language attribute, your screen reader would never come to know whether things written on this application are in English, German, French, what is the language. So it becomes all the more important for us to provide this attribute. And in case you want to know more about any issue, you can simply go and learn more and it would give you entire details of how to fix the problem. Why does this problem even matter? I'll go back and you'll see that we have impacts as well. Now, this is a serious issue. This one is a critical issue, the other one is a moderate issue. Then again, we have a serious issue. So this would help you to prioritize your issues in a way. Now we'll try and replicate whatever we have seen here into our automation framework. We'll be using IntelliJ in order for us to make automation suite from scratch. For that, let's create a new project, an empty project.

So selected empty project. And I write this as.

Test get.

Let's open that into a new window. So this is all we have in an empty project. Now I want to create a package JSON so I'll do NPM init and it would create a package JSON for me. So you can simply press return because none of these are like mandatory.

So now if you'll check here, you'll have package JSON. Now within this you'll get to see all the dependency that you've added. Since we're using Cypress, so Cypress is a node dependency, we'll be adding Cypress here. So NPM install Cypress.

After installing Cypress, we would be doing a Cypress open. So instead of npm we would be using npx, which is our package runner command. So the best part about Cypress is that it would give you a template like structure, which would be having four different folders. Now if I want to walk you through those four different folders, let me take you on to the slides.

So here you would be able to see that we have four different folders. So we have Fixtures, integrations, plugins and support. So Fixtures would have all the data validation files wherein we can have some hard coded data that we can use for assertions. Integration is the area where we would be having a test file. So here is where you put on your test suites and test cases. Then we have plugins. So plugins are required whenever you want to load your project or whenever the project gets open in case config changes are there. So it would be able to accommodate that. Support support is another folder in Cypress which would have all your custom commands. So in case you're making some commands which are not inbuilt, you want to make some of your own commands, you can use support folder. And in case you want to have like project white hooks, you can put them in support folders as well. Let's go back to our empty project, let's see where it has reached. So after adding Cypress as a dependency, you can see it has gotten added here. Now like I said, we have to do an NPX Cypress open and it would create a template like structure for us.

Okay, now let me take you back.

Now here there is a Cypress folder within which we have four different subfolders. So with an integration I'll create a new file with the name todo. Spec JS and here we add our test cases. Now.

We'Ll use describe. So what are we doing is we are actually testing.

To do application.

Okay, so now this becomes a test suite. So now what we're going to do is we are going to add a test case. What we are basically doing is testing the page load. So I'll add an ID and I'll call it as test page load.

Right? And within this.

We can actually visit that URL that we saw. So we'll visit a URL.

So we visited this URL. So I'll fetch it from here and I'll put it in my test case.

As a part of the assertion. Since I talked about, we have inbuilt assertions here. So we can actually check that the URL that we visited is correct or not. So here is.

A should operator and we can use a keyword called equals and then we can just compare it.

With the URL. Right, this is what we have done. Let's try and test if this works. For running it, we'll use Package runner Cypress open again. Now we have Todo spec here and let's try and run it.

And the assertion has got passed. Awesome. Let's go back. And now the next thing that we need to do is to add accessibility on the top of it. So now for accessibility I'll have to add certain dependencies. I'll be adding a dependency called Cypress Axe. So now this is a dependency that would allow the Cypress Node server to access the axe APIs. And then internally the Axe runner is actually using another dependency called Axe Core. So I'll have to install that as well. So it's Axe Core. Now let's check if the dependencies have been added. So now we have Axe Core and ciphers Axe as well.

Now what we want to do is we need to use Cypress Axe in the entire project. Now for something to be used in the entire project, we need to use Project wide hooks and that goes under our support folder. So in the support folder I'll simply do require.

What do I require is Cypress Axe. That's it.

Now I'll go to my specification file and here I'll have to use two commands, right? Let me tell you more about those two commands. Cy.inject axe it would extract the DOM. So you'll have to do this before every test case of yours. And before every test case we'll have to extract all the document object models so that we can access all the elements, whether it is a button or div, an input, anything. Then the other Command is check A11Y. So check A11Y is the short form of check accessibility. So as soon as you run this query, it would run an accessibility check and it would give you the vulnerabilities. Now let's use that in our code. So here I'll have to put before each hook. So before each.

What do we have to do? We'll have to inject Axe. So inject Axe is what we'll do. And then I'll write another test case where we're checking.

Accessibility for page load.

Now we'll check this.

And here I'll use CY check A11BY.

That's it.

Let'S see what happens. Now I'll run it again and PX press open. Now let's run the todo spec. So this was successful because assertion got passed. The other test case was check accessibility for page load. And now we have two accessibility violations. Now if you would have noticed, we had four accessibility violations.

When we did this using our Chrome extension, but here we have just two. So now this is an ongoing issue that you might feel when you use Cypress Axe that some of the elements are not getting exploited using Cypress X. Another solution to this is another dependency called Better ciprosaxe. So let's see if that solves our problem. I'll do an NPM install Better Cypress Axe.

The only change I'll have to make is in the support folder. As soon as I do this, Better ciprosax is ready to be used. And let's run our file again and let's see if we are able to find all the four violations. Test page load is successful and now we have four accessibility violations. Now we have understood all of this. The other thing is that how can we customize the usage of Axe? Now this is the project that we made, but I have a similar project already created which is more exhaustive. It has more test cases, so I'll take you to that. So it has to do spec. So this is all that we did. So we tested the page load, we did the accessibility violation check, but now there is another option wherein you can actually customize your accessibility test cases like in case you want to filter out using particular standards like we talked about different tags. So you can have certain tags specified here according to your product needs and product demands and you can only find out those issues which are violating particular standards like these, right? So in case you are in United States, section 508 is very essential there. So then you can just find out what are all the violations around section 508 by using this run only command, right? So this is a kind of custom parameterization that you can do for check accessibility function.

Similarly, in case you want to find out issues with specific impact areas. Now in case you want to prioritize that in this sprint I'll just be picking up critical issues, then we'll pick up severe and then moderate. So in case you want to go with that ideology, you can use your test cases in a similar fashion. And then as you go along doing your functional scenarios, you're opening another page, you're opening a frame, you're clicking on a button. After performing every functional scenario, you need to perform an accessibility on the top of it, right? And this is what my test suite is doing. So let me just run this for you so that you understand this.

BlueSpec and as I mentioned, I know this may be hard to visualize, but don't worry, you can actually see the video and see this in action@testguild.com a376.

And just focus on the right frame. What's happening here because we actually performing a lot of functional things as well. So what we're doing is we are adding certain items to the list and then we are striking them off after getting them completed. So like you can see here, we are doing accessibility check after performing every functional action, right after adding we are again doing an accessibility check and then after checking out the item we are again doing the accessibility check so as to see what all violations we are getting. And if you see here, it gives you number of instances also. So there could be just one issue, but it could be on four different elements, right? So now this takes your count to four plus three which is seven plus one eight nine. So it has nine issues in total of four different types. So that is how we actually figure out and categorize our different accessibility violations. So all in all, this is what we do in order to create an automation framework from scratch.

And you can always create a robust framework. You can add as many test cases as you want. So like this one was for the starters, but as you go along your journey, you will be able to figure out how and where to add more accessibility cases for you to figure out what could be the possible gaps.

That might lead to origination of different accessibility violations. So we have made to the end of this session and let me just wrap this up by giving you some key takeaways. So today we learned how to make an automation suite on Cypress from scratch and how to integrate that with Accessibility Engine, which is axe. And you can primarily use Axe with any of your existing automation test suites as well. And I promise you that it would work seamlessly and in no time you'll be able to figure out where are the accessibility gaps. And that would definitely give the team a better head start in terms of making their application robust and accessible. So I am pretty hopeful that you gained some knowledge out of this session. In case you have any questions, queries or feedbacks regarding that, feel free to shoot that up. Thank you guys.

Thank you Prigati for your automation awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com a376 and while you're there, make sure to click on the Try it for free today link under the exclusive sponsor section to learn all about Sauce Labs awesome products and services. And as I mentioned, registration for Automation Guild 2022 is now open. You definitely want to discover quick, actionable tips, tools, techniques and best practices that can help accelerate your automation efforts in the new year and accelerate your career to do that. Head on over to automationguild.com and register today. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack pipeline automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.