From Manual Testing to Continuous Testing with Jorge Luis Castro Toribio

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Jorge Luis Castro Toribio all about how to make the transformation from manual testing to continuous testing. Jorge is an agilist who feels passion for test automation, agile, DevOps and business agility. He enjoys researching and learning about business agility and working with cutting edge technologies. He's been learning continuously for more than 12 years and is still learning new ways to foster business agility and team greatness. He has worked in several roles as a developer, tester, manager, agile coach. He's done it all. This has helped him see the big picture and the operational and teams managers dynamics happening inside most organizations. This experience lets him help teams design, build and implement digital DevOps and agile transformation strategies. He encourages focusing on people, productivity, continuous improvement, innovation and having fun to enjoy success in your agile journey. You don't want to miss this journey. Check it out. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Jorge, welcome to the show.

Hi Joe, thank you very much for the invitation.

Awesome. Thank you so much for joining us today. Really excited. I saw a LinkedIn slide deck on moving from manual testing to continuous testing. So I thought that's what we'll talk about today. Before we get into it though, is there anything I missed in your intro?

Well again first, thank you very much Joey for the invitation. I hope that this can be a great experience for your audience. Well, about my profile, my bio, I think you said everything about my experience.

Thank you for that bio. But basically something that I really like is sharing. I strongly believe that professionals, engineers and no matter whatever profession you have, you need to share more and more. I think that is something that is going to help the community. So. So that is my main purpose right now. I want to share all the experience that they have. That is what I'm looking for. Share more.

So Jorge, before we really jump into it, when people hear continuous testing, I'm not necessarily sure they know what that means. So at a high level, before we dive into it, what is continuous testing and how does it differ from manual testing?

Sure well, first, the manual testing is a traditional testing, right? The manual testing is the manual process or the manual activities that a tester perform to test something, right? You know, for example, when you run manually, a test case with some steps, basically that is manual testing. Basically, manual testing is the set of activities that you can do manually to try to validate something, right? Software or a process or whatever. So that is the traditional manual testing. It's a manual process which, as you know, when something is manual, you have more chances to get issues right with that. And of course, with manual activities, there are more chances to take more time. The difference to continuous testing is that the idea of continuous testing is the testing that you can do in all the software delivery life cycle. So for example, no matter if you are in the requirement phase or you are doing, if you are in the development part or in the post production deployment, or no matter in whatever part of the development life cycle you are, the continuous testing is all the testing that you need to do to guarantee the quality and the user experience. So basically there is. Continuous testing is the set of activities, a set of testing activities that you can do in all the delivery life cycle. Those activities can be manual. Yes, those can be manual, but at the same time they can be also automated. So for example, imagine that you have a pipeline, right? A DevOps pipeline. So in that pipeline you can have jobs that run a set of automated tests, for example, unit tests, integration test, API tests, performance test, security tests, and so forth. So in the pipeline, those tests don't run one time per day. Most probably those tests run thousands and thousands of times per day, right? So the only way to do that is with automation. So that is, that is part of the concept of continuous, right? Continuous testing is you have tests that need to run in a continuous way during the entire life cycle development. So that is the main difference, right? The manual testing is also a phase in the software delivery life cycle. After development, you have that testing phase. But continuous testing is in the entire lifecycle development.

We hear a lot of terms with shift left and shift right. It sounds like rather than in the old days, you just had a group that handled testing. Now everyone on the team is responsible for quality and testing all the way through the lifecycle of your software development lifecycle.

Sounds like yes, yes, absolutely, absolutely. Something that another difference, as you said, Joe, is that testing mindset, right? The quality mindset or the product mindset. You can see the advantages of continuous testing, especially in product teams, right? When I say product teams, I mean that a team that is in charge of the quality of a product. No matter if you are a developer, you are a tester or product owner or marketing guy, it doesn't care if you are part of the team, you are in charge of a product and if you are in charge of a product, you are in charge of the quality of the product. So testing is a whole team mindset. And also that is a big difference related to testing as a phase. Right. With the functional manual testing.

So all right. No, I think I'm older than you. I'm pretty sure of it. So when I started, the norm was we had a separate group for manual, by the way, Agile, everyone.

It was understood you did continuous testing. So when you speak with other organizations, other company, is this a new concept for them or is this just helping them tweak their current process to make their testing efforts better?

I think it's a new concept for them because, well, regarding my experience, I noticed that there are some companies that they understand that the testing should be a part of that delivery software delivery life cycle. Right. But more and more due to that digital transformation, those companies are changing the way are changing the way that they are doing testing. So I think that the continuous testing mindset can help us in that digital transformation journey. So I noticed that each time more and more companies, they are getting experience on continuous testing because they find that very useful for the digital transformation. One of the digital transformation has some pillars, right? One is DevOps, the other is agile, business agility and so forth. So continuous testing is as a concept which is related to DevOps and also agile. So it's a kind of mix of both. So I think that it's a new concept for some companies, for companies that are in the data transformation journey is something that they are applying right now or they are trying to apply.

Great. So another, whenever I think of continuous testing, I also for some reason think of continuous learning. I know you've embraced continuous learning, but it seems like in order to succeed in this environment, you have to be comfortable with some sort of always having to learn something new because it's no longer someone else that does a particular piece of testing like security or performance or whatever it is. It is actually everyone involved. So is that something you think people struggle with, learning a new skill?

That is the tricky part. Yeah, that is the tricky part. You know, changing the ways of working. It's the hard part of any transformation that is the most difficult part. Because at the end, technologies like selenium or cucumber or, you know, or tools for continuous Testing or continuous deployment and so forth. Those are tools, right? You can learn a tool, that is technology. You can use the technology for a goal, right? For a purpose. But at the end, people are going to be in charge of that. So no matter you have the best tool in the market, no matter if you have all the money or the budget for your Agile transformation or your DevOps transformation, at the end, the most important part is people and how they can change the ways of working. So as part of that, getting new skills, learning something new, changing the way that we are working right now, it's very difficult. So I think that is the challenge. Actually, my paper is about that, right? Because in my experience I had the chance to work with some manual functional teams in my career, and due to the digital transformation of the company or the data transformation, I had to work with my team to move from manual testing to continuous testing in a debug transformation, for example. So in my experience, what I found is that the most difficult part is getting new skills for the people and changing the ways of working. So I think that is the main challenge and that is something that in many years, or the coaches or the people in chairs, the leaders need to focus on in how they can take a new concept, take a new technology, take a new skills and how they can get into people, right? How they can present this challenge to the people and change their ways of working. So I think that is the main challenge.

So what's the driver for that? I believe you have something called start with the why is that where someone, if they're just trying to make the transformation, what does start, what the why mean?

I, you know, basically all this conversation, Joe, is about my experience. So those are kind of lessons learned. So I think start with the why something which is really, really important, no matter. And actually that is something general, Joe. No matter what is your journey, what is your project, what is your activity or your goal, the first step that you need to do, you need to start with the why. You need to start with why. I'm going to do this, right? So for that purpose, for example, you can use some techniques like okrs, for example. So as you know, okrs, objectives, key results can help you to clarify a vision and to define some objectives and define some key results. So you can align people, you can work with them, and you can set up a clear vision for them. So in my experience, no matter what you are going to do, you need to start with the why, you need to clarify what is the vision, what are the objectives, the key Results.

What are the next steps for the people? And most important, you need to share that with your team. That is something very important. I noticed that Most of the DevOps transformation of data transformation failed because of they didn't have at the beginning a clear vision. So if you don't have a clear vision, people is not going to follow you. So that is something important. At the same time, you need to start with the why, you need to research and you need to define or you need to identify what are the current status of your development life cycle. So that is something important. For example, if you want to change the testing automation part, okay, so you need to get metrics about what is the test coverage you have in unit testing, integration, regression and so forth, how many days your regression test takes, for example, how many issues you have after production, deployment, and so forth. So you need to get some data. Getting the data is part of start with a why. Because at the end you need to share that information with your team so they can be aware of how big is the challenge. They can be aware of what is the vision of this project, of this initiative, and they can follow you. You can make them feel part of this information, you know. So I think that is something important, something really critical for managers, for leaders, for anyone who is in charge of a group of people. Start with a why and also use numbers for that.

Nice. So, Luis, once they know the why or we know our why, I believe you talked about pillars. So where do these pillars come into play? I think there's three of them. When we're trying to make the transformation from manual testing to continuous testing.

Well, after you have a clear vision of your why and after you have your metrics, after you have the current status of your challenges.

And now you can have a clear vision of the place that you want to go, you need to define a strategy. And your strategy is the way right, is the path that can help you to go to reach that goal. So in my experience, we created some pillars as part of our strategy and we defined three pillars which we considered very, very important for the transformation to manual testing to continuous testing.

The first pillar is people greatness. We noticed that. We identified that the only way to be successful in this challenge is getting the people involved in the transformation. So we decided to do whatever activity in our strategy, but we need to focus on people. People greatness.

People readiness is not a concept created by me. It's basically a Google concept about you need to trust your people, you need to take care of them, you need to foster the innovation in your teams, you need to focus not on tools, you need to focus on people and in their development. So in that way you're going to get people greatness. Because no matter the team that you have, they are great. So you as a manager, your main responsibility is how do you, how you can develop these people greatness? Because people is great, I can guarantee you. I am saying this because when I started my project.

From my manual testing, the continuous testing activities or challenges, most of the teams were manual testers, you know, tester that didn't code tester who weren't too much technical. So it was a big challenge because DevOps Cloud and so forth demands that the people.

Code, right? Demands technical skills. So it was a big challenge. So that is why we focus our strategy in people brains. The second one, the second pillar is try, do and learn, right? Basically this is about experimentation, it's about failing fast, it's about innovation, it's about creating a good environment so people can try new things and don't be worried about it.

And this is very useful, right? Because in a transformation, for example, in a new ways of working, adoption, you have a tester, a manual tester who doesn't have too much experience coding with DevOps as well. So you need to tell the message that it's okay, we are going to try together. We are, we are going to help you with this. You can try new stuff, you are going to get some support, try to research for your end, try new stuff, right? And that's okay, we are okay, we are learning together. There is a message, we are learning here. So I am not an expert, you are not an expert. We are changing the ways of working as part of our goal. So it's okay, please try, do and learn. That is the second pillar. And the last one, the third is challenge ourselves. This is something very, very important, right?

We cannot create a detailed plan, right? You cannot tell people what they need to do, detail it. Because at the end, what I am expecting is.

I'm going to share with you a goal, I'm going to give you some tools of some good environment for learning and some learning opportunities. But at the end, what I am expecting from you, from the team members is that, please, you have the goal, you have the environment, you have the learning activities, but I need that you take ownership of the next steps. At the end, the people is in charge of the how, right? How they are going to build something, how they are going to do this testing and so forth. So the only way that we can foster that in people is you need to challenge yourself. So that is the message as part of the strategy. Each team member that wants to adopt new ways of working needs to challenge themselves.

So after you have the pillars in place, you also talk about rounds. So I thought we'd dive into each round really quick. What is round one?

Yes, well, after you have the why, right? With the goals, with the, with the status of your team and the status of what you want to change. And after you set the pillars, when I said the first round, is that the first step that you need to do if you want to change the ways of working from manual testing to continuous testing, is you need to start with working with some teams, right? I think that is the approach. You cannot apply this for the entire organization, especially if you are enterprise. You need to choose some particular team to start with. These changing ways of working. So the first round means that you need to create a first round or the first version of the teams. For example, one key point is if you are working in silos or you have something like the development team and the QA team and so forth, you need to break that. You need to create some product teams, right? Which is something really, really important. You need to create some product teams. A product team is a group of professionals, a group of people who are in charge of a product. It can be a web application, in house application or whatever. They need to be in charge of something or in they need to be in charge of a product. So first you need to create your first round of your first version of a team product team. You need to create some OKRs for that team. And of course those OKRs need to be aligned to the enterprise OKRs or the OKRs for the entire transformation. You need to define some team purpose and vision for this first team. And you need to be sure to foster collaboration, transparency and make whatever you are doing visible. So for example, for this first team that you are going to create, you need to share with them what is the main goal, what are the OKRs of the strategy, what are the pillars and so forth. You need to be very transparent with them about what are the real expectations of the testing activities in this team, right? Because. Because at the beginning this team is going to have manual testers. So what you are looking for is change that, right? You need to move from manual testing to continuous testing. So you need to be clear with them, you need to be transparent with them. In terms of guys, we are going to create this product team, but practices like manual testing, we need to move to the technical testing, right? We are going to use DevOps. We need to create some unit tests, automated unit tests, automated functional tests and so forth. So we need to be clear and transparent with them. I strongly recommend to use this Kanban principle, which is start with what you do. So I think that is something really important at the beginning. Basically you need to trust your people, you need to be transparent with them and you need to create your teams with the people that you have in your company. Because after that, after you have your first team, your first product team, you can move to the second round.

Basically.

Before telling you what is the second round of the team, something that's really important is for this first version of your team, you need to clarify what are going to be the main goals as well. So you need to identify some areas of improvements for the team. In this case, what I identified is basically the testing activities, functional testing, how we can improve that. Also you have identified some metrics so that the testers in that team and the entire team can have a clear vision of what are the main goals and what are the next steps. That is basically, Joe, the first round of a team, the second round of the team is, you know, after you create this first team and you have the goals of this team, you have some metrics and you have the areas of improvement you need to start with to try to enforce this team. That is why that we started with some new hires, which is something important because if example, right, if you need a lot of, if you need to implement automated testing for security or for performance and so forth, and if you don't have that skill in your team, you need to hire, most probably hire something that has that experience, but something that is good is that in this first, in this second round, basically you already have the team and this team is going to be part of this hiring process because they know the business, you know, like challenges. So they are going to be, they are going to work with you in this, in this goal. Now you have the entire team, right? You have that entire team working together. You have the guys that need to change the ways of working. You have some guys that they are going to help you with that because they have experience in that new ways of working.

Basically in this second team, in second round of the team, you need to clarify and have a more mature version of your okrs of your team vision and so forth.

You talk about the future framework architecture at a really high level. I think people are really interested in hearing more about this. What is the future framework architecture and how does it help people make the transformation to continuous testing?

Something that as part of creating, creating those product teams create this vision and those okrs? That technical part is also quite important because at the end, continuous testing is more, is more a technical set of practices. So as part of that, we need to create a new framework. We need to create some frameworks because we identified that cross creating frameworks is the best way that the people can learn, for example, how to create automated tests. So that is why we work together to create some frameworks, for example, for automated functional testing, for mobile testing. So we defy that those frameworks are key enablers for testing, changing the ways of working of our traditional testers. So for example, we, at the beginning, we created some future framework architecture because that is because this future framework is how we can self, how we can see our teams in the future, right? Because as I said at the beginning, basically we were manual testers doing manual testing. But what we are looking for is, what we were looking for is that each time more and more the testers can apply and can understand this new framework. For example, in our experience, we use selenium with cucumber. You know, cucumber use human language to create automated tests. So that is something great. Because in that way you can take people which people who don't have too much skills, technical skills, and we see using cucumber, they can learn how to automate a test using human language, which is something great. So that is the main message when you create your frameworks for testing automation.

You need to make them simple and easy to understand for your people. So you need to think in a simple way. For example, in our case, cucumber was key because it used human language.

It sounds like you actually went through this transformation yourself. You brought a team from manual testing to continuous testing. Just curious to know what did it look like after you made the transformation? Was it like 50% more bugs found or were there any high level things that you got out of this transformation?

Okay, something that I can share with you about it is, for example, after we took some teams, you know, some functional manual testing teams, and we move them to the continuous testing, for example, we got some improvements in the automated regression coverage, for example, right? When we did only manual testing, basically our automated regression coverage was zero. Right? Because all the regression testing was manually, was manual. So after we moved to the continuous testing and after our testers got new skills and they worked together with the developers in the DevOps pipeline, creating new tests, for example, we got 80% of automated region Coverage, which is great. Imagine that at the beginning your regression testing activities took more or less three weeks. After that, after a new ways of working adoption, we got 80% of coverage of regression testing and those three weeks reduced to only three days. Yes, that is something great because at the beginning you had 20 testers working in manual testing regression and at the end, after the transformation, we got only three testers monitoring the automated jobs in only three days, which is something really great.

Jorge, I'm just curious to know, you have something called a Batman and Robin approach within your slide. Is this what you use to help make the transformation, to help people code or what's the Batman and Robin mindset? I guess is the question.

Yes. Well, the Batman and Robin program was a program that we created. So, you know, imagine that you need, you need your team learn Selenium, for example, right? So you know that in your company you have people which that are skilled. So in this case you call those guys the badmintons, right? The Selenium badmas.

So.

And you have of course some guys that need to learn selenium. So those are the Robins. So the program, the Batman Robin program is a program where you have some Batmans which are experts or they have skills in some technology or in some activity and they need to work with some Robins who are the people that need to develop that skill and they work together for a period of time sharing goals. So that is the Batman and Robin program. It's a program that foster team collaboration.

In a fun way. Right? Because at the end of the period of time, the Batman wants to develop the Robin and become the Robin to administer. And Dan Robins want to become a Batman. So it's a kind of team collaboration program to foster new ways of working.

Okay, Jorge, before we go, this episode actually has been full of actual awesomeness. So what's the best way to find or contact you?

LinkedIn is the best way. I try to be connected most of the time, so I think that is the best way to contact me.

Thank you Jorge, for your testing awesomeness. The links to epitheton of value we covered in this episode. Head on over to testguild.com a299 and while you're there, make sure to click on that try it for free today link under the exclusive sponsor section to learn all about Sauce Lab's awesome products and services. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating Full Stack Automation awesomeness. As always, Test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.

Sa.