Salesforce Test Automation Strategy with Mala Punyani

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Mala all about test automation strategy and more. You don't want to miss it. She's worked for some of the biggest brands in the industry like Visa, Home Depot, Splunk and PayPal. Amala is a highly motivated and experienced leader with over 14 years of experience. So she has a diversity in product quality and automation. She's done a lot of things building enterprise quality products, CICD and automation solutions. And she's known in the industry as a successful leader with experience in building and leading teams, which we're going to dive into as well. She has a proven track record of translating organizational strategy into actionable solutions that drive operational excellence and growth. Really excited to have her on the show. As I mentioned, she has a lot of experience with. We don't know where this is going to go because I said, as I mentioned, a lot of things we could touch on. Really excited to have her on the show. You don't want to miss this episode. Check it out. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com forward/source labs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Mala. Welcome to the Guild.

Thank you. Thank you so much for giving me opportunity to be here. I am very excited to be here today and share my journey.

Awesome. Awesome to have you, Akeem. Highly recommended by Tristan Lombard. So I'm really excited to have you on the show. Before we get into it though, is there anything I missed in your bio that you want the Guild to know more about?

I think you nailed it for sure. But I do have a very diverse automation platform from backend to UI to web services and then Salesforce, which is I'm currently working, which I never worked before. So I do have a very diverse background for that.

Nice. So I thought we'd dive into a little bit about that. Going from maybe you mentioned backend automation to API automation to like Salesforce. How's it different? How do you as a leader know, like how do you read the room to know, like, okay, this, this group is different than when I did at vis, so I need to have a different approach and maybe different tool selection than maybe what I've used to before.

Definitely it comes with a very different platform and where the different stakeholder works. So I do believe automation can come from anywhere from developer to quality or automation engineer. But everyone needs to have a buy in that including from your product to your development team. Because it's really important what QA or automation engineers are building should have a transparency to across the stakeholder and we should keep building it. Plus our team should be aware what we have automated and how we are getting benefited. So in a cross of my 14 years of journey it was really really important like to know like what audience you're representing, what works the best for it. Let me give you the example. Before joining a Splunk I worked in PayPal, Visa and one of the startup which bought by Home Depot lately those are financials are very different companies. And then I worked on a website, very different audience and all. But I always had an open mind what works best for each application. When I joined Splunk Salesforce, totally different application, totally different era. I never have even seen the UI of Salesforce before that you know in that much in detail. But for me was to how to build the best automation for the team. It could be automation. My slogan was automation for all rather than automation for QA or automation for quality. Because as we are moving towards cicd, as we are moving towards a more frequently releases, it's very important that you should choose the tool where anyone can contribute in automation with a least learning path and they should have a time and transparency to know your automation and know the advantage of it. So when I joined Salesforce Group I always worked before then open sources like Python build data driven build automation framework and before then it was a REST API based automation framework. I worked then in PayPal there was an internal tool called Cell Cube I worked on it. But when I joined Splunk if as our automation leader I could have chosen the same technology over there but they were not a single person who was aware of Python. Salesforce is written in Epics. People don't even have a time to invest it in Python and can build. Yeah, I could have hired a team which would be like great in automation building but no one else would have contributed it. So I came across this tool called Proverb which isn't written in Eclipse Java and I basically suggested to my management and we did the POC for plenty of products and we go through this time a tool based automation which is really successful for the Splunk Salesforce team because it is very compatible with Salesforce. It has all the compatibility what Salesforce platform has it. When Salesforce is making a changes to their releases prover does it by itself. So we are in a very compatible zone with the Salesforce. So it is not like I have start building some niche technology in a Salesforce platform where no one knows it. So the advantage is today our developers are writing the automation using the tool. It is for them as easy as it is for any other automation engineer. And this is how we are proceeding for CICD for a Salesforce environment. So the key point is taken as here is that you really need to understand your team, you know, strengths and you need to work for the automation, not on the focus of what QA team will build. You need to have a focus what everyone else can contribute to that automation. To have a successful automation platform for any organization, I love it.

And I guess that proves that you're a leader because I think it's kind of gutsy. I don't know why, because a lot of people seem that buy versus build they're always like we'll just build. There's open source where open source will only do open source. They always minimize or don't even look at a solution that may be paid for but it's made specifically for maybe the user case. How did you. Did you have any pushback by promoting maybe a tool that wasn't, you know, open source?

Yes, it is always there. You know, technically speaking, when I was working in Visa I do remember I can still bring that meeting when I said hey I'm going to build a automation framework in Python and they said why not Java? You know you would get always a people who would have one versus another. But yeah, at that point I was early in my career and of course I was not have that organization also open minded. But I'm very lucky in Splunk they're very open minded people like and you need to bring what you want to achieve. I told my manager I am great in Python. I can bring the team for Python, we can build the open source. But I can assure you none of your single developers would be contributing it or if they would be contributing it. You have to have a bigger team who could have spent the time to writing the automation because think about it, they are writing a code in Epics now you ask them to write a unit test in Python, they're not going to do it or they will do it, they will take more time for it and then you are maintaining the two different platform for them. So it did like as I said, I had a very open management. Then I did the poc. We did the POC internally. We presented the options, we and as I said, my slogan was automation for all. So as in our organization we have a technical product manager who calls as a bsa. They can even write automation by themselves because they want to contribute in writing it. Think about it. People who have a least background for coding or maintaining the open source learning curve would have been a very high for them. Plus who wants to maintain those test cases also? So breaking the barrier in this era is really important. Liking, personal, liking disliking is one place. But what works for one organization is a totally different and you need to think out of the box solution. It could be open source or it could be tool. I'm not saying one is better than another. Oh, I accepted the tool which is easy. So you all should do that. But your focus should be hey, I work for a organization. If I'm building this tool, can someone else from QA team can use it? If not, then you also need to bring the learning curve and try to empower your other stakeholders also because this is how automation wins. Otherwise people says to me, I can give you another example. People come to me and say hey, why can't we run, you know, 100% automation for two days release? I said, yeah, we have 2,000 scripts. When do you want me to run? No, I'm not going to run. So if people are not aware what you're building and they think automation building is something which you can like run like a robotics star, which is not there, it will never give other person a sense. Also like you know where the contribution require and how we are contributing it, on how we need to arrange our release in CICD pipeline too.

I love it and I always get asked the question, what's the right tool for me to use? It's like I don't know, it depends on your team, depends on your technology. And it seems like Salesforce maybe have unique challenges that maybe like Selenium maybe wasn't created for maybe, I don't know. So did you notice anything unique about Salesforce? That is automation is different than maybe what you did previously?

Yeah, you can see not previously. As I said, I never used Salesforce. Previously you could have easily built on Selenium. There was a no problem building in Selenium. But now I have told you the another challenge, right? Salesforce has their own releases, right? They have a winter release, they have a spring release. So they make Changes. Now if they're making changes, you have to keep maintaining those changes in Selenium too. Plus the Salesforce has a different objects and as I said, it's written in Apex. People who are building the Salesforce coding system, they are trained in Apex, they are not trained in Selenium, they are not trained in building in JavaScript. They are, they can build JavaScript kind of UI but. But I don't take as a assumption all Salesforce developers are proficient in any other language. For me, they are very well aware of how Salesforce work. So this tool prower is exactly the same how Salesforce component has been arranged. Like it reads your metadata and it creates an ARC for you. Like suppose your ARC is changing. I am not. Like I can tell you, I have a one page which is called opportunity page, which has 60 plus different fields and which is changing. Think about it. If I could have written in Selenium how many controls I have to write and identified when page is loading. But now I. We have written in proverb so it reads a metadata from your ARC automatically and tells you this is a difference when you run last time and now. So basic things are already there. So my team is not spending lot of time to doing the basic task on it. And as I said, if I am training someone who knows Salesforce, for them the learning path is curve is really, you know, minimal. Plus they can keep contributing it without spending lot of time.

I love it. And that's one of the sales pitches of some of these tools is low code or no code. And a lot of people are skeptical. But it sounds like Prover is a low code, no code type solution. Is it really working that way?

No. And I'm being honest with you, and I'm sure Tristan would hear that our ARC is pretty complicated. We do write lot of wrappers for us. Like I can give you example. We write a Wrapper to calculate 50 fields when you change your sales order. We do that, right? We do write a logic from data driven to how my data is, you know, reading which test case you need to invoke. We have written a logic in Java. We have. It's not like it's just the basic thing, you know, the basic structure which we need to maintain it. It is already there. So I'm spending more my time for writing those wrappers rather than maintaining it and rather than doing the basic things daily basis. So it is definitely not no code for sure. And I don't think so. Prover did sell it as a no code tool. They say it they are very compatible with Salesforce and they have all those features of Salesforce and that I agree with it. Right. So like in Salesforce you also have a different profile users. Right. When you're running a one test you have to log in as a five different user to do five different actions. So they have those. I'm just giving example those connection build that one test case can run with the five different profile by itself. So I'm not writing a code for that, you know, it's already there. I'm not writing a code for establishing a connection for five different profile to log in and to log out. To log in, to log out. It's already there. I'm using. But as a logically if I need something to build like I give you the example, I need to calculate something or I need to bring my data driven in a sense where specific test case needs to get executed. If I Pass file values versus 7 value which we are writing it for. Sure.

Cool. So you've been doing this for 14 years. How has the tester's role change? I always get asked, you know, is testing going away? It sounds like you keep saying automation for everyone. So it sounds like maybe your developers and everyone else get involved in automation. So does that minimize the tester? Is the tester role going away? How do you see that?

Honestly speaking, it is very challenging for QA to have in a market with a good skill set and growing with the career skill set. When I'm saying in 14 years it's very different. Sometimes I also do think I already made a mistake. I should have changed my career 10 years back. Because if you have seen now so many companies are there who proudly say, oh, we don't do qa, you know, there is no qa, no automation, we don't do all those things. But still the companies who believe in quality and the company who has more end to end testing and system testing, they do require qa. Right? Like you name the big companies like Amazon, the companies who are very customer centric companies, right. For Amazon, the quality is really meant for them a lot. And they do hire a lot of quality people. They don't care what Facebook is doing, they don't care what Google is doing and they don't even change people title. If you see in Amazon people titles, the QA title is still qa. QA is a vertical, vertical, you know, Arc, it's not like oh, you are going to report to director of development. But as you said, how the role has changed. When I'm saying Built automation for all. QA needs to focus more on end to end or system level of testing where I don't think developer can contribute a lot. When I'm saying everyone needs to contribute in automation, yes for functional, sure. But for a system and for integration, I expect QA do it the most of the things. And that is what I present to my management that how is QA testing is different than developer testing? If everyone would write automation or unit test, yeah you will unit test one function but if that one function is going to work with another 10 function integrated well, that is my team is going to test. So QA really needs to think now on what is their end customer goal is QA just cannot think, hey, I'm doing the functional test how to create a sales order in Salesforce. That is not what I'm expecting from qa. I'm expecting from QA to know once the sales order is processed what will be the invoice for the end customer would look like. That's what is a QA role right now. And when I'm saying why everyone needs to contribute it, equality is becoming the part of everyone's job. It's not like only a QA or you know, we would have three months of development, then one month of QA and then we will decide we are going or we are not going. Currently we could have I have a release in every month but I worked where we have a release for two weeks window, right? People are releasing in weekly basis. How you need to make sure you are. Each release is quality intact. In that way each person in the group has to own the quality. It doesn't mean that QA is not responsible. I'm not saying hey, I'm not going to responsible. If any defect comes, I will just wash my hand and say hey, everyone tested. I don't I define what we are responsible. If some bugs comes, I take the responsibility. I'm not going to say to a developer or to a QA who missed it, we missed it testing, missed it testing, I own it, it's mine. And here is a, you know, plan for, you know, further fix. Is that so I go with this attitude like sometime when I go to the room and say hey, everyone would test, everyone owns quality. Half of the people get white faces in, hey, what are you talking about? Do I need to now take ownership for testing? Because people do get worried when they hear that, right? Hey, you're going to own testing because owning testing is one thing, but coming the defect and taking ownership from the production gets you into a different level level of it. So I said no, that's not what I meant. If something comes up, we will take the responsibility. Developers can contribute the test which will expedite my team to focus on more the cases which you can't test. Rather than I am doing the functional or unit testing for your code that two plus two works. I don't want to waste my time. My team time for two plus two works. My team wants to do system testing. My team wants to use those automation testing to doing a customer and customer behavior which is our end goal to give them a good quality product where we are going to focus. If something comes up, we will take our responsibility. It's not like, oh, everyone was testing, so we don't know whose problem is that.

Absolutely. Now you mentioned something interesting. If a customer bug did slip into production, do you have a formal process in place when that happens to try to make your process better? Maybe you can bake it in earlier to see, oh, we have a miss here. Let's make it better for our next iteration is just like just whenever a bug happens, it's not really formalized. How does that work?

So honestly speaking again, I'm very lucky. Splunk is not an organization where someone would write me an email, hey, how did that bug come? I usually write an email. This bug came up from production. This is what we missed and this is how we are not going to miss in future. I have in three years in Splunk, I think I received one email and which was not from my group, but apart from that. But I did work from the Even when I was working in Visa, they used to have a retrospective how did this bug come? Who's responsible in this room and how they are going to fix it. At that point also I was not as mature right now, but at that point I basically learned if I stand up in that meeting and say, hey, this was a qms, here is a solution for it and I will make sure this will not happen again. That conversation always went great because I think taking a responsibility is giving a peace of mind to other person. Also, my VP was in the room and that happened and he, he said, oh, you don't need to say that, you know, it was your mess. I said, no, you need to know because I don't want you to leave this room thinking how did that happen? Right? We are a team, but we don't know how this error is happening. That's not a great sign. The great sign is we know I am responsible. My team is Responsible we will take care and I think that empower everyone. Like I have my developers, they know if we miss, I will go and accept it. My team miss, we will go accept it. There is no problem in accepting it and giving them a plan. You know this is how we are going to fix it. And I think that is a change in 14 years. Previously most of the org has a QA was a vertical org. So you have a development director and the QA director and they were basically conflicting about it. Most of the companies has now merged the org together which is good. Sometimes like now they are a same higher level. They know both are their team. Whoever missed it, they need to get fixed. But as a qa, whenever you take the responsibility, you give a lot of people peace in mind. At least we know why the problem happened. It's not like hey, I don't know why it is not working in production. So it's always good. And I always try to document. I always send a note to my product holder like this is what missed. This was a test case. We are adding it into our part and I make sure that I have been communicated. Even I don't have a process for it. We don't have a process but I do that.

Cool. So I'm thinking also when I'm thinking of Salesforce, I'm thinking enterprise kind of applications. And every time I think of applications in the enterprise I think of test data. It's really hard. I've never tested Salesforce. Is there any tips or things you need to look out for for test data when you're dealing with these type of systems?

With Salesforce, not only Salesforce, Joe, but in my 14 year careers data plays the most important because if you're testing with not a valid scenario data, just creating your own data set without knowing your usage of system, you are not even close to do good testing and good automation too. You really need to understand what your customer are and how you are going to build your data too. Like currently when I was in Visa two, you know how many transaction Visa get it? Like I remember correctly, one minute was 45,000. It was 2006. No, not 6:12. Sorry, 45,000. Right. In one minute. Of course you don't know. You can't be able to know everything but you still need to have a good sense of to keep checking the production data. Although some production data are under compliance, you can't. But you need to know your customer is a key for qa, for automation, for general. Right now for Salesforce I basically get all the deals, what is in production, we get the deals or this is how what we are sailing in a way. So we try to create a similar data in our lower order. So I try to simulate exactly the same kind of scenarios, plus extra scenarios. But at least I know what is going on production. So I do work with my production team a lot to get the data, you know, dump. And we do sync our org with a production also to see all those data. So I would say keep studying your data. It's not like oh, I studied it one year back. I know my customer one year back. You as a qa, as an automation person, you need to make sure when you're saying oh, 100% my test cases are passing for automation or for functional is your data is aligned with your all production scenarios. If not, keep reviewing when you did last time when you were you have on your calendar, when did you take a dump of it? When did you, you know, sync with your automation? That is another key for me.

Perfect. And besides test data, also cicd you mentioned someone said just run all your tests over two days. Do you have a strategy for cicd? Is it do all your. What? What's that like?

So I'm working with my product owners a lot. When we have a shorter releases, right? Definitely. I want to know more about product like what we are changing it, right? If I know if suppose you are changing A which is impacting B and C in my pipeline. I'm going to put the data which is related to A, B and C. I'm not going to put A data for D, E, E, F, G, O Because everything has to run. If I have A, I can tell you I have a 2000 test cases. If I have a release in three every three days, I cannot even run even I can't even look for the failures. Also right? For two week release, I hardly get three days for testing. That is manual in automation. So you have to be very smart. You need to work with your development part. And you also need to start knowing your system in a way that if my developer is changing to a object, what will be the touching area could be. So that is another way very important skill set you require when you're building CICD to get you the good coverage rather than blindly oh, I'm going to run everything A to Z every other release. So that's why this is how QA is different than development. Of course developer can give you the snapshot for their ticket, but QA needs to think, hey, if it is a changes in sales Order I have to run Opportunity Object, I have to run 10 other regression which is going to impact for this. So we do, we do impact analysis for each ticket and that is a big game changer for automation industry also rather than just running blindly everything without even knowing, you know what is really relevant to it.

I mean absolutely. I worked for enterprise company before and they just said run all the tests, just scale everything up, run it. It's like it's too much noise. Did you have to be able to say I made this change and we have these tests that cover that change just because you know we had mini releases then micro releases. It just didn't make sense. Like you said run all the tests all the time. It just doesn't, doesn't make sense.

Yeah. And honestly speaking, okay, running all test is okay, but maybe your focus was mainly for component B which required a very thorough testing but you just were so focused on running 26.

Oh right, right.

And you basically missed that. And that is why even after having so much of testing you still get a bug. Because my understanding is our focus is so many places we are not able to narrow downing the things and that also causes lot of distraction. That does not get you the where you want to be. If you have a short releases, you really have to be precise. You should have a quality pipeline. You should not have let's run everything pipeline.

That's a great point. So say it was component B. Do you have a team that actually swarms on that then for all different user cases doing exploratory testing where you have your automation run but you also have like a team of actually hands on trying things themselves.

We do that, we have a mixing of both, so we do it on a both way. But still I am as I said, exploratory testing is good for the components which is changing now. Exploratory testing, my team is not going to do it for regression or the touching component. We won't be able to do it in three days, like two days.

Right, right.

So yeah, it is definitely you have to be precise, you have to be knowing your system. You know, it's all about knowing your system, knowing your use cases. Well, it's not like hey, what is the problem when you 80% automation, why can't we run everything? I said no, we're not going to run everything. I'm telling you, even if we run 40% fail, I'm just telling you who's even going to look for 40% fail for like half a day when you have a release in like half an hour or one hour.

Yeah. I'm just thinking once again, it takes leadership to do that pushback because sometimes people just like, okay, I'll just run it. Because I think you get some pushback when people, when you tell people, I'm not going to run everything. Right.

Yeah. So that's why I told my manager that's what you're paying me. You're not paying me to get any agree with you. You are paying me to get you the quality release. Even I'm there in your org or not in your org working on some another company. But this is not a path of success. The path of success is you know what you're changing and if you know what you're changing, I will be able to write the good test for you and good selection for test and then we will not get anything. But if you don't know what you have changed and how it is impacting, you're not going anywhere. Even If I have 200% of automation.

Absolutely love it. Okay, Mala, before we go, is there one piece of actionable advice you can give to someone to help them with their salesforce automation testing efforts or automation testing efforts in general? And what's the best way to find or contact you?

You can definitely contact me on my LinkedIn. Please feel free to connect with me. But I would definitely say that as I'm growing in my career it is very important make your automation visible, transparent to a different arc. Because what you're building should not be only focused on qa. So people should think, oh QA has built some kind of automation, being a transparent being. Find a more customer for yourself. Find more people who can use automation like in Visa. I even trained my business owners to run an automation to do the UAT right. So in that way they can have more exposed to our tool and can appreciate what we are building and can give you the feedback.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode, head on over to testguild.com a366 and while you're there, make sure to click on the Try for free Today link under the exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.