Test Data Management Automation with Huw Price

Five, four, three, two, one.

Drop that.

Welcome to the Test Guild Automation Podcast. We're all getting together to learn more about software testing and automation with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we're going to be talking with Hugh Price, all about test data management and how it can make or break your automation testing. Hugh is a test data management veteran and a serial entrepreneur. He's now founded his fifth software startup. Hugh has 30 years of experience in software delivery, has brought collaboration with a wide range of organizations, both large and small. He's also crafted multiple strategies and innovative technologies for test data success on projects ranging from mainframes to open system migrations, all the way to building best of breed automation frameworks to test microservices. So a lot of experience, really hot topic here, so really excited to have them. So if you have any type of automation that deals with test data management, you don't want to miss this episode.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Hugh, welcome to the Guild.

Hey Joe, how's it going man?

It's going awesome. So we had you back in the day on Test Guild on, on the Test Talks podcast a while ago, but it's been a while, so is there anything I missed in your intro that you wanted to tell us a little bit more about yourself?

I guess. Sort of started up a new company. We sort of talk about it as it's the same band back together again, but we got much better amps and better guitars now, you know, the world has moved on so we can move a lot faster and the whole world of test data has become, I'm going to say a lot more interesting. It's a lot more interesting. It was a little dull before and I'll talk about why later on, but there's so much going on at the moment. It is kind of an exciting time and when you wrap it up with the much used misused word DevOps, it's kind of an interesting area to be in at the moment.

Very cool. Yeah, I'm really excited to know why test data management? Why focus you have a lot of experience, why focus your energy on solutions around test data management?

Yeah, it's a Bit of an accident really. I suppose it's always you're working as a consultant or you're out there and someone asks you to write a small utility and then suddenly the utility gets picked up all over the organization. You think, oh, that's cool, let's explore it and let's build some more softw. So you sort of start that way. But then we kind of moved off into all sorts of other areas. You know, we got into the idea of requirements design and model based testing and coverage analysis, which are very interesting, slightly obscure mathematical areas. And then once you think, well, why don't we put these two together and think about coverage and we can talk about coverage later. But the ideas of different coverage techniques overlaid onto the test data itself. So that then sort of took us off in a different direction. And that's kind of where at the same time the world has moved on a little bit now. You know, the commoditization of the test data products. I mean, if you look at something like, you know, some of the original products from back in the 80s, they're still around. I mean, you know, and a lot of the test data tools are very, very commoditized. You know, you have a subsetting tool, you have a masking tool, you know, and you can spend anywhere from 50 bucks to, you know, $500,000 on some of these tools. But actually none of those tools solve anything useful to do with testing. And also they serve very little to do with test automation, which is kind of where the world is moving towards. And when it comes to test automation, I think biggest problem with test automation is keeping up with the Sprint and keeping up with change within the Sprint. Now if you put think of test data as a component inside that sprint, if your test data cannot keep up inside that sprint, then what happens is you finish the Sprint. You say, yeah, we finished it, but then you're suddenly doing your test automation a sprint later or two sprints later, and the test data has the same problem. So it's all kind of bundled up together in terms of a much more reactive approach to development where requirements can actually spit out the test data that you need to test it. So there's some quite interesting things happening around in the technology and also the way that people approach project as well.

Absolutely. And for some reason, one of the main challenges I've seen with test automation that fails consistently is test data management. Yes. Seems like one of the most overlooked areas. Now, I don't know if because people think it's too hard to solve, I.

Think you touch on, you know, kind of one of the core problems in that, you know, test data automation is so flaky and if you look at most bugs that are thrown up, they're actually false positives because what will happen is all false negatives in that it'll show that there's, it's not really a bug, it's actually the data was in the wrong state at the time and a lot of people will be using a particular account which they assume is in a particular state and then suddenly the automation fails and they think, wow. And they do all this research into it and actually what's happened is the account has been changed by either another piece of automation or a manual test has come in or they've copied down production data and the account has now become a beneficiary account. So all of a sudden everything changes. Now that's, that happens so often that it almost makes automation untenable because you then spend your automation, you know, your valiant automation engineer now spends an awful lot of time trying to research bugs not in the application itself, but actually in the automation framework. And then they discover, oh, it's this one call here which is just kicking it all to the left and then the automation fails. So the amount of work that you have to do to, to dig around to find out the root cause of it and a lot of it is actually to do with the test data itself, you know, which is then, you know, being used by the automation. And the other one is you suddenly introduce a new test, quite often it will pollute another test, you know. So I mean I did a one wonderful thing with an HR PeopleSoft HR system and what was happening was we had small tests in at the beginning and it then changed the status of some of the managers. Now that means the managers weren't able to approve everyone else's holidays. So certainly, you know, the addition of two extra tests which makes life better has suddenly destroyed sort of the other 800 tests as well, you know, so it can be very, very flaky. So I think that is something that we've really, really tried to tackle in terms of stabilizing and I think if we focus on automation, good test data for good automation, then you have all the spin offs which is if you have good test data for automation, you also have good test data for development, you also have good test data for manual testing, for integr integration testing, for performance testing. But the advantage of doing it with the test automation guys and girls is that they are, they're open to New ideas. And they're also really open to not doing stuff manually. All test automation engineers, I've said this before, I think I've said it my last podcast, are fundamentally lazy and they will try and automate anything so they don't have to do anything twice. So, you know, you don't have to fight those battle with test automation engineer. But it's a good place to think about test data inside an entire development lifecycle is to start with a test automation engineers and to build a much more rigorous and robust, reactive and fast approach to test data. I think that's kind of the core message that we sort of push out when we, when we come across test automation engineers.

Absolutely. And I definitely agree with the point with the false positives. One of the biggest challenges with automation is maintenance. And usually if something times out, that's easy to troubleshoot. But if you have crazy data going on with dependencies that you don't know about, like you mentioned, it makes it almost impossible to even fix a test or even get people to believe that, you know, it's embarrassing to finally find out after you troubleshoot. Oh, it's not even a real bug. It's something I introduced due to test data. So one other thing I've seen an issue with with test data is, especially when you're working in insurance or finance, is the ability to make your test data compliant with a bunch of legislation, especially in Europe, it seems to be other things. So I assume that that's another common issue you've seen people deal with when they have to do tests.

You sort of come into two organizations. One is they're saying, oh, we've got this gdpr. And incidentally, I think the US market will follow this. I mean, it's good practice. I mean, there are companies out there trying to fight it, but actually you find with regulation that it's easier if organizations follow the most rigorous set of regulation. And the GDPR from the European Union is actually probably the best around. You wouldn't be surprised if California, for example, followed that. And once California follows it, it's like Castle to converters. Suddenly the rest of the US will follow as well. But the GDPR or the privacy stuff is so full of holes and problems. You know, it's very easy to say, well, we just masked some data and you change the names and the addresses. Then suddenly you've got a delivery system where you've changed the names and the addresses and suddenly all of the, you know, you can't test that. All the deliveries are going to the right location because you've moved everyone around, you know, so there's all sorts of problems with that. The other one, even if you change someone's name and address.

You know, if they're the only person in that, in that zip code and I'll use the nine digit zip code, then you're going to find them anyway. And the secret, the, the problem with masking data is if you have one piece of information about someone is whether you can then track them down. Given that most people in development are pretty smart and are quite curious, it's not hard to say select count star from group by zip code and suddenly you find one. Now they might have changed the name from Colantonio to Price but actually you know who that person is. You then have full information, full access because you'll have their account numbers to that one person. So it's very easy to be able to track stuff down. So people are, I find a bit too fast and loose and they're sort of self deluding if they think that masking is actually any good. I mean the other thing you can do is select star order by salary descending and that's going to give you very unpleasant information to get out there in the public domain. Also from a competitor's point of view, suddenly you get a dump of, you know, a development database. It's going to contain all the, all the pricing rules are in there because those are the things under test. And also it will contain historical data about sales patterns. So masking data is, you know, in my mind a complete disaster. And especially with things like with healthcare. I mean, how can you go through and mask enough data about someone that you know, you're trying to test that someone's had, you know, an ovarian cancer screen at the end of a chain of activity and there's some dates in there, you know, how would you be able to even do that? So anyway, sorry, that was me ranting slightly about the problems of it.

Spot on because I dealt with that constantly in my previous job. I guess another thing before we get into. You have a strategy for modern test data management. Before we get into that, the other thing I've seen people get hung up on as well is the requirement or the cost of storing data. So they'll make a copy of production in a database and it just kind of wonky that way too. So not very elegant solution there.

Yeah, I mean they've created another industry. I mean you get your actifios out there and you know, some really cool products and actually the database Vendors themselves have come out with some very, very cool tech which lets you spin up these databases, which are in effect mostly virtualized very quickly and you're not using as much space. And that's all well and good. And you know, the original argument for the subsetting was that it allowed you to just have less space, which is fine.

But the problem is, even if you make a full copy of production and you mask it to death, if you think about all the tests and developers and people that are working in those environments, suddenly at 9am this morning, we had a fresh dump. All of your lovely data that you happen to be working with has now been reset back to.

Production data from two days ago. So you might have been busily creating a nice account history where you're trying to test if there's more than three withdrawals this month and suddenly that account has vanished. So the whole agility goes out the window. I've seen this throughout my career, sort of starting in the 80s back in good old Wall street when we had a production drop, a drop of data in, which was kind of the only way we knew about that. It would bring the development to a virtual standstill for about two to three days while everyone got the wheels back on. And so there are some significant problems with that kind of approach, even with the wonderful technology that's, which is obviously the, you know, the spin it up wing, you know, bang up a database. You know, it's great for regression testing and it's great for something like a Jenkins, where you could sort of spin up a copy, a full size, you know, two terabyte copy of production data, run all your regression tests, check that the results haven't changed at all, and you're, you're gold. But if you're trying to use that anywhere further upstream in the development and testing worlds, you're kind of in trouble because it just causes so much disruption.

Absolutely. So, Hugh, I was lucky enough to get a demo of one your awesome solutions for test data management. You have a really unique approach to it. So can you tell us a little bit more about how you see test data management and more of a modern type of strategy that people can use? Yeah.

So I think, I mean, first of all, you've got to think about, you know, quality. You know, what if, what is quality? I'll come back to the phrase coverage a little bit. Maybe we can start with that. You know, there are, there's a lot of people say, well, you know what? We have good coverage. You think, okay, what do you mean? Well, we Run a lot of tests and they all pass, great, well, what coverage you can get. And they might then say, well, we're getting 40% code coverage, okay, and you say, fine, but does it actually work? And they say, yeah, for those test cases it works. So you say, well, okay, well how much of your requirements are you covering? And then all of a sudden they go quiet because they don't really know. They've got no relationship between a requirement back to the coverage. So you're sort of deciding which logic gates we're testing randomly actually, what people will tend to, tend to think happy path and then they'll tweak the happy path so that, you know, and if you look at feature files and so forth, which is I guess the closest people come to model based testing, which is sort of, you know.

But the better way to think about it is if you try and model the requirements, then you're going to model in effect a process. And that process could be entering data into ui. However, more likely nowadays, I suspect most of the people are listening, are spending an awful lot of time actually modeling data flows as data flows through a system. So if you're in the banking industry now, you have an app, great to test an app doesn't take too long, to be honest. It might have 70, 80 drop downs, who cares? But if you look at the complexity of actually moving data as it travels through a system and you look at some of the investment banks, you could be talking two to 3,000 systems and one trade might in effect touch 600 systems. So what you've got to do is you've got to start thinking about modeling the data journey. Now a data journey is I start off with a swift message. Now, the swift message contains a whole bunch of data. The first part of the data journey is to make sure that it gets rejected correctly. So we come back to our good friend negative testing, which people spend an awful lot of time ignoring. And actually most bugs will come because bad data sneaks into a system. So if you can basically think about a model of the validation that I need to perform to kick out bad data, you know, if you think of a, you know, you sort of say, right, you know, no hex values, no spaces, must be greater than 8. If it's an IBAN number, it must be, must be 22 long. Unless it's in these seven countries where it should be, should be 21 long. And you create a model. Now a model is if people have used, you know, bizarre visio. It's a model description of the logic now the advantage of a model is that again I've probably said this before, is that the human brain is quite a small cpu but it's got vast great graphical processors. So people are much better at looking at pictures of things. It's much easier to understand how things flows. So you move back into the wonderful world of drawing pictures of what a process is. To most Sprint rooms you'll see lovely whiteboards with lots of pictures. We're kind of doing it anyway and what we've done is we've taken that idea of drawing the pictures to say, well let's think about a data journey. Think about how the data flows. When does it become, you know, a rejected trade, when does it become a reversal, when does the currency calculation get occur and what is the result of that transformation once it goes into the currency calculation engine. So we've used kind of the idea of modeling. Now when it comes to modeling engines, the good thing is if you think about what I've just said, a model is great because it allows people to communicate. They can look at pictures. What you want to do is to get the maximum value from that model. Now what we've done this time around with our new company Curiosity software is we've said well out of that model we can create an optimized set of test cases. Now an optimized set of test cases is basically mathematics. To calculate that you are testing each decision correctly. So that's the first thing and there's a whole science associated with that. We try to make it as easy as possible.

Second thing is the automation scripts. And I think when I met with you.

Wow, it was an HP conference probably about six or seven years ago. It was, wasn't it? So yeah, so you look at that. We basically, if you think about automation scripts, they all kind of a well formed automation framework and that's what we, you know most automation frameworks come across pretty good. Now you know, you have a nice set of libraries at the top, you then have various headers, you have all your page objects, you have some pretty good version control ins git and then you have all of the individual test cases, whether they're data driven and we would look to in effect push those automation scripts out. Now what we'll also do, and this is the kind of cool stuff now which comes back to your point when we first started talking, which was about the idea of, you know, what test data do you need for an automation script? What most people did is they would have a piece of SQL hidden lurking around inside the beginning of a script, go and find an account, they might have an Excel which they go and look into to find an account number with the characteristics that they care about. So what we've done is we've said, well let's take that out and we've created a test data catalog. So all around the organization you will find people have pieces of SQL, perl scripts, UFT scripts, whatever it takes to go and find the data. And they will have created small automation components. So we said, well, why don't we expose those in a catalog so that anyone, developer, tester, product owner can just say for this particular test case or scenario, I need this type of data. What we do then is we then attach at the beginning of each automation script that we generate or you could just cut and paste it in yourself. We'll say for this automation script to run, I need an active account in a different market than the core market. Now that could be a piece of SQL, it could be a lookup in on the mainframe, it could be a rest call. We will then, as part of the automation script just run that, that will go find an account number. It will allocate that account number to make, you know, it'll make sure that that account number has been, hasn't been used for another automation or manual tests because then you get pollution. And it will say, thank you very much, I'm going to give you account number 72. The automation script then knows to use that account. It will then have all of the correct characteristics required to make sure that that test runs correctly.

It reduces the amount of these false failures dramatically. The advantage also is that if you then make a change in your data catalog to the definition of an account in a different market with the same beneficiary, you change it in one place. The next time the automation runs, every single script would pick up that new definition. So it's, we've, we've tried to be much more, you know, like I said, about having better amps and better guitars, you know, you just become, I know I'm, you know, quite old, but most of the company, they're all up sort of in their 20s and they have a lovely flexibility of mind of trying to solve problems. And I think what's happened is that people now are much more open to solving problems in the quickest way and the most elegant way possible. So all of my lovely developers will just disappear and say, well, let's go and look on git. Why would we code this up when there's this lovely thing called git? Out there. They might go and look at some academic research, they might take something in Git and tweak it a little bit and then they will assemble that up so you can assemble solutions very quickly to solve most problems. If you think about that as an automation problem and what we now think of test data is not an environment problem but a test data automation problem. And that's. We've kind of repositioned all of our websites to talk about test date automation, you know, as. Because that's what it is. You're automating the provisioning, the finding and the provisioning of the correct test data to make sure your automation runs seamlessly, effectively and also covers much more of the requirements than you would if you just copy a script and tweak one thing. Copy a script and tweak one thing. That's really the kind of goal of what we've been doing, you know, with Curiosity software over the last six months. And I think.

Quite often people say to us, we don't believe you. That just sounds too good to be true. And the reality is we've been doing this for quite a long time and we've seen most problems and I think it is too good to be true unless you have a flexibility of mind, a willingness to change and also a curiosity to be able to go and explore new approaches. I think if you have those three things then you're cooking on gas. You can really, really make some very. You can make some changes and some improvements really quickly. It's not that hard, you know, I.

Guess the resistance a lot of people might have is if it's a vendor, they're like, well, I have an open source solution, so this isn't going to work for me. I need to start from scratch with the solution. So does your solution work with existing frameworks or does someone have to learn how to model and go through all that? Because you get pushback with that as well.

So yeah, so we have Test Modeler Now Test Modeler is a cloud based solution. It's a modeling tool, you know, bizarre, et cetera. You can go online, you can sign up for it, you know, so for a few hundred dollars you're generating code and test data and covered sets. So it's not much money. But I would say coming back to your original question is wherever we go, there are numerous automation frameworks. Now most people seem to have a pretty well formed Java Selenium framework for front ends. A lot of people will have something, you know, like maybe a, you know, a C Sharp. Rest assured framework for API testing. So but those frameworks tend to be reasonably well formed and well structured. So what we have to do is we have a three point integration. We can make sure we can scan your, your code page objects, which is your definition of the things you're going to interact with. And we have a scanner that does that. Those make them available for modeling. You do your modeling and then we have some code snippets and we will then generate the code back out. So it's not the hardest thing in the world and it allows you to retain all of your investment in your, in your open source frameworks, you know, without having to, you know, go and buy with vendor lock in. It also allows you to move from framework to framework because you think about automation, you don't particularly care what code is generated with a click button. You know, it's just a click button. It's as long as you know the name of the button, you should be able to click it. So if you change the code snippet to go from you know, Lean FT or UFT or whatever the latest today's technology is, then all you're doing is changing that in you know, one location and changing a git endpoint and bingo, you've moved from one technology to the other. So I think what we're trying to do is to make it more open for people, to make people move out of, move from technology to technology and think more. You know, you should probably have one or two really good test automation engineers. They should be focusing on building the most amazing Test automation framework, LinkedIn to DevOps with building dependency maps. They shouldn't spend their time building automation scripts. That's a complete waste of their talents. The people should be designing the requirements in effect create those automation scripts. They don't know they're doing it, they just know that they've just dropped click button on a screen but they're out after that.

It changes the kind of paradigm in terms of how people work. You've got people like product owners and testers, you know what were manual testers are now building models. The test automation are really working on creating the most state of the Art DevOps framework that you can.

There's so much scope of good things that you can do the shift right stuff, go and get the information from how the production users are using it, auto generate scripts out of that using the models. There's just so much stuff that the test automation engineers can be doing rather than sitting there coding scripts.

So I think that's kind of one of the changes that hopefully we're going to make in the industry.

Absolutely. So, Hugo, I guess another pushback I get is anytime there's auto generation going on, people don't trust it. So are you using some sort of machine learning or how can people trust the intelligence of what's being generated for you?

Okay, yeah, I mean, we can reverse. I mean, first thing we can do is we can reverse engineer what you've done. So if you, if you've already got some automation scripts, we can run them, record them, and then build a model out of it. So that's kind of a proof point. And quite often we will find that manual and automated testing is probably covering about 15% of the requirements. And you might have spent years building that framework up. So you've, you've certainly got enough, not got enough negative testing.

You know, and it's very much happy path or it is just record. It starts off as a recording of user activity. They parameterized it up a little bit and that's kind of what they think of as kind of automated testing. So that's fine and we'll start there. Now, the next thing is to think, well, I have to build some test cases. No matter what I do, I have to actually end up with a script. You could almost say, right, in the morning, I'm going to manually code a script and I'm going to run it. In the afternoon, I'm going to create a model using Test Modeler and I'm going to generate 10 scripts. Now, the scripts are physical artifacts. You can look at them, you can actually say, well, okay, let's look at what this script is. And this could be a Gherkin script. It doesn't really matter. You can look at the script and you say, yep, correct, correct, correct, correct. Oh, hang on, something's wrong there. Now what that initially means is that you may have set up our integration with your test framework incorrectly. You know, you might have something where the same code working against an Android and an iOS and a Windows Phone and you look at the script and it's actually incorrect. So what you have to do then is say, well, we may need to move that back up into the model, back to the model, you change your components and bingo, you fixed it. So it is not magic. Like I said, you have to spend some time building those scripts. Now, once those scripts are run, what we also do is we then take the run results and we have a small API that we just need at the end result of your framework. And that will then tell modeler what what's happened with the scripts? You know, it failed at this point. That then overlays onto the model the fourth useful thing, which is where did it fail? So that's also very useful because you can then think about maybe I need to test this particular node more thoroughly and I've just done a basic test on it, but actually it looks like the security module or this password validation routine has got some serious problems. So we might think about generating them. Now that's more human interpretation of the test results. Now the other thing to think about there is, and I'm not going to use the word AI. AI is much malignant, a much misused term. I think most vendors use the word AI. And the problem with AI is that you can only really make intelligent decisions or artificially intelligent decisions. If you are sure of the provenance of the data that you're gathering and that you're actually gathering the data correctly in a systematic manner, then you can apply, you know, expert systems to apply rules on top of it. And those rules can either work on what's happened in the past or what you think should be happening. So we have plugins to all of those. Again, I'm slightly hesitant to say use AI automatically. I mean we can do things like, you know, we can have salt, we do have self healing automation scripts. So if we detect that a column has changed on a fit on a web page, we can go and look for a similar column and then automatically plug in. There are tools around like that. But where it's more interesting is to think about how you auto generate models to test what's going on. And you can do that sort of using rule based systems. So a lot of testing nowadays is actually if you think about an ERP package, you get the latest release and I'll just use the word SAP and it applies to all of them, Oracle, World, PeopleSoft or whatever.

In theory you should just get the new release, it's been tested by the vendor, you plug it in and everything goes. But no, everyone has to test their own configurations and their own rules rules because they've created their own sets of rules and configurations which need to be tested every time there's a code drop from your ERP vendor. So you could get into based on my rules inside my ERP vendors which are configured by myself. Let's auto generate a set of tests to cover the different combinations of currencies of returns, how often a product is returned. If a product isn't delivered, how many times do we retry before we do a refund all of those things are actually sets up in rule packs. What are you going to think about is auto generating the model out of the rules and then from that you can then auto generate your test automation scripts, or potentially just, maybe even just test driven test automation scripts. A lot of automation scripts are actually just spreadsheets containing complex combinations of data and pushing those through as well. It's not magic, but. But I think we have enough of the components for you to do that. But again, I come back to the test automation engineer. I think it's up to the test automation engineer and maybe the head of DevOps, someone who's got a good holistic view over the entire development process to come up with the best place to apply these tools, which are incredibly powerful, but if they're in the wrong hands, just cause more trouble than they're worth.

So Hugh, any thoughts on how containerization helps or hinders test data management?

So we're sort of saying, if you think about, I talked about the commoditization of test data management with subsetting and masking and a little bit of synthetic data generation, but what we're finding nowadays is that it's all changed, everything's gone out the window. If you look at Docker, if you look at things like Solar Kafka, big data, all of these things are throwing up some really, really interesting test data challenges. And that we have decided that no challenge is too hard for us. You tell us what you think is too hard or really complicated or it's going to take you forever to do it. And it could be even just generating complex xml, which, you know, lots of documents now are very complicated XML documents. You know, you've got to test the test that they are processed, understood and can be, can be viewed correctly in numerous platforms. But this whole containerization is actually, I think it's a fantastic opportunity for being able to. And with the advent of being able to spin up these VMs, these Docker images, these Docker clouds or whatever you call them, being able to ally that with the ideas of coverage and virtualization is actually a massive opportunity to be able to create a really dynamic test data environment. If you think about why we're doing this, everything we're doing is generally to make our systems better so we can compete better with customers, with our competitors. Now if you think about what Docker containerization can do in terms of speed to market.

It is stratospheric in terms of that because you can parallelize so much and everyone's working on specific APIs now. If you deliver those APIs and each API is 100% robust, 100% well tested, can be entirely virtualized, other people can test against it. That allows you to just, just roll out new features at the rate of, you know, at a really rapid rate. And I genuinely think that this is the new frontier and this is where it's all going to be at.

Okay, Hugh, before we go, is there one piece of actual advice you can give someone to help them with their test data management efforts and let us know the best way to find, contact you or learn more about your awesome solution?

Yeah, I mean, I think actually, I mean it's actually a really simple solution. Piece of advice is if you look at a requirement, just read it and if you read through that requirement, you will see that actually people mix up data terminology inside a business description. If you go through and you change your requirement document to have just a section on data, and that could be the data that is required to run this, that this requirement relies on. It relies on.

My employee table, it applies on my employee rules TABLE TABLE Great. When you refer to something that's in the requirements document that's linked to the data, that makes life so much easier for your test data automation engineer, because they then have in effect a blueprint on how to create the test data for the people that are going to develop this system. And you should think about test data as something that happens before you even begin development. So if you've got a good set of test data, if you look at someone like Richard Jordan at Nationwide Building Society, they have, their job is to sit down at the project inception, draw out of their users what they, what they need from a test data point of view and then they will beaver away and create them a really rich set of test data and development data. And that is probably the best thing you can do. And you can do that with your own technology. You don't particularly need anything new because it's just a bit of a mindset change in terms of how you do that, in terms of getting in touch with us. Just look at Curiosity Software island, just search on the web, we've got a few websites on there. Or just look me up on LinkedIn. Hu price, huw price. And that's probably the easiest way to get in touch with me. And then we just chat on LinkedIn. And then, you know, if you want to do any demos that you can also do, there's plenty of trials online if you want to play with it. The thing we really do like is working with customers on hard problems. So if you've got an interesting problem, let's do a web meeting. Let's do a sort of web workshop and try and genuinely just kick it around to look at the best way to solve some of these really interesting and hard problems.

Thank you Hu for your data management awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com aaliqautomation282 and while you're there, make sure to click on the Try it for Free Today link under the Exclusive Sponsor section to learn all about Source Labs awesome products and services. And if you really want to kick up your automation game in 2020, you need to get a ticket to the biggest online conference in the world dedicated 100% to helping you succeed with creating automation awesomeness. And that is the Automation Guild. It's going to be the fourth annual event. We have some of the best experts in the automation field joining us, sharing a tip tool technique, a best practice that's going to help accelerate your testing efforts in 2020. You don't want to miss this event. Head on over to automationguild.com and register today. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to test guild.com for full show notes, amazing blog articles and online testing conferences. Make sure to subscribe to join the Guild and continue your testing journey. This has been a Joe Calantonio production.