232: HipTest Living Documentations BDD for 2019 with Laurent Py

Hit it.

Welcome to the Test Talks podcast, the place to go to geek out on software testing. And now your host whose mission is to help you succeed with test automation, Joe Colantonio.

Hey, it's Joe. And welcome to another episode of Test Talks. There are many benefits of bdd, but most folks completely ignore them and just focus in on the automation piece of behavior driven development. Living documentation as well as communication are often completely overlooked. So that's what we'll be test talking all about today with Laurent Pie, Founder of Hip Tests. So discover how to take your BDD to the next level in 2019. Listen up. Speaking about bringing your automation testing efforts up to the next level in the new year, make sure to check out Automation Guild, the largest online conference in the world. Dedicated 100% to helping you succeed with creating automation awesomeness and being the hero of all your automation efforts in the new year, as well as make you the hero of all your team's testing efforts. It's really going to help your career as well. So definitely check that out. Register today, head on over to automationguild.com and actually one of the sessions we're going to have at this year's Automation Guild is Angie Jones will be here talking all about the devil in the details. Good gherkin for complex automation frameworks, so you don't want to miss that. Register today, head on over to automationguild.com.

Test talks is sponsored by the fantastic folks at Smart Sauce Labs, the cloud based automated testing platform that eliminates the need to maintain your own selenium grid and test infrastructure. Try it for free today. Visit testtalks.com and click on the sign up now link under the homepage sponsor section.

Hey Laurent, welcome to Test Talks. It's great to have you back on the show. It's been I think two years episode 114 and it seems like a lot has been going on with HIP Tests. So that's what I like to talk about today, Hip Test. Also I think you have an updated coming out around your living documentation feature, so I'd like to get into that. But before we get into it, I think there was a huge announcement, I don't know, maybe a few months now already about you being acquired by SmartBear. So I'm just curious to know how does that affect HIP Test? How does it help your users of HIP Test?

Yeah, so back in May this year we've been acquired by.

By SmartBear. So just as a reminder, I co founded the Hip Test company four years ago and since we, I Mean we were growing very well with the focus on behavior driven development, agile test management, and we started to have a discussion early this year with SmartBear and we found that.

The challenge, the opportunity was super interesting for us because of the resources they have, the other products that complete what IBTest does. So products like TestComplete for test automation, SoapUI for the automation at the UI level, and also cross browser testings. So these integrations, these use cases that we are now able to address as part of a Smart Bear family and we were not able to before, were the reason why we said yes. And now five months later, I think it's great news for our community because we have almost double the size of the R and D, so we have more developers. It means that we can execute faster on our roadmap and once again we can also provide a smooth experience for people who want to test and automate at the API level or GUI level with the integrations that we now provide with the rest of Smart Bear portfolio. They have the best of two worlds, a cutting edge platform, technology for continuous testing and many awesome integrations.

Awesome. So for the folks that didn't hear episode 114, this is the first time they've ever heard of HIP Test at a high level. I know we covered this in the previous episode, but for folks that haven't heard, what at a high level is HIP Test?

Yeah, so Hip test is a testing platform that helps software teams to test their software the feature continuously from ID to production. So so basically we help first the team before starting development implementation to create a shared understanding and have a clear definition of what the feature should do. This is where behavior driven development, this approach, BDD approach is very useful and with iiptest within iiptest, you can create easily BDD scenarios if you are a product owner. We provide some capabilities like.

Test step auto completion. So we help the team to create scenarios using a consistent business terminology. Then once the team has created a shared understanding of the feature, we generate executable scripts for more than 20 frameworks now. So we can generate executable scripts for Ravi RSpec Python if you use Gherkin based automation framework like Cucumber or specflow, we can also generate the feature files for these frameworks. So whatever the language and let's say test execution framework the dev team is familiar with, we can generate the script for them. And the last part, once we have tested the we have checked the implementation, check that it meets the business expectations, then we generate living documentation and that's Part of the announcement that we have just done around living documentation better. So we can turn out the automated test into a documentation of the product that is always up to date. Because the documentation is generated from tests that are executed as part of the System CI CD pipeline. And you are sure that what you read in this documentation reflects the true behavior, the actual behavior of the application in production. So that's how we help these teams test continuously the features from the IDE to the code and then what happen in production.

Awesome. And I guess one of the killer features of using a product like yours is if you work for a large organization and you have a bunch of different teams creating feature files, a lot of times you get duplications. So I believe your tool also helps with. I don't know if this is still the focus it's been two years ago. Help you avoid duplications because it's almost like you can write your features within your tool and it has IntelliSense that lets you know of existing features and if there's an update, you could just make it to one location automatically. Updates and other locations that still. Am I understanding correctly?

Yes. And that's of course still one of an area of investment and a focus for us. So yes, as I mentioned, creating scenarios with a consistent business terminology. So reusing the same test steps, the same terminology, instead of writing you have the same intent, but you write it in a different way. And so. So you create some duplication.

Yeah, SIPTest definitely has been designed first for that. Helping teams to avoid duplication, create scenarios with this consistent business terminology. We have also bring that one step further because now we help teams from different projects to share this business terminology. So let's say you are part of a large organization and there are some components that that you reuse across multiple products. So for these components you will have a defined business terminology that you will use in your own project. But let's say another team working on another project and using the same components may want to reuse the exact same business terminology. So now that's possible across multiple projects with versioning because sometimes you will need a version that is different from one project to another. So that's one of the big feature we have released in the product just a couple of weeks ago.

As I mentioned a few times, last time we spoke I think was in May 2016. And a lot of times when I speak with folks that have an automated tool that they've created or a tool in general, a lot of times they pivot or they improve based on customer Feedback. So is there one theme you've heard since 2016 that made you maybe change a direction or focus in on an existing direction that customers really have been struggling with? You think your tool could now help better with since you've been focusing on that area the last two years?

Yeah, definitely. Two directions, one of them being Git integration. I mean native git integration. So being able to reflect in eptest the workflow of things that are happening in your git repository. So either you use GitHub, GitLab or BitBucket. And the other big theme we've seen consistently popping up is the integration with jira. So having the ability to do more and more things directly inside jira. That's very interesting that you mentioned this feedback. How important is feedback from customers? Because I'm a serial entrepreneur. IBTest is my third startup. I strongly believe in a lean startup approach. So we do small software increments. So we have an idea, we build a small increment, we deploy it as quick as possible and we learn from that. And the philosophy behind iptest, the philosophy of continuous testing is exactly that we first test did with bdd with customer interview, we develop a small increment and based on the script that we generate the automated test, we check that the small increments meet the business expectation and then we have the living documentation that show you how the application behaves in production. And we also had to that to this living documentation some product analytics so we can learn from the usage, we can learn from the impact of this new feature in production. And based on this.

Data, so on this insight then we can inform the new Sprint, the future development. So this is in the DNA of our development process. But that's also what we wanted to provide to the user with HIP test. With this continuous testing platform.

And approach, I mean life doesn't stop at the CI. What I usually see with dev teams, when I say dev teams, I mean not only developers, but testers, product owners. It looks like once the CI pass and the software is in production, you did the job. But not yet. We need to make sure that what we did really makes sense and have an impact on the customer journey. So that's as important.

Awesome. So anytime I hear analytics now, I automatically think AI machine learning. I just see now HIP Test, AI machine learning, analytics functionality. So that's a big theme that's been going on the past few years. Is that something you see maybe on your roadmap, using this analytics to give insights to your teams automatically based on machine learning or something like that?

Yes, definitely. So that's part of the the backlog and we have started first step in this implementation. So what's interesting, if you imagine the new living documentation page of Heap test in a single place, you can see the content of the feature. So what are the different use cases for this feature? You can see the history of the feature, when was the feature created, when it has been updated, and you can see some usage metrics we integrate with mixpanel segments IO so there are many interesting and great product analytic tools. So when you combine the test, the living documentation, the test results with product analytics, you can realize that one feature that has been developed and was successful maybe one year ago is not that much use anymore. But when you look at the test, you have a couple of tests, unit test integration test, BDD scenarios for this feature that takes a while to execute, one, two minutes. And you know we always struggle with test execution time. We want to reduce test execution time to provide faster feedback to developers. And using this analytics and some machine learning techniques, we can definitely help to prioritize execution, maybe in IF tests. So far I think we have more than 8,000 automated tests. So all the pyramids that are executed every time by the CICD and we have streamlined the process so now it takes just 20 minutes. But 20 minutes is still long. And as the product grew this time, execution time is increasing. So a way to reduce that is use prioritization, use machine learning. And maybe for the feature that is not much used anymore, maybe we can avoid to execute this test as part of every new build, but maybe we can just execute the test once a day or once a week. And that's typically where machine learning can be helpful.

Awesome. So this may be an obvious question, Laurent and that is HIP test only for someone that's using behavior driven development.

So no, the way we fought and we designed the platform was we know it's a long journey. The focus for us is definitely continuous testing. Teams that use behavior driven development do massive test automation. But we know it's a long journey. And we have customers that start from the Excel sheet. So they import Excel, the excel sheet in iptest and they start with just manual test execution. No BDD. IBTest is also an agile test management platform, lightweight with some interesting but basic reporting capabilities. Same in terms of test planning. I mean as part of an agile team, you don't need huge test planning features like the one you have in HP alm, for example. So we provide these capabilities. That's the beginning of the journey. But what we do is helping these customers. These teams elevate the performance, progressively start creating the test case with a consistent business terminology. So with auto completion, step auto completion, and then once they have a better design of the test cases, we can help them with the automation so we can generate the scripts using HIP Test publisher. So step by step they can increase their maturity and progressively start adapting bdd. I mean bdd, like Agile, this is what we've seen in the past is not it's a transformational approach, it's not something you're going to adopt within a day. It changed the way people collaborate, it changes the organization, it changes also the process of the whole team. So.

It takes time for this reason because it's a long journey. In the DNA of PIP test, we have this progressivity from agile test management to BDD and test automation.

Maybe a few years ago I think behavior driven development was the hot thing, the hot trend. So I know you speak around the world, different conferences, is BDD still popular? Are people adapting it? Any thoughts around where the state of BDD in 2018 going into 2019 we.

Definitely see more and more teams adopting BDD. You know, it's like agile a couple of years ago they were the pure agile natives and then everyone was agile. And we see exactly the same thing with bdd. So now in every organization there are not all the teams but at least a couple of them that use bdd. And we see the same pitfall as the Agile with Agile adoption. I mean you use the given when then syntax and you say, okay, that's great, I do bdd. And I think there are more and more teams using BDD or at least thinking they do bdd, but a lot of them do it the wrong way. So one typical pitfall that we see is, and that's one of the reasons why we've created this, giving documentation is the first. So the dev team says, okay, we're going to write our scenarios, we've given one, then we're going to use cucumber and we'll do bdd. But when you do this, maybe it helps in terms of test automation practices, but it doesn't help collaboration. And BDD is first foremost about collaboration. So having product owners, business users and developers, the tech guys inside the same room, discovering the new feature to implement, creating a shared understanding is key. Everyone should do that. So either you call that BDD or not, either you use.

Gherkin syntax to write your scenarios, acceptance criteria or not, but you have to create this shared understanding. 100% of the high Performing team I've discussed with can be a team at Twitter, Facebook.

Atlassian also is a great example. When you look at the team playbook, one of the first things they do before investing a penny in development is create a shared understanding of the feature. That's key. This is where techniques and approach like BDD can help. There are many ways to do that, but yeah, BDD is one promising approach for that. So don't miss the opportunity to discuss with all the team and to create this understanding. Then you can use cucumber, you can use other frameworks, it doesn't really matter. It's not because you use Cucumber or specflow that you do bdd and that's one of the most common pitfall that we see.

Awesome. I definitely agree. A lot of times teams get the beginning and the end of BDD wrong. So they don't focus on the communication on the front end and they don't focus on the end result for the living documentation. They're just focusing on the middle automation piece. So you talked about communication. Another approach I think people forget about is if you create your BDD correctly, the output, your live in documentation can almost serve as either a guide for a new employee, how to use your application, or directly for your customers. So you don't have a separate help document you're writing and bdd. So can you just explain a little bit more what live in documentation is?

Yeah, sure. So if you have created from the beginning your examples, your acceptance criteria for a given feature, the right way it will be written so using a consistent and a common business terminology, that's the shared language between the business and the tech guys, so everyone can understand it easily. And because then these acceptance criteria, these BDD scenarios are automated, so they are turned into automated tests. They are executed as part of the CI CD pipeline. We can generate from the automated test and this acceptance criteria in business terminology, defined in the business language, we can generate the documentation of the application. So for example, for me, I used to be developer 15 years ago, but I don't do development anymore unfortunately. And when I look at a new feature of heap test how it behaves in production, I just open my living documentation feature of Iiptest and I can see how the feature behaves, I can see the different use cases, what works, what doesn't work, maybe what broke the build last time. So what works in production? And it's super easy to understand. I can also look at the history of the feature, I can see when the feature was created, when the first time the test was executed as part of the CI CD pipeline. I can see when the feature was updated because maybe we have added a new scenario. So new use case for this feature. So I can see the evolution, the history of the feature. And once again it's live because it's executed every time as part of CI CD pipeline and it is stored in your git repository which is one of the source of truth. So we are sure that it's aligned with the code itself of the application. So that's the value of living documentation. It's a great help for communication and you write to mention the two different use cases. So we have just hired a couple of new developers in the team and one of the way for them to discover what Hyptest is and learn more about the product was not only to play with the product but to read documentation, the living documentation of IPTest. And same goes for the support. So now we start to have many, many users. So we have a support team in app chat in the product, the support team that discuss and helps our users and when they have a question about feature or behavior of the feature, they can rely on the living documentation of the application of hittest to say this is the nominal behavior. So that's the way Heat test supposed to behave and behaves in production.

Awesome. And so one of the newer announcements I heard recently is around a new beta program around Live in documentation. So what does that mean? What is the beta program? Is it a paid program? And what do people get when they sign up for the beta?

So it's not a paid program. Basically there are two workflow. When we think about bdd, the first one that we have implemented so back in 2015 is the top down approach. So you are a product owner, you are a tester, a developer, and during the discovery phase or the free amigo session, you create your acceptance criteria and then the dev team automates it and we can generate the living documentation. So everything starts from the discussion and once again that's the right way of starting adapting bdd. But what we've seen in the market is that there are many, many teams that started BDD with test automation. It's an important part of the process, but it's not the main one. So what can we do to help these teams that have. And honestly there is a huge value in these scenarios. The problem is they are locked in git, all the value is locked in git and they cannot collaborate because it's hard to ask a product owner or business user to use an ID and to use Git in order to read these feature files. So what we've done is started to implement the opposite workflow. So we start from what exists in git and with this new beta program we enable the dev team and the business user to unlock unleash the value of what they've created this asset in git. So we create documentation from the feature files that are in git. Either it's GitHub, GitLab or BitBucket. And we show this living documentation in a nice web interface. So that's the one of eptest as you know it. And it's much easier now for product owners and for business users testers to look at this documentation and tomorrow to add some comments. So be able to discuss around this living documentation. So that's one of the things we are investing on right now.

Okay Laurent, before we go, is there one piece of actionable advice you can give someone to help improve their Live in documentation or BDD test automation efforts? And let us know the best way to find, contact you or learn more about the Live in Documentation beta program.

Yeah, I think the first and main starting point for if you want a try adopt BDD is to have the conversation upfront before starting investing a penny in development with all the stakeholders. So product owner, tester and developer, this is where we create most of the value. And for this we can use Postit, we can use tools like keep test, but tool is not the most important thing. Having the conversation and creating this child understanding is definitely the biggest outcome of.

BDD and Lorraine, is there an easy URL someone can use to get a hold of that beta program?

Sure. So you just write hiptest.com so go to the address hiptest.com living documentation and you'll be able to access the beta program.

Thank you Laurent for your BDD awesomeness. The links to everything and value we covered in this episode. Head on over to testtalks.com 232 and while you're there, make sure to click on that. Sign up for a free trial link under the exclusive sponsor section to learn all about Sauce Labs awesome products and services. And there's also a link there for you to click on to go to automationguild.com to register today. So that's it for this episode of Test Talks. I'm Joe and my mission is to help you succeed with creating automation. Awesomess as always, test everything and keep the good. Cheers.

Thanks for listening to the Test Talks podcast. Head on over to www.testtalks.com for full show notes amazing blog articles and other automation awesomeness.