#24 – Google Optimize Sunsetting & How Do You Measure the Impact Of a CRO Program
1 Mar 2023 · 31 min
Khalid Saleh and Simba discuss Google Optimize shutting down and how to measure whether a CRO program actually works. They cover tool options like VWO and Optimizely, what to look for when choosing one, and four ways to track ROI: side-by-side testing, data modeling, customer satisfaction metrics, and conversion analysis.
Questions this episode answers
Episode #24, recorded/created 2023-03-01.
Free Google Optimize supported 5 concurrent experiments.
About 500–550 milliseconds (Google Optimize test inst script ≈550 ms).
FigPi stated ~150 ms typical; one site showed <30 ms for first-time visitors.
VWO announced free up to 50,000 monthly visitors; FigPi offered free up to 75,000 monthly visitors (per episode discussion).
Chapters
My name is Khalil Slade and I love to talk about conversion rate optimization.
I'm Simba and I love asking questions about conversion optimization.
This is CRO Live Hour, a show all about a B testing, experimentation and conversion rate optimization.
Each episode, Khaled amazes me, answering some of the difficult CRO questions, dropping insights like it's no big deal, while pretty.
Much every episode will take on a new set of conversion rate optimization questions that are not easy. We will talk strategies, we will talk process, and we will talk tactics. Simba will be bringing all the questions.
Oh man. I bring tough questions like do a B testing results fade over time? How do you go from low to high testing velocity? How do you measure the success of a conversion funnel? And how do you align your CRO program with a growth strategy? Yeah, Khalid, these are very, very tough questions.
Yes, they are. But we always answer them here. And if you love conversion optimization like we do, and certainly like Simba does, subscribe to the CRO Live Hour podcast today, wherever you listen to your podcasts. I had this experiment of removing social media from my phone and removing email from my phone and I think I can comfortably say that is a failed experiment just knowing what I do. I need to be on LinkedIn. I need to be on email all the time. So I am going to be bringing them back. I did last 26 days without any of this stuff.
That's good. At some point last year I was also thinking of removing the Facebook application from my phone, but because of the DGMG group I couldn't. So that was the only reason why I was on Facebook. Exit. Is it called Exit 5 now?
Yeah, Exit 5. Yeah, I know you changed the name at some point. I am not sure why. I guess it's exit 5. 9 to 5 job. So that's what he named it. But how was your week, Simba?
I had a good week, but it was so, so, so, so, so busy. Like I was so busy. I mean I had like a lot of things going on. I didn't anticipate this last week because last week I knew what I was going to work on this week. But then a lot of things just came up. So focus on those. So otherwise I think it was just like a good week. How about you? How was yours?
It was also very busy. Crazy busy actually with the announcement and we were talking about that earlier. That came out of Google that Google Optimize is going away. That made for very hectic time at figpy. Exciting time at Figpy actually, because we have been Releasing new features week after week. We had to re architect the application in the last six months just because of the amount of companies that are jumping on it. So really many, many things had to be redone. So we've been releasing new features and then this came which meant that, oh, there's more stuff to work on. Invest work had picked up in a crazy way, in a way that is just very strange. We went from, oh well, we might talk to like, you know, four or five companies a month into. I think our sales team is so busy now with multiple meetings every day. Very unusual. Loving it. I'm not complaining. I am not complaining. If you run an agency, you know, kind of the ups and downs of running an agency, but there's just so much, so much demand. Everybody is trying to figure out a way to improve their site conversion rate. So it's been exciting, hectic. And I get up in the morning very early. Usually I'm up by 5am, sometimes 4:30am that means also that I sleep early. I mean if you get up at 5am usually by 9, I'm just wiped out completely. There is no energy to do anything. I had to stay up until 1am couple of nights this week. So, you know, it was interesting trying to kind of manage all of that. Hopefully things will calm down. I don't think next week is going to be calm, but I think the week after that should be a lot better. Smooth sailing.
Yeah, I think for you that's also like, that's a good kind of a busy, you know, if you're getting like a lot of sales calls or if our sales team is busy, at least like that's good. Can't be complaining about that.
I'm not complaining. I am not complaining. So let's see. I mean there's lots of inquiries. What's interesting is we've actually have been talking a While with our CROs to upgrade some areas within our client delivery. We've always delivered absolutely amazing results. And I'm not saying that just because I'm saying it, but I've had a conversation with the CTO and one of our clients, been working with them since May of last year. So, you know, we're going on seven months, eight months and I always have those regular calls and I was asking him like, hey, how is it? He's like, it's like absolutely amazing. He's like, we went from 4% conversion rate to 5 and a half percent conversion rate in a span of seven months. It's one thing when you tell your clients hey, we've achieved these results. It's a whole other level when the company that you are working with comes and tells you, oh, you've done absolutely amazing work and we've done a joint case study with them. So that's exciting.
That's good. So today we have two questions to talk about or two topics to talk about. Obviously we can't ignore like the Google Optimize news, as you have already also mentioned that we'll be talking about it today. I would like to hear like your thoughts on that because you have used Google Optimize like since, since when it was introduced. I would like to hear your thoughts on how is it going to impact the CRO industry. And I've been reading a lot of comments from different CRO people around about this news. So it will be interesting. And you've been quiet about it on social media so it would be like, interesting to hear your thoughts on it.
Yeah, yeah, it's sort of interesting because everybody has been expecting the news to come that Google Optimize will go away. That was expected. It was just a matter of time. And there has been rumbling about that, I would say since September of last year. So people are expecting it. And especially with the push that Google have made asking companies to switch from Google Analytics universal to GA4, which they've set a deadline and they said by end of July everybody has to migrate to GA4, regardless whether GA4 is ready or not. And by the way, next week we should talk about GA4 versus GA Universal. There are so many questions and many people don't get it right and the cost of migration and what doesn't mean to migrate. So that's a whole other topic. But with the movement From Universal to GA4, it was just a matter of time before Google made that announcement that Google Optimize that they will be sunsetting Google Optimize. Finally last Friday they came out and they said no more Google Optimize, no more Google Optimize360. So even the free version is going to go away. And it's funny because they said we are going to continue to be fully committed to enhancing the customer experience of those who are using the product and the user experience on the website. It's fascinating to me as many people don't remember this few of us who've been in the industry for a very long time in 2007, 2008, and I might be mixing up the dates. Google had released Google Website Optimizer, which was the first widely used AB testing software back then. And I attribute the growth of the industry back then in those years from 2007 until about 2011, 2012 to Google website Optimizer because all of a sudden AB testing software became available to anybody. It was not a very user friendly software. Nonetheless, everybody had access to it and it's absolutely free. And you can launch AB testing and you can launch split URL testing and you can launch multivariate testing. And then all of a sudden you wake up one day and Google Website Optimizer was gone. It's like what happened? And all these companies that have built experimentation programs around it, they just woke up one day and it was gone. And then Google brought it back through what they called content experiments. And then they again that evolved into what we know nowadays as Google Optimized. But again you wake up one day or at the end of the Friday when you read an email or a message from Google to say, hey, Google Optimize is going to be going away as well. So it's interesting because Google Optimize, many smaller companies have relied on Google Optimize to run their experimentation and personalization programs and now is going to go away. Now there's two different versions for Google Optimize. There's Google Optimize and Google optimize 360 very much the same. There's minor differences, but the biggest difference that everybody talks about is the number of concurrent experiments. Google Optimized, the free version came with five experiments concurrently. 360 had no limits. I think Google Optimized 360 gave you a lot more data points that you can manipulate. And if you're paying $10,000 a month, yeah, it makes sense to get access to all that data. Now companies are faced with the question, well, there's some questions that are really not very clear. Am I still going to have access to my data? That's a big question. I think you will still have access to your data. I can't imagine that it's going to go away, at least not in the near future. A bigger question is, well, what do I switch to and what are the alternatives? And now you need to factor the cost of the alternative into your program. So that's a whole other consideration. What's the selection criteria that I'm going to use to select an alternative? There are probably 30 or 40 different testing software platforms out there. And choosing the right tool for the right business within the right budgets can be challenging.
That makes a lot of sense. But when it comes to choosing the right alternative or the right a b Testing tool. What are like maybe five things that you think people should look at? If you can give us six or 10, that's cool. But five? I know you can get to 10.
Oh, I can give, yeah, probably. I think you start with your requirements correct because your requirements and your specific business requirements are going to drive your tool selection. For example, at the very offset, if you have a mobile app, so you have your website, your platform, but you also have a mobile app and you want to run AB tests on a mobile app in addition to the website, that limits the markets. Not every AB testing software will allow you to do mobile app testing. So from the get go, if that's what you're looking at, that's probably. There are two players in that space that I would seriously consider. There's other players, but it comes down to Optimizely versus vwo. Those are kind of the two big players there. There are some A B testing platforms that offer mobile app testing, but in my assessment they're probably not at the level that is required to be able to do mobile app testing. And then you want to really ask yourself the question, am I really going to do mobile app testing? Because sometimes people say, yeah, we have a mobile app, so we need a B testing software that allows us to run a B tests on mobile app. And I've seen clients and companies that we work with do that and they never run a single AB test on their mobile app. It's a lot more complicated. But that's the very first consideration that will help you really understand kind of the lay of the land. The second is the type of AB testing that you do. Are you going to do AB testing that relies on front end changes or are you going to rely on server side AB testing? That's also very critical to understand. Most of the companies that we work with rely on the front end changes. Very few companies do server side. The philosophy is fundamentally different between the two. And if somebody wants to know more about the difference between if somebody wants to know the difference between front end testing versus server side testing, please leave a comment and I'll cover that in some other video. But that's another question that you want to ask. Am I going to do server side? And if I do server side, which platforms are really good for that? So that's the second area. Now let's say we remove the first two elements here. Okay, so there's about 30 AB testing software available. Which one should I be looking at? Most AB testing software offers the same core functionality. They allow you to create An A B test or split URL test. They allow you to use a visual editor to manipulate the page or use a code editor so you can insert your code. And that's what most serious testing programs use. They allow you to set as many calls as you would like. They allow you to schedule the test to run a specific time, do some targeting, specific targeting to specific pages. So that's the core functionality. The devil's always in the details. So one of the things that I look at, number one, is what kind of targeting does the software allow me to do? And this is critical, but you want to make sure that you are able to run different type of AB testing on different type of pages. Really, there should be no limit from the software when it comes to targeting. Number two, you should definitely look at the performance impact of that A B testing software on your website. We live in a time where Google tells us that the load time of a page matters. It matters in two ways. It matters in your SEO ranking and it matters when it comes to the user experience on your website. The load time of an AB testing software can slow down your pages. And remember, whenever you're using an A B testing software, you are going to take that JavaScript and you're going to put it across your website. So if that JavaScript is slow, it's going to slow down your website. Now the load time for an A B testing software can range anywhere from 100 milliseconds when a page is loading up to a second, sometimes a second and a half. This has nothing to do with your page. Correct. So you drop the JavaScript, the AB testing JavaScript on your site. And because the JavaScript is slow now your pages have an additional second, second and a half of load time. That's a Google penalty that says a user penalty automatically is going to cause a drop in your conversion rates. So you want to spend some time evaluating the load time of different AB testing platforms out there. Just to put things into perspective. Google Optimize by the way, the load time for the Google Optimize test inst script was about 500 milliseconds. 550 milliseconds to be exact. There's so many different ways you can measure the load time, but this is kind of a standard way. Some of the other A B testing software out there will take anywhere between 800 milliseconds up to 1200 milliseconds. There are some amazing software out there. I'm very proud to say that FIGPI takes about 150 milliseconds to load super, super fast. And we're looking yesterday at one website that is using FigPi, it was taking less than 30 milliseconds to load for first time visitors, it was that fast. Another criteria that you want to look at is what type of algorithm does the A B testing software use. Now lots of people don't pay attention to that. But you're splitting your visitors between the original, the site design and the challengers. You're splitting the visitors and you're tracking conversions. Very simple. Ultimately you're going to make a decision that this particular variation is the winner of the test. There's different statistical models that AB testing software uses to determine the winner. So whether it's frequentist or liaison statistics, that's very important that you understand because with frequentist statistics, which I would say about 50% of the AB testing software out there uses, there's a higher chance of having false positives. So Bayesian statistics is a better approach. Another thing, so we've ran the test so we can look at conversion rate, but usually we like to slice and slice and dice the data even further. I don't only want to look at conversion rate, I want to look at maybe bounce rate, I want to look at exit rate, I want to look at revenue per visitor, I want to look at the average order value, I want to look at the pages per visit time on sites. Those are secondary metrics. They're not the primary metrics to make a determination of whether a particular variation or a challenger won in an A B test, but they are good to know as if you are serious about conversion optimization, you want to look at some of those metrics. Finally, if you are running any type of limited time campaigns, so Valentine's Day is coming, Father's Day is coming, Mother's Day is coming. So you run a B tests, you run a B test specifically for that period. I'm going to run an A B tests for four days, five days. Well, traditional AB testing doesn't work by the way. It's not a good idea to do traditional AB testing. What you want to do is what we call a multi arm bandits and it's initially split the traffic equally between the different variations. But as one variation is winning, you give it more traffic. And you're constantly doing this, monitoring and adjusting the traffic allocation. If you don't run campaigns in a specific time frame, if you don't care about these specific, like oh, I'm going to run a campaign for five days or seven days, don't have to worry about this. Most e commerce websites we work with worry a lot about these campaigns because they make a ton of money in them and they want to maximize their profit. So traditional AB testing does not help you in this case. Matter of fact, it's probably not a good idea to run traditional AB tests. So multi arm bandits is a much better approach. Those are a few things off of the top of my head that I think of when I'm selecting an AB testing software.
How about the pricing? Because I'm sure like maybe some other companies, they might.
Oh, I forgot the pricing. Which is. Yeah, how can I forget pricing? Oh my God. Pricing is important. When you peel the onion, two things happen. You cry and then you discover an onion is an onion. Most AB testing software comes with the same features. Some might be more on the enterprise scale, some are not. But really, in all honesty, at this point, I think most sophisticated AB testing software can handle enterprise and can handle the smaller clients. However, the price tag for an A B testing software can vary tremendously. So a couple of things on that, just to give you the range of how much something like that cost, VWO started this. They came out with an announcement on Monday saying that up to 50,000 monthly visitors, you get to do any type of AB testing for free. So now they've added some limitations I've noticed as we played with the software. So you can only have five goals. You can have. The visual editor is a little bit limited, so they've added some limitations to tweak that fake pie. We came up with an offer where we launched it yesterday, where we said up to 75,000 visitors a month, you get to do AB testing absolutely free. No limits on the number of goals, no limits on anything. Do whatever you want. Here's the A B testing software. Now let's go to the other side of things. You can pay up to $10,000 a month for some of the enterprise software out there. Nothing less than that. Sometimes you're paying $1,000 or $2,000 a month, so it depends. Usually a B testing software is priced usually by the number of visitors coming to your website. And it can get a little bit pricey. In addition to the price, there's something else that people don't pay attention to is the length of commitment. There are some companies there that will lock you in with an annual contract. You're going to start using our software, you're stuck. Even if it's $2,000 a month, you're going to pay 12 months, that's $24,000 and sometimes $10,000 a month. Guess what? That's $120,000. In addition, they front load additional costs. So I would look at the price, make sure that it makes sense for your specific budget. I'm going to add just a couple of other things which is migrating whatever AP test that you have in personalization Google Optimize, whatever platform, you can do the migration yourself. Your engineers can do the migration, your marketers can do the migration. However, optimizely offers for example paid migration service where they're going to help you migrate. FIGPI offers completely free migration service where the NGOs are going to jump in and migrate all of your active A B tests and personalization to figpi. I think vwo, I have a call scheduled with them next week. VWO will offer something similar. I'm guessing. Don't quote me on that officially yet.
Nice. I was listening like carefully to what you were saying and you dropped a few names like from some of the players. So just for some of our listeners maybe who might be considering moving from Google Optimized to another AB testing platform, which are the top five players that you think they should look at? I know that you've dropped just like a few names, but which are five that you can say at least? Maybe if you can look at this and this and this and this, I.
Can comfortably tell you that our clients at this point are on one of three platforms. Those are the three main platforms. And then there's a ton other. Again there's about 30 other platforms out there but the platforms that our clients are on. Figpi is one, VWO is two, Optimizely is third. We have a ton of clients and they work with ton of different platforms. The only two platforms that we specifically say from the get go that we do not work with are Omniture. Adobe Test on Target Omniture Test on Target was the old name before it got acquired. We don't work with it for many reasons. That's a whole other episode and side specs, although I know the guy's a side spec but we've never worked with side spect. It's a lot more complex setup and it takes us longer to launch experiments so we've avoided that. We have some companies working with Camlon with abtasty again plethora of AB testing software out there. Some companies work with Convert, but really again the top three platforms I've worked with ThinkPy, Optimizely and VWO.
So before we move on to the next question, outside of this Google Optimized topic, I just want to ask you one more question, then we move. Do you think this end of service, the shutting down of Google Optimize in September will in some way or shape impact the CRO industry?
No, I think at this point the CRO industry is very strong. There's many, many companies who are committed to the CRO. They understand the value of experimentation. As a matter of fact, companies who are super excited about experimentation, companies who are super committed to experimentation, most likely they are not using any of the AB testing software out there. The Amazons and the Facebooks and the Netflixes of the world, they probably have their own homegrown software. I remember when I worked at Citrix. GoToMeeting, GoToWebinar. I'm not going to date myself, but many, many years ago, over 15 years ago, they've had their own A B testing software before AB testing was even a thing in the markets, before even the term conversion optimization was invented or used. I think just like how what Google Website optimizer did to the experimentation CRO industry at some point, I think that's what Google Optimize did. It's helped really excite people about the industry. It helped get a lot of marketers thinking about a B testing. A tool at the end is just merely a tool. The industry does not rely on a tool. And I think people nowadays just like how everybody expects every website to use heat maps or session recording. Back in 2006, 2007, that was a new concept where people really did not think about. Nowadays it's just a standard. And I think nowadays a B testing, more and more companies just take it for like, okay, we just need to pick the right tool. The more difficult part of experimentation, conversion optimization is the people and the analysis that takes place. That's the hard part.
Yeah, that makes a lot of sense. Okay, so we have a couple of minutes left. So I'm just going to ask you like this one question. Then you just tell me what you think. It's a question that we got from one of our listeners. I'll read it out. The question is how can E commerce businesses measure return on investment of their CRO efforts and track their progress over time?
That is how can you measure the return on investment for any CRO or experimentation program that you are running? That is perhaps the hardest question that every, any company that runs an experimentation program or hires an agency to run a program is always asking. And people struggle with this. There are four different ways that you can measure the return on investment. There are four ways that you can Measure the return on investment for any type of CRO work that you do. Let's mention those four and then we'll get into the details of them. Number one is by testing your progress against the original sides that you've had. Number one is by testing the the progress that you have where you're at at this point against the old side. That's number one. Number two is by using data modeling. Sounds really hard, and it is really hard. Number three is by measuring customer satisfaction and usability metrics around your website, which is not real roi, but it is helpful. And number four, by measuring and evaluating conversion rates. Rpv, aov. And we'll talk about those as well. But. And let's jump to each one of those points. Number one is measuring where your site's at right now after doing conversion optimization for a while. So usually after you've done conversion optimization for six to nine months, you want to say, okay, so we've ran a whole bunch of experiments. The site has changed and evolved in those six to nine months. We know that the site design has changed, the user flow has changed, the messaging has changed. But how about our conversion rate? And the most solid way, and it is the most difficult way to actually know the exact impact, is by keeping what we call a stub, the old side design. It's kept somewhere. So at the very beginning of the project, you take the stub and you keep it somewhere. And then nine months later you come and you say, I'm going to run a side wide test between the current side design and my old stub that I've kept. Now, mind you, this is not easy, by no means. And you really, really have to think if you want to do this or not. Because running a test like that might take anywhere between two to four weeks to just prepare for and then maybe two to four weeks of running. So you are losing two months. But at the end of that test you can actually come back and say, you know what? This new site design that I have had actually increased my conversion rate. So my conversion rate was 3% six months ago and now my conversion rate is 3.3. Based on that a B test that I've ran, that's number one. We try and avoid it, but every once in a while we work with companies, they want to do this. I'm like, sure, we can have at it if you want. And it's always fascinating. Number two is through data modeling and it sounds difficult. Yes, it is difficult. Remember running an A B test and looking at the results of An A B test doesn't really tell you the actual revenue and conversion increase that you have, because not all A B tests are created equal, and not all A B test results are created equal. The fact that you ran a B test on a homepage and shows you a 10% increase, so you have a new homepage with a 10% increase, that does not mean that you have a 10% increase in revenue. I would love for it to mean that, but the reality of it, it doesn't mean that. So you need to apply some statistical formulas to that test result that you saw. And if you're interested in knowing more about these statistical formulas, leave a comment, I'll be glad to cover that. But at a very high level, what those formulas take into consideration is the type of page, type of test, how long it ran, how many visitors are coming to the page, the confidence level that you've ran the test to, or the statistical significance. All of these come into play when it comes to your data modeling to determine the final value. And it's not an exact final value, it's an estimated final value. Number three is measuring customer satisfaction with your website. There's multiple ways to do this very rudimentary NPS score. Are we actually improving customer satisfaction with our site? And there's other metrics that can help you measure the customer satisfaction with your sites. Now, again, we're measuring customer satisfaction. It doesn't mean that we're measuring roi. Correct. There's a big difference between the two. People are happy. But maybe I don't have more money in the bank. Nonetheless, happier users, better usability usually translates into more money. So you want to measure that. That doesn't specifically help you measure the roi, but it's something that I would pay close attention to. My favorite thing to do if we're going to go that route is to actually take a measurement, usability measurements. How long did it take somebody to place an order? How many steps did it take for somebody to place an order on our website? How much time did it take somebody to place an order on our website and see if there's improvements in those metrics? Finally, you can look at your conversion rate. You can look at RPV revenue per visitor, and then you can look at AOV and measure those. But if you're merely saying, well, you know, our conversion rate right now is 4% and it was 3.5% back then, that means that we've increased our conversion rate, that doesn't work. You need to add a bit more sophistication. To your measurements because you need to look at your traffic sources. Maybe you've increased the traffic, you've doubled your traffic when you started the work until now. And as you increase traffic, automatically, conversion rate would have dropped. So it looks to you like you've increased conversions by 10%, but in reality, you might have increased conversions by 20, 30% or something like that. So there's more analysis that you need to pay close attention to. Did we change our traffic mix just to make it even more complicated? You need to think about the economical situation. Everybody nowadays is worried about the recession. Guess what? Your conversion rate, without you doing anything, no experimentation, no testing, automatically is going to drop a little bit. People are worried has nothing to do with conversion rates, Just the economical situations pushing people and telling them not to place an order. So you are seeing less conversions. Those are the four main methods that we use to evaluate how much money a conversion rate optimization program helped us make. And then we look at the cost of the program and that way we can calculate the roi.
Yeah, that sounds good. When you were talking about like point number two, which is the thermodynamics, you mentioned something to do with the impact of the page was at some point at first, like, I remember the first days when I joined invesp, just like about a few years ago, that used to confuse me because at some point I used to think that maybe if an A B test results in a 10% increase in conversion, that will literally like translate to 10% increase in the revenue that you get or the money in the bank. You then wrote an article about the impact of page on the website conversion rate some few years ago again. And that kind of like cleared everything for me and I got to understand how everything works. So that makes sense.
The closer a page is to the bottom of the funnel, the more impact it has on your overall site revenue. So, for example, a page in the checkout, if you increase conversion rate there by 10%, if you run an A B test on the last step of the checkout and you see a 10% increase in conversions, guess what? That's a 10% increase in your revenue. It's a lot different than running an A B test on a homepage and seeing a 10% lift, because not everybody who converts will go through the homepage. So you're only improving conversion rate for those visitors who went through the homepage, which could be a small segment of the site.
Yeah. So in other words, some pages are more important than others when it comes to conversion rates, right? Definitely.
Yeah. I mean, like when we start on a project, we like to start with the bottom of the funnel pages because they help you and you can see the impact and it's beautiful. I mean it's a balancing act because bottom of the funnel pages are also more complicated, more security around them. Everybody, not everybody's looking at them, but they can have a direct impact, positive or negative on a website.
Okay, so one more last question before I let you go. I know you just said that you always like start optimizing the bottom of the funnel. What if like you did some research, right. And then you noticed that some of the best practices were not like implemented on the top of the final. Won't you also like maybe just try and focus on those first? Just like to clear everything out as since they might be low hanging fruits and then you work yourself to run the bottom.
Yeah, let me finish this. A really great question. Maybe we should dig into it by the way, next week. But here's kind of a teaser. So it's a balancing act between the complexity of the test, the impact of the page on the bottom line and the number of quick wins that you have on the website. So it's a balancing act between those three legs. It's almost like a stool that stands on three legs. Is there such a thing? Yeah, I think you can make that. Usually the very first experiment I will launch is an easy experiment, simple experiment, but at the same time I have a list of quick wins that I'm trying to hit. The first six to eight weeks working on a project because I want to show somebody who started with experimentation, I want to show them the value. And the value at the very beginning when you are trying to establish trust is in two things. The type of research that you are doing. So they should be very impressed by that and the results that you're achieving. First eight weeks, just like eight weeks. First eight weeks on any job is about showing the quality of the work and the results. Same thing with an experimentation program. But let's cover that in more details on our next week episode.
Yeah, sure. So I think that's it for this week and see you next week.
Awesome, Simba, as usual, it's always fun and we shall see you everyone. If anybody has any questions, I'm on LinkedIn. Simba is on LinkedIn. Yeah, drop us a message and we'll be glad to cover your questions until next time.
Sure. Bye.
More from Khalid
Unlocking Website Conversion Secrets Through Data-Driven Testing | Khalid Saleh from Invesp
Unlock the Power of Post-click Experience: Roadmap to Convert Leads into Customers