Why retail marketers need to embrace failed tests | René Tingskov (Søstrene Grene) | Coffee Breaks

Do you have any tips or advice on how to share out the results of these experiments, especially when the experiment maybe isn't positive, right? Where it didn't beat or it did worse in some way. Like, how do you communicate that in a way that makes it so that people want to keep experimenting and don't want to just like stop doing that?

Every year when we do the marketing budget, I actually set aside percentage of the budget that goes into testing only. This is not something we should count on will make business right away for us.

Renee Tingskov leads marketing at Sostrene Grene, where his first-party data strategy more than doubled customer lifetime value and dramatically improved paid media efficiency. A Meta Growth Accelerator pilot participant and frequent conference speaker, he's been featured in multiple industry case studies for pioneering predicted customer lifetime value as a media buying KPI and solving omnichannel attribution challenges across a 90/10 offline to online business. On today's episode, we touched on how Søstrene Grene pivoted to e-commerce, what their test and learn framework looks like in practice, how to communicate your test results to leadership, their hypothesis generation and prioritization strategy, how they measure their advertising when brick and mortar is such a big part of their mix, LTV and high-value customers, and so much more. Rene, thank you so much for joining me. It is great to have you here this morning for me and this afternoon for you.

Such a pleasure to be here. Yeah, it's the afternoon now, so wife and kids are waiting at home, but I've been really looking forward to talking to you today.

Well, thank you so much for making the time. We are so excited about it. First of all, for those of you who— or for those of our audience who don't know what Søstrene Grene is or how to pronounce it, like me, can you maybe one, give us the good pronunciation? And two, explain to us like what this business is, what it does. I think some people in the United States might be quite surprised to learn this is actually like a really big, really important business, but it might not be familiar to everyone.

Of course. Yeah, the name is a bit tricky and that's because actually we have some Danish letters within it, which is very unique to the Danish language. It's a letter called Ø and the I guess the right pronunciation is Søstrømme Grene. Søstrømme Grene, yeah.

Søstrømme Grene. I'm going to practice.

Exactly, yeah. Fantastic pronunciation. Basically, we are an old Danish retailer. We were founded back in 1973 by two people who lived in Denmark's next biggest city called Aarhus, which is where I'm also stationed at work today. We are really close to where they founded the first store, actually. And it's an old, as I said, Danish retailer. We are now a really big one in Europe, at least. We have more than 360 stores. And we are actually present in, I would say, I think it's 18 countries now. And we are planning to open more countries in the coming year. But if we move back all the way to 1973, we were founded by a woman called Inge and a man called Knut. Knut was a ballet master. I think it's— I don't know actually what's it called in English, but he was really much invested into the ballet business. And you can actually see that when you go into a Søstrene Grene store. When they opened the store in August in '73, it was with the ambition of creating a retail, new retail experience that was built more on experiences than just going in to buy something. So, you should actually, the way we think about it is that you have a break from everyday hectic life when you go into one of our stores, where all your senses are activated. So, we play classical music in the stores. We have a distinct light setting showcasing the products like a ballet dancer on the scene. You can go and touch and feel all the products. They are displayed in open wooden caskets where you can see them and feel them. And then, we have a design route throughout the store. So, it's like a maze, somebody calls it. Where you, where you actually— it's designed to go— you're designed to go in a certain way, so you're exposed to products and smells and everything. So yeah, very distinct and different experience, at least back in '73. Um, and what is also very interesting to Søstrene Grene is that we did actually not have a digital presence up until 2020, where Corona was the big trigger for us. So before that, we actually didn't have a digital presence. We are a huge franchise business. We were at that time. But Corona put a stop to all retail businesses, at least in Europe. So a lot of stores were closed down. And we were forced to look in other directions. And of course, the digital part and building an e-commerce business was very interesting to us. And I myself actually joined Søstrene Grene in '21, where we had a foundation to work from. And I've been on board— I was taken on board in order to help us grow the digital business and the marketing part.

That's super helpful background, really interesting environment, right? So you've got this fairly, you know, old successful business with a really unique approach to, you know, the in-store experience moving online really recently, right? Only in the last couple of years.

Yeah.

And it seems like y'all have had a ton of amazing success. So in preparation for this podcast, I went and read some of the interviews you've done and listened to some of the podcasts and what really struck me is, first of all, how much success y'all have had. I think, again, making this pivot into e-commerce, totally different concept, so quickly. Um, and a lot of what it seems like has contributed to that success is what, you know, you have referred to as your test and learn approach to, to marketing, and specifically this, this digital marketing component. Um, I'd love to hear in your words, like, what does test and learn mean to you? How do y'all live that at Système Egrene. Just tell me about what test and learn is for y'all and how you're building that practice in your team and across the company.

Yeah, of course. Yeah, so test and learn, of course, came with also the digital part of Système Egrene because it's much easier for us to do a lot of testing digitally. And we are big partners with some of the big tech companies like Meta, for instance. Where they've, for us to see, they've actually built a platform that works with scale within Test Learn. So we actually, we built a framework for that. So basically meaning that we always need to have a new hypothesis that we want to test in our backlog. Of course, we've built a big setup with business as usual setups, but we have a lot of new hypotheses of us is we want to test all the time. And basically, we have the saying that if we don't have a test running, we are missing out on potential. So basically, what we do is that, for instance, on Meta or other platforms, we, on a monthly basis, we run a new big test where we run it up against our business as usual. And then we see if we can beat our current best practice. And if we can beat our current best practice, that becomes our new best practice. And then we move on to a new test. So every time or every month, we have a goal of learning something new for each platform we are engaged with. Of course, as I mentioned before, it's very easy for us today to test on digital platforms because, for instance, on Meta, you have the opportunity to have a test and a control group in a closed environment, also because you have logged-in users on that platform. So you are able to test a new material on a test group and then keep a group out so they don't see your test, and then measure against A and B to see what is the incremental or what is the uplift by doing A versus B. And that has worked really, really well for us, and it's an approach we have actually taken to all the platforms we are invested in.

Sounds great. Sounds very straightforward. When I talk to people working at other brands, other companies, sometimes they have this desire to implement a system like what you're describing, but they run into roadblocks. And those roadblocks might be that people at the company, people are very risk-averse, right? And so they would feel embarrassed if their experiment didn't yield positive results, or the company is risk-averse in another way where they say, like, okay, we have something that's working. We can't use some of our marketing spin for this experiment because it might be inefficient. And so, I'm curious, like, how do you deal with that internally? Have those ever been questions? How did you get the whole organization to adapt this idea of embracing the risk that is inherent in experimentation?

And that's a really, really good question. I guess it's also something to do with our organizational structure or the way we are structured as a business. We have a, we have a guy placed at our C-level table who knows a lot about digital marketing and digital e-commerce. Yeah, this is a business I've worked with him actually previously before, uh, Susanne Greiner, at an age on the agency side. So he knows a lot about what's— how, how you approach a digital business, and we really, we really tapped into that. So he is— we are able to, we're able to bring along or tell him what, what works from our standpoint, and he can actually take that to the, to the decision table and make sure that all the relevant people hear what, what he has to say. And also, our CEO is, is really invested in having people around the table that is better at him than him to make the right decisions within the specific fields. So, if my boss, Rasmus, goes to him and says, "This is the right thing to do," he really truly believes that this is the right thing to do. And luckily, Rasmus and I often agree or do agree a lot that testing, if we don't test, we never become better. And then, we will lose at some point because other than that, other companies or our competitors that do testing, they will outrun us at some point because they'll figure out things faster than we will and be able to move faster at some point. So, for us, it's very important that we never believe that we have the best setup or the best way to do things. It's only best practice until it's not. And if you don't test, you'll never find out when there is a better setup for you as a company. So, I guess that's really, really important for us.

Do you have any tips or advice on how to share out the results of these experiments, especially when the experiment maybe isn't positive, right, where it didn't beat or it did worse in some way? Like, how do you communicate that in a way that makes it so that people want to keep experimenting and don't want to just, like, stop doing that?

Yeah, of course. Every year when we do the marketing budget, I actually set aside a percentage of the budget that goes into testing only. And I talk to Rasmus, my boss, around this and telling him, this is not something we should count on will make business right away for us. This is something we need to have as a budget in order to stay ahead of the curve, in order to make sure we have the right setup at all times. So, there's no promise from me to him that this is actually something that will give us a lot of business. We're also really good at testing things with our partners. So, often we can also— we're also lucky to get ad credits from our partners, which is, of course, very nice because then we don't have as much on the stake as as we would have otherwise. But I guess also part of the discussion, at least with my boss, is that I need to be able to explain him why we want to test this. What do we see as a potential outcome? So, what will we gain from this in the long run? And then, normally, we do a lot of iterations. I just explain it out blank and say that we'll perhaps need 5 iterations of this test because before We have something that we can work with. And then, of course, he buys into that or not. That could also happen. But then we actually start running the tests. And of course, if we see after 1 or 2 iterations that this will never be something good, then we can easily say this is not something for us. This is not something we want to work with and then move on to a new test. But it's totally okay for us if the test doesn't go well in the first 2, 3, 4 runs. But we need to see things improving along the way.

It makes a lot of sense, and it sounds like a really healthy, high-trust environment that is part of what makes that work. At the beginning, you mentioned this idea of having lots of hypotheses, right? You have this sort of scientific method approach, which I love and really resonates. One of the— I've sort of seen two different problems when trying to roll this out. In some organizations, I feel like it can be hard to get people to frame their research questions in terms of falsifiable hypotheses that are easy to test. A lot of people they feel like they operate on gut feel, and it can be difficult to get them to try to express an experiment. So that's on one side. And on the other side, once organizations start doing this, you sort of get too many hypotheses, right? And you get this long list of things that people want to experiment with or test, and you have to sort of prioritize them in some way. Yeah. So I'm curious to get your thoughts on what is your process for soliciting these hypotheses? Do they just come from you? Do they come from the wider organization? Where does that come from? And then two, what's your strategy for prioritizing them, right? If you're running one big test a month, how do you figure out what's the most important? How do you all think about structuring that experimentation roadmap over time?

Really good question. The easy one for me to answer is that sometimes our partners come with a really good opportunity, and maybe it's not something that is directly prioritized by us, but if it's a really good opportunity, for us to tap into, maybe if it also contains AdGrids or something like that, we are really often, we are pleased with that and we will actually run the test just also to see what can that do for us. So sometimes there's not, we are running tests that are not directly prioritized by the business but can give us a lot of learning in other parts, but for other parts of the business. But we have, there are two things we are really focusing on at Sustained Gaining. Of course, being a retailer, being a business, we are really focused around sales. So we are, and we are still, we are still learning a lot about being a digital e-commerce player. So if we have a low funnel opportunity, so to drive sales, if we have performance marketing, new performance marketing setups, we can, we can run, we are really eager to do that. But more important to us is actually the whole first-party data agenda. So being able to find loyal customers and being able to find a base of, yeah, a base of loyal customers who we can, who we can build a relationship with, who we can actually, yeah, improve their customer lifetime value together with us at Systemic Training. Um, so, so leads, uh, lead activation and lead collection is, is very, very important for us. If we can find new ways to, to test setups that can help us drive high-quality leads, that's something we are always interested in, and that has some of the highest priority for us as a business.

Interesting. So it sounds like there's a mix of partners come with new features, new tools, new ideas, and then also even like long-term business strategy where, you know, we have this long-term thesis about improving customer lifetime value. I want to talk about that in a little while. Um, and so where it's worthwhile for us to experiment with things that are going to help us drive that or give us more signal on what drives forward.

Exactly. Of course, if we can drive fast sales and we can drive, uh, fast revenue, that's something we are always interested in. Again, being a business But if we can invest in long-term relationships with customers who we know will come back again and again, and we have, we can build a direct communication line with, that's of highest priority to us. We invested in a CDP actually when I started, a customer data platform, actually because we saw what was happening, at least it's a big thing in in Europe with the customers being more aware of what data they give and actually also putting a lot of roadblocks, at least for the marketing side, around what we can track. So, how much can we actually track for a customer that visits our website and how long are we able to keep that relationship if it's just a cookie, for instance? So actually, when, when I came aboard, we, we decided that we wanted to invest more heavily in the long-term relationships with our customers. So building, building a much bigger lead setup, bringing a much more CRM setup for us back then. And that is some of the groundwork of what we're doing today as well, and what we are succeeding really well with at Systonic.

Sounds like a great success story, and this tees up one of the next questions that I wanted to ask, which is you have this interesting business where you have lots of physical brick-and-mortar stores where people come to shop and buy, and now you also have this online business. We've been talking a lot about primarily online experimentation, um, and I'm curious, how do you think about closing the loop between online advertising across these different platforms and then sales that might happen, yes, on e-commerce, which are not easy to link together, but let's say relatively straightforward to start to connect, versus sales that might happen offline in one of your brick and mortar stores. Someone comes into the store in Denmark, pays in cash, takes their thing after seeing an ad for the product on Meta 2 weeks before. How do y'all think about bringing that together? How are you thinking about measuring the full impact of your advertising in the sort of omnichannel world that you exist in?

Yeah, exactly, because it is an omnichannel world and our, our businesses, physical businesses, uh, are the biggest part of our business still, uh, even though we are, we are fast pacing, uh, for the digital part as well, but I guess in round numbers they are still counting for like 90% of our total revenue. So the, the physical businesses are really, really important to us. But as you, as you mentioned here, we are at least up until now, we are not able to actually track that much in stores. Of course we can, we can look at footfall and we can look at how many transactions do we get in a day for the specific businesses, but there are no way for us to track have we ever talked to you on our social platforms through CRM or something like that, and can we see you moving down to the stores? But that is actually a gap we are trying to close right now. We are— we have something, I guess most businesses do, we have something we call must-win battles at Sustana Gaming, which are yearly goals we set for ourselves. What do we want to accomplish this year? And actually, one of the goals this year is to close that gap with something we call offline data. So we have an offline data project. And the way we've chosen to approach this is actually by starting to give customers digital receipts in the stores. As just an ingrained, we don't have a loyalty program yet. It's also something we are looking into because that's of course something that could help us close that gap. But we also have, we have a saying at Sostane Graene that we want to be something for everybody. So not just our loyal customers. So not just loyal customers should be able to get a digital receipt at Sostane Graene if that's, that's what we choose to, to do to solve this problem. So we, we're working together with, with a partner called, called Receipt Hero now to, to set up digital receipts in, in our stores. We are actually right now, we are running the final unit tests to, uh, to see how, how this can actually look in our stores. So we're testing in, in one store right now, and the concept will be that the customer who has done a purchase in the store can, can get a digital receipt in two ways: either through an NFC device that we have placed at the counter, so they can scan it with their phone, and then they will get a browser and they can actually send that receipt to their email. Another way is that we want to, to link that credit card on that profile on our website and app, and then just by actually tapping the card to pay, they will automatically get a receipt on their profile. And, and by doing these two things, we'll get an email, and at least if the customer consents to it, we'll actually be able to do a lot more tracking, and then we'll have the, the full circle when it comes to, to data, because then we'll be able to see, have we talked to this customer on email before, and can we see them moving into the stores? What's also very important here is not just being able to track that, but also to see what tactics, what activities do we run that are most efficient at driving people to the store, because If, if the physical stores are not performing well, that's obvious. It's just, and again, it's not performing if they're accounting for like 90% of total business revenue. So this is how we are trying to approach it. And all the data will then be of course transferred into our CDP, and from there we'll both be able to do analysis, but also sending raw data to our platforms for them to optimize against. So that's gonna be super, super exciting.

Sounds like it will be an exciting 2026 project for running those experiments. I'm curious, so have you experimented at all with doing any sort of like geographical lift testing to understand if running ads in certain geographies lift sales in those stores without necessarily needing to do the connection? I'm curious if y'all have experimented with that, if that's something that's on part of your program.

It is, but it's not for me to see. That's— there's no direct correlation between running ads in a certain area and seeing an uplift in sales in certain stores. The uplift in, in the stores can, can, can be due to a lot of different things, also things we are not able to understand. So that's why I, I like to see the direct connection between data and what we're doing on data platforms. We, as Søstrene Grene, we have, we are, as I mentioned before, we are a big Danish company and we have, everybody in Denmark knows us. I think 98% of all Danish people knows us because we are more than 50 years old retailer. But if we go to a country like Germany, for instance, which is actually becoming our biggest market um, it's only like 12% of all Germans in our target group that knows us. So that's, that's of course a big, big problem for us, because if we're not top of mind and you have a product need and you don't think of Sustaining Greening, you will not visit the Sustaining Greening store. And of course we want to change that. So right now, this year, we've also, we saw, we've also been working on our branding project. So what, what should we tell customers that don't know Sustaining Greening yet in countries outside of Denmark? So we've tested a lot on that, uh, actually before the summer holiday this year in Germany, where we did a test like you mentioned here, where we, uh, we chose 4, uh, 4 geographical locations in, in Germany, and then we, uh, we activated the branding material for 2, so 2 of those cities, and then we left out 2 other. Of course, we, we, we chose 4 that were comparable in footfall and sales, and also a number of people staying around the stores. Um, and we did see an uplift, uh, but the thing around branding is that it can take time because you are building top of mind. You're not tapping into an in-market behavior. So if, if you want to, if you want to measure that, you want, you need to be able to, being able to be able to measure over a longer period of time. Of course, you can expect to see some short-term sales, but I would say that you also need to calculate with that some of the sales might come after a longer period. And again, that's why this can, for me to see, it can be very unprecise to do testing like this.

So it sounds like you ran the test, got actually positive results, but there are these lingering questions about how precise are the results. Yes, that makes perfect sense. I think that that's a very common experience when people are running these sorts of geographic studies.

And that's, that's again, that's why I think it's so important to get the ID of the customer in the store. If you don't have that, you don't have a direct link. And then I think, for at least for my sake, and if I have to, uh, argument, uh, that we need to do more of this, I need to have that data to showcase that what we're doing is actually successful and we are driving people to the stores. And that we are also not just driving people to the stores, but we are also driving sales.

So this is a really interesting topic because there's another flavor of this. So you mentioned a little bit earlier how important lifetime value is to the business and how much y'all think about really wanting to, to drive high lifetime value customers. But lifetime value obviously gets realized over a lifetime, right? Or over longer time periods. And so when you're running these incrementality tests or uplift tests, how are you thinking about combining this idea of driving sales from incrementality versus this thing that you care about of longer-term lifetime value, right? When that can't be measured in a month, right? Or the time that you're running this test. How do y'all think about connecting those ideas?

The question here is really, really interesting because it taps right into a test we actually just finished or I'm still in the still working on. But we teamed up with a partner called Journey who works with modeling of data, and they're really good at causal machine learning. And it was actually a company that was pointed out by Meta because they know that we care a lot about the connection with our customers and we have a lot of data. For our customers. So what they could actually help us do was actually do some calculations on all the customer data we have, and then through causal machine learning, look for patterns that we could not see, and then actually find signals we could, we could give to Mesa and then use for our low-performance marketing to drive— go after not just everybody but go after high predicted customer lifetime value customers. This was really, really interesting because we could actually see it gave us more volume in sales, both short-term but also longer-term. But the thing around this was that we were only able to track these customers with, I don't know, UTM tracking and cookies. And what we saw was that we are not able to get the full picture. And also, um, we did not have, uh, we did not have access to communicate with this customer if they were not on Meta. So what we thought about, how can we, how can we take these short-term sale customers we can, we can approach by this, uh, this setup, uh, and turn it into something more long-lasting? So we instead, we told, we talked to Journey and Meta around a new setup where we actually wanted to go after still our predicted best customers, but instead of having it in a short-term sales campaign, use that signal to go after the leads. So have a lead set up in MESA, give them the data around our best customers, and go look for our new best customers. And what is really, really interesting, and we are running this test actually right now, is that we can basically see this is working. So we can see, okay, compared to another lead setup, we get fewer leads, they are more expensive. But we can see when we have the lead and they give us consent, so we can track them. We can see over, for instance, 3 months, they become a lot more worth to us, like times 2 or something like that. So even though they are more expensive to catch, the long run or the long-term profitability is much, much better. So what we need, what we need to figure out now, how, how can we, how can we put even more scale into this so we can, we can find as many new best customers as possible? And I think if we nail this one and we can also put in offline data into this setup, we will have, we'll have an extremely powerful setup for, for the future. And this is actually pretty interesting because this is actually something we figured out while doing another test. So this taps right into the test and learn mindset. So while doing one test that was really, really nice and worked really well, we did a lot of iterations for that test. But during that test, we figured out, okay, if we actually do something a little bit different, so not just invest in short-term sales, but invest in the customer relationship and we keep the backbone as is for the setup, we can do something that's much more powerful and much better for us. So, again, yeah, you learn a lot by having a test-and-learn mindset.

Absolutely. So, I just want to make sure that I understand the setup that you're describing.

Yeah.

So, it sounds like there's— we have some way of Well, so we have high-value customers from history and we know that and we can do something like creating a lookalike audience for them. I know this sounds like Meta from 7 years ago, but it sounds like that's part of it. We want to target these people that look like high-value customers. And then to confirm how high value they're going to be, we have some other system that's looking at their behavior and predicting their overall lifetime value based on just a month or 3 months or something along those lines. And by pulling those pieces together, we're able to target customers that we think are going to be high value and then confirm that they're high value by looking at these early lifetime signals and seeing that they are like actually look like these overall high value customers during those first couple of months. Is that, is that the right way that this is to think about this?

Exactly. So, so Journey is looking at all the data we have for our current customers. So, and here they're able to see, okay, this customer is worth a lot, this is top 2, top, top 1, and then this is top 2, this is top 3. And then they look at the patterns for those customers. How do they behave on the platform? What are— what, what do we know about them? And then they connect those patterns, and then they said they use that to create a signal for Mesa to buy, buy in on. And we use in Mesa, if If anybody is wondering how we do this, we use something that's called conversion leads. So where you can actually, you can set up a direct link between Meta and your CRM setup. So you're actually able to not just create a lead ID and send that off to the CRM, your CRM, but you're actually also able to get a signal back to Meta. So telling Meta, when does this become customer? So you can use the data signals we get from journey in Meta, and then you can establish a direct line between Meta and your CRM. So you're not optimizing against leads, you're optimizing against customers because you're getting that signal back from your CRM. And that's the tricky part around it. And if you have the right setup to do this, I think this is really, really a nice feature from, from Meta that you're able not just to optimize against leads but optimize against customers.

Okay, cool. Really, it sounds like a really powerful setup, um, and it'll be even more powerful once you get the—

once you—

if you can connect to the in-store, like, brick-and-mortar sales. So I'm curious, I think this is probably the last question that I have, what is your sort of most important piece, piece of advice for Let's imagine someone starting on this journey, you know, maybe, you know, where Søsterne Migræne was like 4 years ago, right? They're embarking on this journey. They want to start embracing test and learn. What's your advice to them or what are your recommendations on the most important things they need to put into place?

I would definitely say that you need to start invest in your own data and invest in your own customers. You cannot be reliant on Meta or Google or other tech platforms. You have to have your own data. Also, at least from a European context, we are losing a lot of that data that those systems can actually provide us with. So the competitive advantage for us is definitely being able to have that data ourselves to provide, to give to our partners. Because the, the more data you have and the better quality of data you have to give to your partners, the better outcomes they can give you back. Um, and yeah, basically investing, investing in, in your own data. I think that's, uh, that's—

what are the, what are the key aspects of the data system that a business needs? Is it— you mentioned the customer data platform, so CDP. What else is, are like the are parts of your data system that support this?

We also, we are, we are a huge customer with Salesforce. We use their CRM system part. And I think, of course, investing your own data also means for me investing in a CRM setup. So being able to not just, not just relying on outreach media like Meta or Google, but also have, have part of your marketing setup where you have established a direct line to your customers so you can communicate with them directly. I would say CRM is probably our most effective platform for generating both short-term sales. So if we have something, a new product line that is coming out, we often see then when we burst that out through CRM, That is where we get the highest cashback, short-term-wise at least, but also on the longer run because you can actually use your CRM to build your relationship with your customers, to have a way to continuously communicating with those customers. So yeah, CRM and first-party data, I think that's really what, what's most important from our perspective.

Amazing tactical advice, René. Those are all the questions. We're going to wrap up. We're going to get you home to the wife and two sleepy children. Um, thank you so, so much for joining us today. Really informative, really interesting discussion. Um, really great to hear from a practitioner that has adopted and embraced test and learn and has, you know, made such huge changes in, in a very short time in this business. So congrats to you, and, and thanks again for for coming on the show and telling us all about it.

Thank you so much. It was a pleasure to be here. Thank you for all the good questions as well.

Thanks, René.