How to succeed as a data leader | Cecilia Dones (3 Standard Deviations) | Coffee Breaks

Cecilia Donez is the Founder and Chief Data Officer at Three Standard Deviations. Cecilia's work focuses on the intersection of technology, data and decision making, with over 15 years of experience as a marketing, analytics and data evangelist, transforming how organizations create value through data. In today's episode we had a fascinating conversation in which Cecilia shares insights on how analysts and data scientists can cross the chasm into leadership. Examples of how to build a testing culture within an organization, the right way to think about communicating uncertainty to executives, and how analysts and data scientists can really focus on having a big impact within their organization, not just generating interesting insights. There's a whole lot more in this conversation and CC is a great communicator who shares a number of different practical frameworks that you can use in your work. I really hope you enjoy the conversation. Czech welcome. We are so excited to have you. You have been working in analytics and data science for your whole career. You've done data science and analytics. As an executive at l' Oreal and Mohead Hennessey, I'd love to hear from you. Just jumping right in. In your experience, what's the most important skill for analysts or data science or data scientists to work on if they want to follow in your footsteps? Follow your path. As a leader and executive, I would.

Always recommend follow your own path. So create your own story. However, if someone is interested in kind of having the topsy turvy non linear path that I've had, what has been my North Star and what I tell my clients tell my students, tell my mentees, follow your curiosity. So keep asking questions and have the tenacity and the bravery encouraged to find out those answers. So I've everything I've ever done, every choice I've ever made professionally and personally for the most part has been a result of oh I didn't know that, oh I was completely wrong or oh you're changing my whole worldview. Help me understand that better. Oh, I get curious and it puts you in places where interesting things happen to you in the sense of you discover new information for yourself and two you learn so much about what you're good with, what you're cool with and what you're actually not so cool with. And so I find it's a beautiful way to 1 learn about yourself. But 2 find your purpose in terms of what you want to do. So curiosity is the first one. The second one I tend to talk about a lot is creativity. I think the data discipline, the analytics discipline, what we do is creative. It's just our medium of choice Is data. And so maybe sometimes the art is sometimes hard to understand for others, and that's okay. That's why we create it. Art is meant to be created for ourselves, not necessarily for everyone to understand it. And the third component, I would say is critical thinking. I think about this a lot. I talk about this a lot. And I think it's really important to evaluate what's being told to you. Don't always accept and also be mindful of who you're speaking to and how they're conveying their message. So combining curiosity with creativity and critical thinking, I think is a good formula for success, for having wonderful adventures, as well as misadventures.

CC's, three C's, I love it.

Yes.

I want to dig into a couple of these different components. One that jumps out to me. So critical, like curiosity and creativity, those sound like these a lot. I think everyone feels like they have at least some access to. But critical thinking feels like that can sometimes be a hard skill for people to wrap their minds around. Like, what does that mean? How do you develop that? Like, if you're talking to someone who's junior in their career who's trying to think about developing that critical thinking skill, like, what do you tell them? What does that sort of mean? How can someone start to work on really developing that in order to be able to help them along their path?

Yeah, we can definitely telescope into that, I think, two places where you learn how to do this very, very well if you choose to go down that route. I'm a big fan of law school in the sense that you have to be able to think through both sides of the argument very, very well, so much so that you can argue it. And because of that, I find lawyers to be very excellent at critical thinking. I also say scientists, and it doesn't matter if it's the hard sciences or the social sciences. Scientists are naturally critical thinkers because they come from a place of, let's assume I am incorrect, and let's start to take apart all the reasons why I may be incorrect. And if everything is left and I've already exhausted everything I could, then the logical conclusion is, there must be something right in all of that. And so if I was trying to teach someone whose very early career about, okay, let's hone in on this critical thinking, I would ask them, first of all, think of a Persona that isn't you, who's your stakeholder that's sitting across from you, that you have to convey this information, you have to communicate this information. And then I would Say, what is their context? What decisions are they trying to make? What assumptions are they making already? So in the context of marketing, you're talking to a brand manager, you're a junior analyst, you have this report on the campaign, and it's a mixed bag of results. What is the brand manager thinking? Oh, they probably did a really good job. This worked last year, so it should work again this year. There's all sorts of assumptions and context people bring into the table, into these conversations and putting yourself in their shoes to understand what are the assumptions they're making and therefore are those reasonable assumptions. Given the data, I think is a good way of beginning to practice those critical thinking skills. And another good statement I tend to make is assume you're wrong. Why would you be wrong? And trying to understand those points of view.

I love that advice that sort of start from assuming you're wrong and trying to figure out like, how do you poke holes into this is I think, exactly the right thing to do. And you actually just, I think, described it better than I ever have as to, like, that's the thing that I'm looking for. I, early in my career as an analyst actually worked with a lot of lawyers because I worked doing econometric consulting, like, largely working on court cases. And it is just now clicking for me, like, how much that honed my way of thinking of just knowing we're going to have to put this, we're going to have to put this analysis in front of the other team's lawyers and they are going to poke holes in it. And so it has to be ironclad. And like, that shaped so much of my approach to work. And I think it's, it's a really important skill and difficult to learn, but when you can do it, it makes everything that you do, especially in the analytics and data space, just much, much better. So thank you for talking through that. So you talked a lot about, like, just, you know, how do you make it? A lot of this advice I think was like, how do you have a great career in data? But I want to dig into specifically like working as an executive or working as a leader in data, because I talked to a lot of young analysts and I'm sure you do as well, who, you know, they're very solid technically, they're very excited about doing analysis and doing data, but they really struggle with thinking about how do I make that jump into being an executive or being a leader. What are the different skills, right, other than Excel or SQL or R or statistics that are going to make me successful in that sort of next role, how do you think about that? Or how do you describe what that difference is or what analysts or data scientists need to do to cross that chasm?

So a couple things. So first I spend an enormous amount of time with mentees and students trying to figure out is the motivation for becoming a leader to be leading others or is it more of you want to have a change in title or you want to have a change in compensation? And so being a leader isn't necessarily the only goal. As an analytics and data person, I think one of the beautiful things and why I love this discipline so much is that because we are so creative, you can have a very fruitful, expansive and influential career as an individual contributor, as a principal data scientist versus being an executive level analytics person. So it's not necessary to always shoot for the next title up. Sometimes shooting for the next experience laterally might be more suitable. So first becoming self aware about do I really love the actual process of the grinding through, asking the questions, getting the data, getting the answer and that I see type of individual contributor, type of workflow, or are you really about oh, I love understanding why is this person sitting across from me with a quizzical look? How do I influence the stakeholder? How do I get them to make the right decision based on the data? How do I help them influence their stakeholders? So we as an organization move the ship in the right direction. If that's really exciting to you, then yes, of course, leadership, executive leadership. But how do you get to the next level manager? I think the biggest challenge for those who once they become aware I want to be a manager is okay, your technical skills are great and you can continue to hone them with side projects, exploratory projects, all those things keep your portfolio fresh. However, the skills that you need to be a leader and a manager different from being an individual contributor. And those leadership skills have a lot to do with trying to understand the person in front of you and trying to understand their context, their ecosystem, their stakeholder set. And that takes an enormous amount of cognitive effort in learning how to listen, learning how to practice empathy, trying to understand the bigger picture that sometimes the precision is not the most important thing in the conversation, sometimes making concessions now so that you can win the next battle, get to the next loop, get to the next stage gate is much more important. And so understanding and learning how to balance the trade offs with that, that's what's going to help you get to a management position. And then you do that even better. Um, and then hopefully someone takes risk on you and you get to be an executive one day.

I think that was just a very clear elucidation of some of the really, really important skills. And I think 100% agree with everything that you just talked about. And I think one of the things that I would add is that it's like, it just requires a totally different energy and approach to work. And so if, if you're in IC right now and you're trying to develop these skills, you need to, like, think really hard about how to balance that energy, how to switch between, okay, I'm, you know, heads down on some analysis, and I'm going to practice some of these other skills that are going to be sort of draining and exhausting often, especially as it's a new muscle that you're developing. You talked about a lot of these techniques about empathy, about listening, about, you know, thinking about the person across from you with the puzzled look. What are the tactical ways again, that some this more junior person can start to practice? That is it just paying attention to this in meetings? Are there, like, books that you recommend or, like, how do you go about really, like, taking those first couple of steps? Especially if you're not in that leadership role today, you're just an ic, like, how do you start doing that?

So I always recommend to everyone to have a side hustle, volunteer situation going on. And what I mean by that is you're passionate about something. So put some of that energy that fills your cup into whatever nonprofit that may be of interest to you in those situations. You're going to run into so many people who have such a different background from you. So when you start to speak in a way that maybe sounds like a, oh, this sounds like a technical dialect, or ooh, that's too many AI words, or, ooh, you're saying words and maths, and I was not a maths type of person. You will get the feedback very quickly that, oh, gosh, I need to find a new way of communicating. I need to find a new way to be understood. And so I think having those experiences, volunteering could be quite helpful in building some of those, what I call durable skills. I don't call them soft skills because, to be very honest, they're kind of hard to do. Active listening, empathy, understanding the other person's perspective, especially when you don't have that lived experience yourself. So difficult to do so. It is not soft by any means. It's quite difficult. It's. It's hard in many ways. And so I Call them durable. Because regardless of the programming languages, whatever is the latest thing we all have to learn, whatever the technologies, as that continues to change and evolve, these durable skills transcend all of that. They'll be important no matter what tools we use.

The idea of working with a nonprofit to hone these skills is an amazing suggestion. Yeah, I think that that's a great idea because everyone, hopefully people who are listening to this, who are early in their career, you are passionate about something, and that really is a great way to get to experience working at different levels in an organization. A lot of nonprofits, you know, they need people who just step up and do things. And so it's a great. There's a lot of opportunity to do that that you might not have in your, like, professional paid job. So it's just a really wonderful suggestion. So most of the people that listen to this podcast or what interview series or whatever we call it, you know, they're in the marketing science realm. And you've had a very varied career, but a lot of the work that you've done has been in sort of very marketing first organizations or marketing second organizations. But we're marketing and talking to consumers is a very important part of what, what that business does. I'd love to, you know, you've had, you've mentioned sort of a topsy turvy career. You also mentioned lots of instances of just being like, that's interesting. That sparks my curiosity. I'd love to hear, do you have any good examples of times where that happened specifically related to marketing and the work that you've done in your professional career?

Okay, so I will limit it to marketing. So very, very early in my career, which I will not date myself, but has been a while. It was the earlier days of Programmatic, so we were transitioning from calling everything ad networks to start calling it programmatic. And I was working on a client, a large telecom, and they were producing a lot of digital advertising. And at the time they were giving different offers depending on specific parameters around the geolocation. And the traditional way was, okay, we have maybe four, five, maybe ten sets of different messaging and then everywhere to the US or everywhere nationally. And I said, oh, I think we can do better. Because not only were they pushing a lot of volume through the ecosystem, they were getting a lot of feedback. So if someone were, if a potential consumer was to click on an ad and redeem whatever the offer was, that was feedback. That was a binary yes or no, whether or not that advertising work. And so I said, okay, we are Going to teach this client how to do experimentation, but we're not going to use those words at all. It took me a few rounds of figuring out what were the right terms, because there's an internal vocabulary that clients have and each organization has their own to describe what they're doing in the marketing space. And so we finally landed on Champion Challenger. And the idea was that we would have certain segmentations that would be the champion for the week and then certain segmentations that would be the challenger. And holding all else equal, since we're all marketing researchers and analytics people, we would run that experiment, and every single week, there would be enough volume going through the ecosystem and enough feedback going through the ecosystem that we could actually calculate. Oh, okay. These segments, they continue to be champions. Oh, wait, they were challenged and now they've been pushed off the pedestal. Now we have a new set of champions. And every week we would practice, practice, practice that experimentation. And I was really proud of it. One, because it was novel at the time, and yes, it was a while ago. Two, it was teaching the client that they can do experimentation. And it wasn't so painful that it was too complex. We can't. We don't have time for this. We need to move faster, but we don't have time for experimentation. No, we show them how it was done. And more importantly for me, at the time, I was building a team. I was demonstrating to them that you can have influence far farther than just the output of a report. You can actually change the way your client does their business and you can use it. You can do all of that with data. And so, like demonstrating that the. That a team could have this capability, demonstrating the client they can do experimentation, building a model of experimentation without using the word experimentation. It was a beautiful experience. I was very proud of it. I've seen variants of it over the years. Not always the same name, but the idea, we would call it now AB Testing, but that's okay. But it was.

I like Champion Challenger better, actually. Like, that's a super compelling name that you did the branding on this. It's incredible.

Oh, thank you. Thank you. Well, it took a minute to figure out the name. The other time I would say I was really proud of this marketing team. I had marketers in a silo. Oh, no surprise. That happens quite often. So I had an CRM marketer, an email marketer, e commerce marketer, all competing for the same consumer's attention, all trying to drive sales. Okay, those seems like good goals. And everybody has their toolkit. Great. But they were Working against each other in many ways because they were uncoordinated. So what I had to do initially was I literally plotted the consumer's experience. Look at these emails this consumer is getting. Look at the messaging from the e commerce team this consumer is getting. Look at the advertising that's being pushed into the marketplace that this consumer is getting. Let's plot all of this. Hmm. Okay. So we're spending all sorts of money to yell at our consumer. And the big aha moment when I brought in finance with me to start the conversation was I said, okay, I am this consumer now. I just purchased this product. The lifecycle of this product is 30 days. It's supposed to exhaust. I'm supposed to replenish. Okay, great. Wait, what is this message I'm Getting on day 34? Oh, an offer to a percent off offer to replenish again. So not only have I maybe already replenished, two, you're making me have buyer's remorse that I didn't wait long enough for the offer, and now you're degrading the brand equity. Because I'm like, oh, well, I can get it cheaper if I just wait a little bit longer. Oh, no. Poor consumer experience, poor expectation management. And it's all not because anyone's. Everyone was going in trying to do the best for the consumer. It was just uncoordinated. Once I brought in finance, showed all the experience of the consumer, and then started talking dollars and cents, that's when the data started flowing and I could assign an analyst. And we ran another experiment. And so we were trying to figure out what was the optimal communication cadence to not only retain the consumer so they purchase again, but also minimize the subsidization because of offering. And so that was like the beginnings of real coordination within the marketing organization that did not exist before because everybody was in their own lane trying to reach the consumer, forgetting that there's only this. The consumer is one person. So when they get too much messaging from too many modalities that aren't coordinated, that's a poor experience. And so that was beautiful because I was getting a marketing organization together, but also I was giving them the credibility to speak to their finance organization to say, look how responsible we're being, and let's really discuss what we mean by this roi. And so building all of that in together was quite a beautiful experience. Also good experience for the analysts as well.

I love that you're talking about the relationship between marketing and finance. I think in. In a lot of the organizations that I speak to that is a very fraught relationship, a lot of tension, sometimes, you know, outright animosity between those organizations. I guess I'd love to hear from you, like, what do you think is the best way for marketers and finance to talk to each other? I think you just hinted at it, which is like, let's get together, let's talk about roi, what do we mean? But maybe can you talk about how you fostered a really healthy relationship between those organizations in, in, in your work?

So I. From two sides. So from the marketer side, what I was trying to educate them on was finance has very clear goals to hit certain budgets to report to the street. So they must make their numbers, especially publicly traded companies and hit the numbers that they need to hit. And so the agility is critically important. And so when you have very tough decisions that organizations need to make, you have to empower your finance folks to be your advocates when you are not in the room. And so being clearer to the marketers, here are the levers that your finance organization is playing with inside of either the budget, the P and L, whatever it is, such that they can hit their numbers. You need to understand how do you play into that? Which lines do you actually affect, which lines are really stressful for them. So understanding their perspective, their context in the conversation. On the other side, the challenge for many finance organizations, especially ones that are treated publicly, is that you have to hit your numbers for the year, you have to hit your numbers for the quarter, whatever the cadence is. And so there is already an incentive to manage short term returns, whereas marketing branding sometimes has longer term returns. So from the finance organization, helping them to understand the relationship with the customer isn't so transactional and utilitarian that it's only one purchase that matters. It's the purchase over time. Ideally, and I used to teach this as well, ideally, every organization would have radical clarity around what their customer lifetime value really is. I think many organizations have good estimations. I think many organizations have a good idea of it on single channel. But the holistic, true omnipresent, omnichannel, omni modal consumer, unless you're a digitally native type of organization, I have yet to find an organization that truly has it. And I think that's a key unlock if you really want the two sides to speak to each other in a more fluid way.

I guess, I mean the dream of understanding the full lifetime value of a customer obviously sounds wonderful, but obviously I think the reason why a lot of organizations don't have it is because it's Hard, like it's hard to know what it is, right? Because we don't observe everything like, are there key steps that businesses should be taking to get closer to that? Or what's the closest you've seen? Like, you know, if I'm thinking that I'm at, you know, one of these really large companies that distributes through lots of different retailers and sells online, you know, how do I start pulling that together to know what that full customer lifetime value is so that I can collaborate with my finance team?

I would say always start with your cleanest data sets, which is typically actually somewhere inside of your operations organization. So it's reaching outside of marketing because I think that's where you find the more clean kind of distribution data. And then when it comes to marketing channels, I am a big fan of digital channels. First, just because the data tends to be more plentiful, there's a lot of expertise in that area and therefore you're going to get people who really understand it and then bolt on and build on the other channels as well. Don't ever give up on trying to combine them. I was working in a sector where somewhat niche and so when it came to third party data suppliers for that sector, there really wasn't any there any there. The panels were weak. Like there was a whole bunch of challenges in trying to get data around the consumer experience for that sector. And so what they had done was they actually created a consortium saying, okay, me and my competitors, there aren't that many of us, we all need this data. And so actually, obviously, you know, avoiding any kind of conflict of interest or, you know, antitrust situations and all those things like, obviously within regulatory guidelines, they figured out a way to capture the consumer experience that could benefit the entire sector as opposed to just working on one specific firm. And so they didn't wait for a third party to solve the problem for them. They realized they would have to figure it out as a sector in, as a subsector inside of an industry, how to gather this data. I also think many organizations can always use additional support when it comes to consumer research because just as the marketplace is dynamic and changing, so is your consumer experience. And so I find it very ironic in many ways, and maybe that's not the appropriate word. I find it quite interesting in many ways whenever someone brings up marketing research prior to 2020, and the reason why I say that is all of our lives fundamentally changed as a result of the pandemic. The way we consume things has changed. Some of it has gone back to baseline. Yeah, sure, sure, sure, sure. But then we're also a few years older, so maybe our life stages have changed. And so I think there's a real need to do proper consumer research again and proper consumer experimentation, because the way we live is different now. And so to acknowledge that consumer experience and to try to better understand it, this is the work of brand consumer research that we need to do as an industry to really deliver upon consumer experiences that are effective not only for the organization, but really delivers value for the consumer.

You hit on a lot of different topics there. One of the things that I'm really interested in, you know, is this world that we operate in, especially on the analytics and data science side, whether it's doing surveys or trying to piece together data from different sources, or interpreting the results of an experiment or running an analysis, is the idea of uncertainty. Right. You look at, even if you're running an A B test, often there's some uncertainty associated with the results of that. You might be again, stitching together data from different sources where this shows some of the picture but not the whole thing. When I talk to a lot of analytics people, they often express this frustration where they feel like in order to explain the results of some analysis, they have to talk a lot about the uncertainty that's embedded in that. But they also feel very nervous because when they have to present these results to, like, executives, they feel like the executives don't want to hear about uncertainty. And so they really feel this tension of like, my CEO just wants an answer, but I have uncertainty and I don't know what to do about it. And I know that you've sort of lived in this tension. And so I'd love to hear, like, what is the advice that you give to people who are sensing that tension or feeling that tension in their organization?

So the not so easy answer is that you're going to have to work the stakeholder set. So if the uncertainty is a potential organizational risk, meaning if we bet on this product launch in this market and it doesn't go well, it could mean XX dollars for us in lost revenue, et cetera, et cetera. And so it's kind of a big deal, kind of. It's not a good time to present at the meeting all of those gigantic concerns. What it is a good time to do is to discuss. Oh, I have discussed this with stakeholder A, B, C, X, Y, Z. And from their different perspectives in their expertise in the business, we collectively have this recommendation, or I'm leading the recommendation. However, we've already looked at it from ABC xyz. And therefore this is the best option given the circumstances that we have to date. That's the best way to go about it, I would argue. In those types of meetings, what does that mean? What is the work behind the scenes? The work behind the scenes is, okay, so now I just described six different stakeholders. A, B, C, X, Y, Z. Maybe you have a finance person, operations person, the head of sales, a brand marketer. If you're in a multinational, you may have to go back to home base because you have stakeholders there working those relationships so that they understand the risk and that they have the alignment with you on the final recommendation. So critical, so critical to have a smooth meeting because by the time you're getting into C level conversations and you're talking to the CEO or you're talking to the president of a business unit, and these are individuals that every 15 minutes they're already making another decision that could potentially change the stock price of a firm. That is not the time to pontificate. And so for them, they want, they trust you as an expert to do the homework with their teams such that you're coming up with the best recommendation given all of that uncertainty. So it's a long way of saying you're going to have to talk to a lot of people who are not on your team who have different backgrounds from you and different roles inside of the organization. You're going to have to understand their pain points and what are the levers they have to help move the business forward and understand how your uncertainty in the analysis may potentially impact that and figure out a solution together on what the best recommendation is. So it's a lot of extra communications work and stakeholder management work, especially when you're working with the C level. But it is worth it because then that's how you get smooth meetings. And more importantly, that's how you get to a decision and action for the organization without a whole bunch of spinning.

I totally agree with everything that you just said and the way that I frame it. And this is sort of just, I think restating what you said is I say, I tell analysts you need to separate the recommendation from the analysis. And I think a lot of analysts, especially more junior in their career, they're very honed in on like exactly what the analysis says. But when you're talking to executives, you need to sort of abstract from that and think about what's the recommendation. And again, you sort of mentioned this like, executives are actually comfortable with uncertainty, but they want the recommendation. And so that is the Thing that you have to make sure that you're getting across or at least understand what is your role in this meeting or in this discussion? Are you here to make a recommendation, in which case you need to know what the recommendation is, or are you here to describe, you know, the full, like, the pure outcome of the analysis? And I think in what you were describing, it's like when you're talking to ABC xyz, you probably want to talk about, look, this is the risk. This is the range of outcomes. This is the full analysis. But then through those different conversations, come up with the recommendation. It's the recommendation that's going to the board or to the CEO, and that's the final step. And so both are important, but you need to be thinking about which conversation am I having, which part of the process am I in right now? And am I having the right conversation? Using sort of. That's based on the analysis, but that is also driving towards some decision for.

The organization and to add. Yes. And if I'm an analyst and I'm struggling to figure out, okay, okay, I need to have a recommendation, what is my recommendation? I don't know. I would think about, okay, well, if you were in that person's shoes, what is the choices to make? And for each of those choices, let's say it goes horribly wrong. What is the effect size of that choice? Because if we're talking point.0001 of a budget, okay, yes, losses are bad, but it's also learning also to make mistakes. But those mistakes are very, very, very small. So might as well learn something and take a risk, and maybe it works out, especially if you're very confident in the data. I think a lot of the times, especially at the beginning, lots of folks, myself included, I was very reluctant to make recommendations because, frankly, I did not spend enough time with my stakeholders to understand what is their scope of decision making, what is the scope of their impact in the organization. I didn't understand enough about their world to feel confident that I could make a recommendation based on the data. I seemed like an outsider, and it wasn't until I changed the mindset around that. And I spent time understanding their world, their universe, their decisions, and they need to make their incentives as well. That's when it started to help me build the confidence I needed to make those recommendations.

I'm gonna. Yes. And on top, I'm really loving this conversation because I think we're hitting on some.

We're doing improv. Yes.

That's a really important point that especially junior analysts and data scientists that they don't fully grasp. But I think what I'm hearing, I'm hearing a couple of points that I really like. So one is it's really helpful to convert the uncertainty into some business outcome, right? So like bringing in saying like, oh, I did this analysis and this is the P value and here's what a P value means. Like, you're on the wrong path if you're in sort of an executive level conversation. But it's really like executives don't often want to hear that. But talking about risk or what is the downside case and what's the upside case in metrics that resonate with the business, which is not a P value, but is dollars, right? What, how many dollars are at risk? That is a great. If you could talk about that, that really puts it into the language that the business understands. And that's a conversation that the executives are going to be really comfortable with. Here's the downside case, here's the upside case. Here's where I think we're going to land best guess based on the data that we have that I think in any executive is going to understand that and that can help them make a decision with you or think through what the actual recommendation is going to be. And so putting things into those terms, I think is a really great sort of next step after whatever analysis you've done. That's super important, important that gets you a lot closer to the business. And then the second thing that you're talking about is just context, right? And again, I think a lot of analysts struggle with this is they don't have the business context because maybe because they're early in their career and they've been focused on developing technical skills. But if you want to make that jump to having a larger impact on the business, which, you know, is sort of required if you want to go in the executive route. And also I just think good for careers, right? If you want to have that really big impact, you need to have the business context. And part of that is like understanding how to read a P and L so you can talk to the finance team. But it also is like if you're working in a big complex organization, you need to learn about how the operations team works and like, what do they care about so that you can do the right analysis and communicate with them and you'll have the context that you can convert the results of some analysis into something that is actually like an action that a business can take. And both of those things, I think are really, really Important skill sets for analysts that want to take that next step and not get trapped in, oh, I did this analysis and it had this P value, but no one listens to me, which I feel like. I talk to a lot of people that are sort of swirling in that and can't figure out how to break out of that pattern.

Something I also offer up to analysts who really want to push themselves to the next level. And I would argue pretty much all of us in all levels probably could use this too. The KPI I'm really looking for is time to business outcome. Why? Because to get to a business outcome, you needed a whole bunch of people, not you to take an action. Prior to them taking an action, a whole bunch of other people not you had to make a decision. And then prior to that, a whole bunch of people not you had to understand the output of your analysis. I think a lot of. A lot of the time myself, I forget this too. I sometimes conflate output and outcome. Oh, I outputted this analysis. Look, I made the machine spit out a number or I made the machine spit out an analysis. Oh, there's a number now. Great, fantastic output. Great. Done. No, you're not done until you get to an outcome for the business. And there's a bunch of steps in between that. And so you have to ride that wave. No matter how choppy the waters are, you gotta keep riding that wave to the very end. So that way, when it comes time to, okay, is this person ready for the next level or is this person thinking bigger picture enough that we want them to go into the next round of leadership? Oh, they understand the bigger picture. It's not just the P value, it's the actual outcome on the business that has impact on P and L, which has impact on what we're saying to the street. And so keeping that as the actual goal. Time to outcome. I think sometimes having that mindset shift can be quite, quite helpful.

Yeah, I subscribe 100% to that. I think at Recast, we work with a lot of people that are very analytically oriented. They come from, you know, statistics, PhD programs. And we spend a lot of time talking about this of, like, just writing the report isn't enough. You have to get it over to that next step of we have to get the things done. We can't just talk about it. We can't just write it up. We have to get other people to make the decisions to get the thing over the line. And I think it's super important. I think a lot of, you know, and I also had this, especially at the beginning of my career where I was just like, I just want to be an academic. Like I just want to think about this stuff and write things down and then everyone should understand exactly what I wrote and know what to do with it. And the sad truth is that that's just not how the world works. And if you want to have a really big impact in your career, like you have to work with people to not just do the analysis, but help them take that next step and drive some business impact. That's how you have an impact. That's how you end up getting the promotions and getting rewarded monetarily. But it can be very hard, especially if you're like me and you have this like very academic, research minded orientation that, you know, you're like, oh, I just want to like write the code and do the research report. Um, but if you just leave it there, the business isn't going to know what to do with it and you're, it's not going to have the impact that you really want.

Yeah, exactly, exactly. So again, there's nothing wrong with having a IC career. There isn't. And, and I think that's like something to unlock in terms of becoming self aware in terms of how you define your self worth. Individual contributor, such a beautiful career path. Just think like, just think about it. It's like a master craftsman in data. That's beautiful. You're going to create beautiful things with that and help other people understand the craft even better. But then it's also great if you want to go into leadership, leadership and executive types roles because wow, you get to impact a whole bunch of other people who may not have the same background as you. That's also beautiful. And on top of that, what's the best part? And this is again why I love this discipline so much. You can go between both. You don't just close one door forever. You can go from IC into leadership to IC to leadership big organization to small organization. Our skills are transferable and because our skills are transferable, the options can feel overwhelming because there are that many options. You just have to make a choice.

Totally agree. And for what it's worth, all of the, I think the impact stuff that we've been talking about can totally apply to ICs. And I think that again, those ICs that have the biggest impact and get promoted, they're the ones that are not just stuck in the Python notebook, they're the ones that are out talking to people and thinking about how do we take this thing and then get it out into the world. And so I think, you know, again, even if you want to be an ic, absolutely, that's great. But you still need to be thinking about how is this work that I'm doing going to have an impact on the organization? How are we going to go from analysis to real business outcome? And so you have to be thinking about it. Even if you never want to be, you know, informal leadership or be a manager, you still have to think about business impact. And I think that that is super, super important for everyone. I mean, sort of everyone who's working in, you know, a professional organization like, you sort of always need to be thinking about it. But especially for like analytics and data science people, it's really important to not forget that you have to take that extra step.

Yes, very true. Very, very true. Agree.

I think we, we both, you know, based on your earlier discussion, we both really believe in the power of experimenting, experimentation. But you alluded to this, which was that, you know, this early organization that you worked with, they, for a couple of different reasons, it sounded like, felt like experimentation wasn't for them. It was going to be too slow or too painful or for whatever reason. I'd love to like, hear you diagnose this. And I talk to a lot of organizations that feel similarly, like I think they should be experimenting a lot more than they are. But there's roadblocks I'd love to hear, like, why do you think that is? What are those roadblocks? Why don't companies experiment as much as maybe I think they should?

Okay, so maybe if we take a simple framework of input, some kind of transformation output, just three stages. So on the input side, I think many organizations, because resources are finite and not infinite, can have challenges in having enough resources for input. So if you don't really have much variability inside of some kind of experiment, if you don't really have a treatment versus control, that's going to be very difficult to get an output that's meaningful for your organization. And therefore you have input issues or you may not have the internal talent to do this because everyone is stretched already so, so thin. And so therefore we actually don't have any research specialists or analytics specialists to run this for us. And so we don't have people who can put on the hat of scientists for us or sometimes on the input side, oh well, the budgeting components just fell in so quickly that actually we thought we had six months, but actually you only have six weeks and therefore speed to market and being on time is Much more important than running in setting up all this experimentation. So there are frictions on the input quite often. And then I also find for organizations that have a non uniform geographic footprint, so I come from fmcg, fast moving consumer goods. So it's very often that you don't have. Not all markets are the same size. LA, NY are very large, other markets less so. And so you will always find the president's heads of marketing that says, oh, oh, you will not be doing experimentation in our largest markets that are going to handle our budget. No thank you, you can do it in the tiny ones. And I'm like, oh well, okay, we're going to violate all else equal. We'll have to figure out how to create a solution around that. So those are input issues on the trans, the middle component, the transformation component. Okay, well given the amount of feedback it would normally take, okay, I'll take maybe three weeks to find out whether or not there's a meaningful difference between treatment and control. Oh, I can't wait three weeks. I have to make decisions now because we're moving the business now. And so there's always this internal impatience around experimentation. So trying to hold all else equal is extremely difficult inside of a dynamic organization. So that's a challenge there. Or again, we don't have the talent internally or even externally. We may have the talent, but we have them on so many other projects that there isn't any bandwidth really to pay attention to doing the maths properly. And oh goodness, a truck from the D.C. left late. So actually we lost some points of distribution and we didn't realize it and therefore, oh no, experiment no longer. All is equal. And so that could be issue with execution on the output side. I find a lot of organizations, we don't necessarily have plans in place for if the outcome is A, we will do one, if the outcome is B, we will do two. If the outcome is C. Oh my goodness, we weren't expecting this. We'll have a default plan 3. So they don't actually think about what changes are we going to do as a result of the output from this experiment. And so not thinking through the full life cycle of experimentation down to action. And outcome, I think is sometimes a blocker for some organizations to do experimentation. The good news is there are, there are partners, there are ecosystems that help to remove some of that friction in the organization so that you can do experimentation. And I'm just talking about the technical components. I think there's also people in a process side of things that has to be addressed with experimentation. And so no one likes looking wrong in front of their bosses. And so sometimes when you run experimentation, you find, oh, we learned something new that totally did not work. And having that be shown to your boss, not through a framework of learning, but through a framework of, oh, that did not work in that market. If you don't have the right cultural values in place to have that psychological safety to run experimentation, you're going to find people very reluctant to do experimentation. I think also if you don't understand what are the decision structures inside of an organization, it can make it very difficult for internal stakeholders to. To take action or make a decision based on an experiment. Why? If we don't align incentives with their motivations, with their behaviors, it's going to be very difficult to create change. And so tying your experiments to the incentives, which leads to motivations and then behaviors, is a good way of trying to figure out, okay, how do I get more experimentation into this organization as opposed to bolting it on the side or trying to do it as an afterthought? I think a lot of organizations will do a lot of confirmatory analyses, meaning I've already made this decision. Great. I'm going to spin up resources to tell me I was right. So now I've done all my homework. It's in my back pocket, which is, okay, it's good to confirm. However, it's not necessarily expansive. And I think that's where the real value opportunity is for analytics folks to add value to the conversation.

So I think this was a great enumeration of all of the different challenges to experimentation. And now tell us, what are the tips for overcoming some of those challenges or some of those blockers. And I think interesting to direct this especially to people who are in the analytics organization, maybe sort of a little further down the food chain and are trying to, like, press this change up into an organization. Like, what advice would you give them about trying to work towards building more of this culture and practice of experimentation?

I would say don't always be looking up. Sometimes looking side to the sides laterally is also a very powerful move. Meaning you may be earlier career analytics. There are early career finance folks, there are early career operations folks, there are early career brand managers. They're all feeling similar pressures, trying to drive the business forward with limited information, build those allies now, build the understanding of what, what pressures their bosses have now. I would also say that I think it's critically important that I already alluded to the framework. So incentives, motivations and behaviors, trying to figure Out. And if this means having your boss help you with one initial conversation with maybe a n plus one or a peer to them to better understand their world, to understand what are the incentives inside of their world that leads to the motivations to the behaviors that will help you to make recommendations around. Oh, these analyses could be tied to what's incentivizing that stakeholder to take action and make a decision. And then I would also say every function, regardless of if you're analytics in a center of excellence type of situation or analytics at the edge in a business function or a line of business. Understand your financial calendar. Understand when the big meetings happen, where our finances, okay, resources are finite, we have a number to hit. We have very, very difficult decisions to make. Now, these things don't happen at a whim, hopefully not, because that would be quite tumultuous if so, usually they happen on cadence. So there's a right sizing that happens on a cadence. So understanding when that happens gives you immediate feedback about when the bosses are going to be stressed because they need to demonstrate either this is working or that's working, or we're going to hit our numbers or not. It's going to give you an understanding of how much time you have to make to take influence inside of an organization, because maybe you have six months to influence your stakeholders, or maybe you only have six weeks, or maybe you only have six days, or maybe you have an international visit happening that's going to kind of create more, more conversations around how to move the business forward. Those financial calendar conversations are so critical because it, it's that pulse of the business, like a heartbeat almost, of the business that helps you to understand, okay, this is how I'm going to ride this wave. When you work against it or if you're too late for it, I mean, the work is still good, you're still doing good work, but it may not be as effective as you wanted it to be. It may not be as productive as you wanted it to be. So understand incentives of your stakeholders. And two, understand your financial calendar when decisions are made.

Cece, that is incredibly sage wisdom and amazing advice. I think it makes perfect sense and hopefully all of our listeners will go take this and bring this to their jobs and their careers and maybe even their personal lives. This has been a wonderful conversation. You are such a clear communicator. Your frameworks, I think, are incredibly instructive. I have learned a lot today and I think now have a bunch more tools for communicating some of these really important ideas to my team. And the people that we work with. So thank you so much for joining me and for this wonderful conversation.

Thank you for creating the space.