AgileData #25 - Agile and Analytics - Shaun McGirr

Welcome to the Agile Data Podcast, where we talk about the merging of agile and data ways of working in a simply magical way.

Welcome to the Agile Data Podcast. I'm Shane Gibson and today I'm talking to Sean McGurr. How's it going, Sean?

Good, thanks, Shane. Good to see you again.

It's been a long time. So I think today we want to talk about agile and analytics and no doubt we'll rift into teams and people and process and all the good stuff. But before we do that, how about you give a bit of a background about yourself for people that don't know you.

Sure. Today I work at Data IQ leading a team of AI evangelists. And before that I've had a long career doing all kinds of things with data on many different kinds of technologies and.

Leading data science teams, being the first data scientist in a place and realizing on the second day of that job they don't need data science yet. That time it was lucky. I had worked for you at Optimal BI in Wellington and had learned lots of other useful things about all the rest of the data stack and the data practices. And I got my start in data by working for Statistics New Zealand as a holiday job doing admin things. And one of the tasks that someone gave me one day was A list of 2 printed out 2 printed out lists of numbers, I should say. And they asked me to tell them what numbers were in list A but not in list B. And at the time I didn't know what a merge or a join or really anything was. And I didn't want to rock the boat up front. So I sort of looked through and struck things off with a ruler and a pen. I guess this is 2001. And then I realized those printed out numbers were probably in a file. Can you email me the files? Put them side by side on the screen, let me copy that data from there to there. And that was how I realized if you stay curious and interested in solving these things that we call data problems, people just give you more interesting work to do.

Yeah, as you said, we worked together many years ago in sunny Wellington. I think from memory, that was before the term data scientist actually existed, wasn't it?

It was absolutely right on the cusp. So I had come there towards the end of my PhD program and had thought, okay, I'll just finish that off in six or 12 months, because that's easy. And it took longer. In the meantime, I fished around for ways to apply the data econometrics, statistical modeling skills I'd gained as part of my PhD and at the time I think we called it predictive analytics, advanced analytics, things like that. And I googled some things and optimal BI came up and I sent you an email and said, hey, I know how to do these things. Do you do those things? And then I think we started working together and I think that was maybe early 2014, and it was just when that term data science was really starting to take off, which, like all terms like that, you know, they have a usefulness.

Yeah.

And so let's talk about that one. Right. So what do you reckon, data scientist? Is it dead? Is the term, is the vision being realized and we're moving on? Are we still in the world where that term is still finding its feet or did it crash and burn?

I think the term is not going away, it's just here to stay. Like many others.

I remember lots of predictions over recent years about how there's going to be increased specialization so people will become a marketing data scientist or geology data scientist or finance data scientist. That still doesn't seem to have happened. And.

The reason that I still like and respect and maybe use the term is that I still believe even God forbid, in that Venn diagram of, of the value of a kind of person who can live in multiple worlds at once.

Who knows about statistics and the usefulness of statistics in understanding how machines have learned patterns from data.

And the ability to make those machines do some things at some kind of scale. And most importantly, the one that I always.

Lean on the most, and I'm most thankful for the training in my career that gave it to me, was the domain expertise. So do you even know anything about what you're talking about before you look at some data or write some code in it? The great lesson for me has been.

Any domain expertise from anyone I've hired or worked with. Any domain expertise in anything helps them gain domain expertise in other things. So I think the term used abused people. Employers claim that they can't find enough, but then everyone knows people with great backgrounds and, and skills who can't get hired. So it's all messed up. It's not helpful. But I still believe in the value of a person who can do those three things in that crazy event diagram.

Yeah, and I'll probably. For me, I've always. I know we started out with three, but I kind of formed into four steps in data science or four sets of skills. Right. So definitely with you in terms of the stats background. Right. The ability to experiment, treat it as a science project, prove, you know, prove the hypothesis and use statistical terms to do, you know, ways of doing that definitely around the domain, understanding, you know, the business and, and why it's important and the next best action or some action of insight. I think LinkedIn post about just yeah, what are we going to do with this? What value is it going to have for the organization? How's it going to help us make a better decision or increase the value or reduce the risk or whatever? I think there's, there's definitely for. When we talked about the data scientist unicorn in the early days, those equivalent of a full stack engineer, that person that can do all four things. There was a whole thing around business facilitation, plain English language, being able to talk to C levels in a way that they understood. I think that was a key skill of one of those unicorn scientists. And then the fourth one for me was engineering. They could write code, they could do a push pull, PR request, get merge thing.

And they engineered their code so it was reusable. And I think what I've seen when you talked about data science specialization, that kind of tweaked mantras because I think now we're starting to see specialization again in the engineering domain. So we now have a data engineer, a supposed analytics engineer, an ML engineer, blah blah, QA engineer. We're going into hyper specialization, data ops, data ops, data ops and engineer. But I'm with you. We don't see the specialization for the data scientists because we're specializing on that fourth part of the Venn diagram, the engineering part. But we're not really specializing on domain like you. I don't see a supply chain data scientist versus marketing attribution data scientists haven't seen that happen, which is interesting. But what I do see is we moved from an agile point of view. We got good ish at, at combining the data people and the visualization people and bringing them together. Right. We removed often the this team did the data work and they handed it over to the VIZ team who did a pretty dashboard and then the dashboard went to the user and the user went well that's not the right number and it's not the right way. And we all fought and blamed everybody else. We started merging those people together, right? Creating teams of squads that work together. But the data scientists tend to always sit outside, right? It was in the old days we went to ETL developers because I am that old analytics people, right?

They used to sit in another team in the analytics.

They floated somewhere, didn't they?

Yeah, they were somewhere secret. And the ETL team used to hand over data in the warehouse to the analytics team and see what happened.

Do you see that now? Do you see that.

Specialization? The data scientists still sit outside a core data team in the organizations that you work with?

Yes, yes.

The number of places they sit is bewildering. And in my career, I've been that kind of person and sat in many, many places, most recently aligned to strategy.

And it's really interesting why? I think part of it is that.

The most charitable interpretation I can give is that one of the best things you can give data scientists to do, if they do those four things or if the team of them covers those four things you said, because there's no one that does those four things that you said. If that team has those qualities, then they need to work on very complex questions and solve those questions and problems. Otherwise how do they justify their investment? And to do that.

They'Re going to need to look in all the corners and under all the rocks of the data. And having been that person, having received that data from that ETL team or data engineers or whoever.

I've always been suspicious about who made the decisions about what data is. Not in this data set in particular, I think the most charitable interpretation is if you get your data scientists working on those hardest things.

Part of their value is in actually questioning all of the other decisions that everyone else has made about data. That doesn't make them very good friends of anyone else in a data organization.

Yeah, it's often for me when I'm working with new data analytics teams, I get them to focus on handoffs, right? So even when they're working, if they're not pipelining, where they're handing across teams, right. They have t skills within the team so they can do all the work that needs to be done, they still hand off, hand off across people. And so one of the focuses I get them to do is how do you hand off, right. How do you describe what you need the next person to do in a way that they can understand it? And we know it's not 100 page requirements document, right. But it's also not. I just need five columns, right? And one of them is called age. So some of the stuff I'm seeing now is this term feature engineer, which really is, to me, it seems like a person who writes code that creates the feature flags and a bunch of columns to enable a data scientist to run the algorithms or the machine learning models. But what I struggle with, again, is now we have a handoff problem. How does a person that needs data for their model, explain the features they need in a way that somebody else can write them and prove that actually those features are the ones they want and that there's not been bias in the rules that have been used to create those features that they haven't seen that affect their model. And therefore it becomes easier for them to just create their own code. Right, because they're exploring the data, they're seeing features that may be important, may influence the models. They then create those features, they put them in the model, they see what the impact is, that closed loop happens amongst themselves, it's faster and, and in theory safer for them, but it's not scalable. Right now we have a person who's trying to bore the ocean. So what do you see? Do you see that feature engineering as being devolved outside of the science role?

This reminds me of a conversation I had with a Facebook data scientist maybe three, four years ago at a meetup group when they gave a presentation about whatever crazy cool stuff that they work on. And afterwards I asked them about one of my boring day to day enterprise data scientist questions of how do you get the data that you need? Because you have even more data than I am trying to go after and I can't get people to give it to me and you surely have all the data, but how do you know what to look for and how do you make sure you've got the right stuff? And the guy looked at me like I was an alien and he said, oh, we have an engineering team and we just slack them what we're after. And then, you know, 15 minutes later they point us to an S3 bucket where that data is sitting. So I think so many of these new jobs and new ideas are generated out of.

Companies with effectively unlimited resources to do this work, which is not the reality of 99% of everyone working in data.

So that's one end of the spectrum is gargantuan companies with infinite resources. At another end is the scrappy individual who has to make it all work themselves, otherwise none of it will will work. There's another interesting one, which is the company called Stitch Fix, who have a recommendation engine driven clothing subscription service and a very impressive data team and a chief algorithms officer. And they believe the opposite of either of those two positions, which is that a data scientist should be able to acquire all the data that they need, experiment with models, productionize and maintain all those models. And of course that means that their 200 data scientists are supported by 100 platform engineers to build all the things that make that possible?

I've not seen feature engineer as a job. It strikes me as an extraordinarily bad idea for traceability, for productivity. And also I think the thing sometimes, whoever is pushing that, if they're a data scientist and they think that they're getting away with avoiding some hard work, if you've got other people choosing what data to put in the model and machine learning is increasingly automated, you're pretty soon going to be out of a job.

Yeah. Do you see, when we work with teams, often get them to experiment with peer programming, and it's a really interesting experience every time, because for a seasoned developer, a seasoned engineer, typically they'll come back and go, but that'll slow me down. You know, me staring at a screen with somebody else, just let me get my headphones on, let me bash the keyboard and I'll get it done for you. But when they're open to experimenting with it and they do it for a while, typically what we get back is, oh, wow. Actually having somebody sit next to me made me think a little bit more about what I was doing and why. Because we're talking and they're asking questions, it stops me going down rabbit holes. Normally I go down a little side lane for three or four hours and often the person I'm working with goes, well, maybe we don't need to go there, or we just do this so they get some value back by having a pair. And we all know that when we use that approach, the quality of the code that comes out is much better for various reasons. Do you see that happening in the data science world, in the analytics world, where a data scientist and an engineer will actually sit together and work on the same code to create those features?

I think it's an extraordinarily good idea, and it's one that organizations are just engineered to be allergic to. Because this idea that you could get more value faster by having.

To what the bean counter looks like, two people do one job. This is not how any organization.

Understands how it creates value. I know personally, when I've been able to achieve that and it is easier actually sitting physically side by side, it is tougher over a shared screen because then you've also got all the other distractions of having five monitors and notifications. But the times when I have work side by side with another data scientist or an analyst, visualization type or an engineer type, all those benefits have become clear to me. And one thing that I always wanted to do when I was leading teams and never got a chance to do but I've seen a few customers push it this far in datarayku is to have those cross functional teams working on our problem. Because one of the things that makes creating the time for peer programming difficult is that. Well, because all of these data projects are such uncertain bets and because we have to promise so many things to so many stakeholders, we spread ourselves thin.

And so we're all working on too many things at once and so we can't even find the time to do all this work together. So what I always wanted to do was to say was to reach a point where the thing to build was so well defined that you would take an engineer and a data scientist and two analysts and you would make them a team and say solve that problem and you have no other job while you're trying to solve that problem. And I think in that context.

That version of Agile.

Would be revolutionary if we could ever get data teams to that level of clarity on what they're trying to build and that level of insulation from BAU and multiple priorities and waiting on data.

Yeah, it's interesting. I see the kind of. So one of the things I'm really passionate about is agile is a bunch of patterns and those patterns may or may not have value to the way you work. So while I like a lot of the scrum stuff and the flow flow stuff, to me they're patterns. And so what I encourage teams to do is find out they've got a problem and then find a pattern from the agile world that may solve it, experiment with it, see if it fixes your problem right, and then go on to the next one. So you know that that's great, but often we need to start somewhere. So I will often encourage a team to start off with more a scrum centric way of working. So you know, I articulate that well actually Al Shalloway articulates it better than me and I use his terms. As you know, we are doing small batches, so we have a team that are dedicated to doing one thing, one data product for a period of time and we're trying to reduce the cycle time for that batch to two or three weeks. And ideally we want to move to more flow, more lean based ways of working with data because it's more natural for the problems we have to solve. But if we think about that from a data point of view, we've started to see traction of Scrum being used in data teams. Copy the Spotify term, a squad of people that are cross functional and T skilled that can collect the data, combine it Consume it, present it. But we don't tend to see that in the data science world, right? We tend to, again, to see the data science teams sit outside that team. And one of the reasons, I think is coming back to that comment you made about knowing what you need to build. So when we have a team building a data product or an information product, we're getting better at understanding what it looks like, you know, what's the output. Oh look, it's a dashboard and it's got these KPIs and you know, we still haven't figured out what value we're going to get it from it ideally. But you know, we kind of know what it might be and we're getting good at reducing the scope, right? We're saying, well look, doesn't have to boil the ocean. It's not all the data. It's just a bit about a customer and a product and an audit, right? And so we can reduce our scope down and get it within that batch. But from an analytics point of view, we're working on a hypothesis, right? It's a bigger problem. There is no known set of requirements. There's a theory, a theory that I can grab all this data and if I use some kind of models, it can recommend to the users what they should do next, what movie they should watch next based on that. And that's a hypothesis. We don't know whether the data supports that. We don't know whether we can create a model that does that. We don't know that that model actually has value and the user would find it useful or annoying. So we've got a whole lot of hypothesis testing. And for me that's why having T school teams and analytics and micro batching the work to be done is a lot harder. What's your theory on that?

Yeah, completely with you. Every time I've tried to scrumify data science work, you end up with a whole bunch of tickets or whatever your construct is that explore data, explore data again, fix data, get more data, explain data.

There's, it's so easy to generate a series of discrete steps that still lead to absolutely nowhere.

Because.

The actual steps taken to test that hypothesis would typically actually be much simpler. What I think the deficiency is is having organizations form hypotheses to begin with. Because a hypothesis is doubt, hypothesis is uncertain, hypothesis could be wrong, a hypothesis can't be promised to be true. And so if you have data scientists and you set them out on these ambitious problems to solve, you have to accept that in many or most cases the answer will be there is no solution that's useful and feasible and, and implementable. And so a big part of the work of data science, any science actually, is to disconfirm to disprove hypotheses. And investing in being wrong is difficult for everyone. And I think over time. I guess what I've settled on is that.

You either need to split data scientists.

Into two groups, one who actually prefers the world where they test hypotheses and are mostly wrong, and a different group who take the ones that have been somewhat validated and turn those into products.

Or you have to have the same individuals be able to switch between those two kinds of work. And depending on the team size and the people that you have, one of those would be a good way to do. And the way I've visualized that today is you need a small cog very rapidly turning to very quickly invalidate bad ideas. And everything that survives has to go on to a much larger, slower turning cog, still powered right by the same little cog, but it turns much slower. And that's when the ideas that survived need to be implemented. And that's when it's mostly politics, getting other people to do stuff, enlisting engineers, making things robust. And so that view of the world that I have is a little bit inconsistent with the view that others have, which is that every data science experiment that you start should be built to production grade from the very beginning at times I've believed that, but I just don't think that's a way to solve the problem you outlined that you don't even know what is worth building out of data science in the first place to begin with.

Yeah, it's interesting because in the agile pattern world we have this idea of a research spike, which is there's a high level of uncertainty, so there's no point going and actually getting the team to work on something until we have more clarity on what it might look like, what it might take, how long it might be, some hints that this is worth investing in. I definitely agree. I think that idea of bringing research spikes into analytics, where we have a hypothesis and it's a high risk hypothesis, we need to do some early work up front to see whether it's got leaks. We all say fail fast, but nobody believes it because as soon as you fail, everybody jumps on you. But those research spikes, most of them should fail because you're starting to explore things of high level of uncertainty. And then once it looks like there's some possibility that this thing could have value, that it could be done. Then it goes into a second round which is in my view different people because it's a different mindset. Now I'm saying, okay, we know this is possible, but it's still hard. How do we do it? And I'd probably add a third one which I think is just turning up in the market, which is a reinvigoration of things that happened 10 or 20 years ago, as most things are. But it's this idea of analytical models that have been done time and time again in other places so the patterns are known. So if I take that concept of a recommendation engine, that's a pretty well known pattern now, right? So yes, the data is always different. We did an experiment where in our startup we've got a data catalog, right. So a bunch of tiles that you can see, the tables that are there and the data that exists. And I wanted to see how easy it would be to create a recommendation engine under the covers. So based on who viewed the data or who used the data and what you look like it would recommend. So effectively copying the model from Netflix but for data. And so we did a research spike on that. We call them McSpikeys and.

Well look, we thought this model was in the third cog, right? It was a proven model with lots of patterns and we're on the Google stack. So there was a product called Vertex AI or Vertex AI and it had a recommendation model for shopping carts and it was just as much of an out of the box model as you're ever going to get. So we did a mixed spiky on that and it killed us because that model was so tailored. The implementation of it was so tailored for user. Stick something in a shopping cart.

Let me guess, e commerce website with product description text, maybe images with colors in them.

Yep. But think about it, right? For me to apply that model for my team to, because I didn't do was like, well, we've got a tile that's just a product and we know somebody clicked on it or viewed it or queried it, right. So that's just some kind of feedback of that it was valuable. And all the model said was there's a thing and there's some feedback on how often that gets used and therefore you're going to look like that and I'm going to recommend it. But it didn't work for us. Right. I mean the, the key thing for us is, is part of McSpiky. We time boxed it, right. We said we have this amount of time to prove it has Value or it doesn't. And we went, no, right now it doesn't have value for us. So we, we binned it. So for me, even you know those, that, those third cogs, right, those reusable ones, you still got to, you still got to do the first little cog, right? If I take that thing and I apply it, is it going to work? What's the risk level? What's the value? And then, okay, we're going to invest in it. I think that's what we're going to see more and more of those third cogs. And then we're going to see people who specialize and take in those known patterns, analytics and applying them. And they'll become, because we've seen this before 20 years ago, industry models or domain models or those kind of things, and people will start to apply them.

We talked about that five years ago, didn't we though, Shane?

Yeah, I think we talk about it on a regular basis. Right?

It's nirvana. One day someone's going to define the meta churn model and customer churn model, customer lifetime model. I think that you're right, that the patterns on how to do it are clear. The underlying mathematical algorithms are clear. We have the computation now to do it all easily. What's different across the cases is what does a zero mean, what does a one mean? What does a 999 mean?

What does bad data mean? And that's, I think once we can use AI to solve that problem, then we'll be cooking.

Yeah. So I think one of the things I'd always recommend people do is when they're talking to their analytics team or their data scientists, be very clear about which of those three cogs you're asking them to do.

Can they though, who's this person talking to data scientists who is supposed to know which cog they are asking the data scientists to turn?

Well, let's go into that because you use the word Chief Algorithm Officer, right? And I was like, oh my God, I have heard this before. It started to come out. And so it almost got me on my rant, which was the cio, the Chief Information Officer. The key there was information.

It's not the cito, it's not the Chief Infrastructure Technology Officer. It's about information. But because that role never delivered for whatever reason, we then came up with Chief Data Officer. And now we're going into Chief Algorithm Officer. And so talking about roles, what the hell does a Chief Algorithm Officer do?

That's a great question. There aren't that many of them, I think Stitch Fix, who I mentioned, was one of the first to have one.

And one of the good reasons to have one at a company like Stitch Fix is that its whole business model is built around algorithms that select which clothes to stock. They build a bunch of clever human in the loop stuff so that a human stylist would select clothes for people, but the recommendations would be generated by algorithms. So they put algorithms everywhere and.

That'S what runs their business. So if that's what runs their business.

That sounds like a perfectly reasonable job title to have. But if you're. That's not what runs your business, if what runs your business is machines and manufacturing or.

Money or.

Anything a little older than.

After anything a little older than the 2010s I think we were mentioning before, then.

I don't know if you need a chief algorithms officer. And so that does raise that point of. If you and I both agree that there are two to three cogs whose job it is to know which kind of work fits in which cog. I don't think the business, which is not a term that we should all love, but I don't think the people out in the business who need problem solved should or could know which of the three cogs needs spinning for which kinds of problems. I don't think data scientists are particularly good at knowing which one. They like to go down rabbit holes. So something that I'm seeing emerge is analytics, translator, data product manager, people who can actually straddle the world of what actually is going to solve some real problem in the world, but who knows enough about the work that they can help the right people get on the right cogs and spin them.

Yeah, there's a term out there that came out of Juan from Data World and he talks about knowledge, knowledge scientist, I think it is, which is that translator. And I love the idea, I hate the term.

So I think what we'll end up seeing is analytics product managers, or maybe algorithm product managers or algorithm product owners. And they will be that person that sits between the team that's doing and the executives that are funding to describe the trade offs and make that call, that value call. Right. So we see that with data products, with a data product owner, they're saying, I can invest the team's time in product A, I can invest the team's time in product B. We've done some initial work and we kind of understand potentially how big it is, how much effort, maybe we understand what it might be used for so we can have some form of value And I'm going to make the trade off decision because somebody's got to. And I think when we talk about those three cogs, we're going to start seeing that as some form of algorithm product owner that goes, oh, this is actually COG1. Right. This is a hypothesis with a high level of risk. We need to time box it. So they will make the call that, yep, it's worth it, but cog won it team. And this is when you run out of time or they'll go COG two okay, we have a little of surety. You know, this is good, let's go, let's invest in it. It's going to be a bit Rocky or eventually COG3. Right. It's a proven pattern. Let's grab, let's put it in, let's get the value and let's move on to one of the others. So I mean we have to come up with better terms than cog one, cog two, cog three, because it sounds like Dr. Seuss to me. But I'm sure somebody out there will make up a term for classifying them.

It's good. And I think the missing link there for me is that product word which we.

You and I discussed it a lot 5ish or more years ago. And I think it's great to have seen a little bit of maturity come to that term in data in general. And I think through that it will and it should eventually impact this data science and this world of algorithms as well.

Yeah, I mean we might almost see a term analytics as a product, but the problem is it starts off with the word us. So it's.

So interesting with that cao though, I'm assuming that person's sitting at the top table in that organization, right?

In that organization, yes.

So for me that's the key. Right. Whether you call it a cdo, a cmo, a CIO or cio, whatever, the person who's helping with data and algorithms is sitting at the top table. So they have a voice, they have money and people to make things happen. Either they own the team or they can control what's done by the team that somebody else has owned. So they have the ability to execute and they are empowered to make the value call on behalf of their other C level peers. So obviously they've got to talk to them and all that kind of stuff. But the CFO makes a decision typically of when finance needs a new finance system or more finance people or a new portal for procurement or whatever. Right. And so the person that's got The C level for data or analytics needs the power, the mana to be able to do that as well, I think. And I think that's one of the things we look at when you look at a large organization about how data driven they are, is who's at the top table with the data and analytics view.

It's a great proxy, It's a great proxy. Who's at the top table, who's got that. And then also given that that's almost never true, how far away from the top table is the person with that credible view of data? And I see a range of people in my work. I see people who should be that person, who have almost no team and almost no influence. There's some kind of cheerleader. And then I see people who, who do run a whole organization, but they're not high up the organization enough. And often that's just data is not actually that important to that organization. I think.

It'S rare still to see a chief.

Data officer, algorithms officer, information officer, who is truly as influential.

And accountable and trusted as those other C level exists.

And it's quite interesting. So if you put the 2010 Koolkid lens on it, right? For companies that started in that genre and are data driven, just thinking about it, I wonder how many of them it's because one of the co founders is a data person.

And they're at the top table by default, right? So they're driving the vision of the company to say data is more important than most other things and that's why they behave that way, where for companies before that time, the hierarchical fiefdoms have already been created and therefore there is nobody at the top table because that's not how the organization was created. That role didn't happen 20 years ago.

Yeah, agree completely. And.

I talk about it often with large enterprises.

Government departments.

Who keep saying that they want to be data driven, and even CEOs say that, but they leave the pesky details of how to other people and those people don't have a mandate. And then someone starts a data literacy program and then a bunch of people learn how to read data, but they have nowhere to apply it, no time to apply it. And even if they did, what would change about the organization? And so unless you've got a shining example at the top who is constantly changing behavior and the direction of the company on the basis of data in a way that.

Meets all those quality standards and rigorousness standards that we like, it's really hard to see where it comes from at an organizational level.

Yeah, and we see that same with agile. So outside of the data space, when we work with teams at the bottom and they start adopting a new agile way of working, they.

Most times they are successful, depending on what you describe as success. But when we see an organization try to do an agile transformation where the goal is become agile, not change the way we work, and agile patterns might help us be a little bit better. And the CEO is not coming from that background. So they don't meet with their peers on a daily basis to check in on how they're going and they don't plan together and all that kind of stuff. They're not agile, so therefore the organization never will be. And so I suppose it kind of comes back to with data. That Tom Cruise movie Moneyball, I think it was show me the money. If you're in an organization where somebody at the C level is yelling at you, show me the data, then you've got a chance of being data driven. If they're not doing that, then it's hard because the organizational culture is not about data.

Just on that point of.

Bottom up versus top down agile. What's your experience with safe?

The way I describe it is I am unsafe. So I try and keep an open mindset as all agile people should. I have a closed mindset. When it comes to safe. I see it as a prescriptive methodology that doesn't help you describe a way of working. It gives you a structure that looks like your organization with slightly different slippers and names on the the doors.

I've had limited experience, but it does come up sometimes in my work where people ask us at Data IQ to explain how.

Our product, which is just a data science and machine learning platform, will fit into a safe organizational architecture.

And it can be quite challenging.

Yeah, well, as we know, we probably should talk about on another podcast, my prediction is this year and next year you're going to be asked how it fits into data Mesh. Just as much as safe. But going on to a different subject then. So you've talked a lot about what happens when the first data scientist turns up at a company. And I think one of the stories you talk about is data science turns up. There's no data. The answer should be data scientists leaves company until data turns up. But in some of your other podcasts that you do, you also talk about this scenario that I love, which is data scientists turn up and they start hassling everybody in the organization to send them that spreadsheet. They start crowdsourcing the data that does exist and they're not putting it in a warehouse. They're not going and doing automated data collection. They're just getting somebody who has it to send it to them and number eight, wiring it for a while.

One of the things I struggle with is that chicken and egg scenario, right? So if we wait for the data, then we don't know what the hypothesis, right? We don't know what that small cog is. We're just going to go collect data. But if we have somebody come in and there's no data, then they have to have their engineering capability, right? They have to be able to grab whatever data is there and make it usable before they can go in and prove that small cognitive. So for me, I don't understand why the model isn't. You hire a data scientist and an engineer and they sit together, pot of two, squad of two, right? That's the de facto what it should be. And yes, you're going out and harvesting data that's never been harvested before in a semi ad hoc way. But you try and automate as much as it makes sense on day one, right? You're not doing a sprint zero where you're building an analytics platform out for 12 months. You grabbing a bit of data in a couple of days or weeks, building something that is semi reusable but not really, you know, that has value, you'll do more work on it and make it more automated. You know, what's your view? You know, what's happening out there is, is the pot of two something that's going to happen? Are we going to start doing, you know, you can't have a data scientist until there's a squad that does everything. Are we going to stay with, you know, data scientists don't turn up until the data warehouse or modern data platform's in place.

I think. So when we talk within data IQ to companies of all sizes about do they have an ambition to apply data science at scale and to have lots of different kinds of people do that, different skill levels do that. One of the most common objections to that kind of scaling or democratization of data science is well, we first have to get our data lake perfect and then we will think about that. So there's one common response which very much fits the pattern of get all the data, get it all right somehow. I mean, I've never seen perfect data.

But it's a very nice dream to have. And then we'll think about interesting things to do with it. But we really need to at one end, there's that at the other end there's the to hell with any reusability structure. We're just going to let a whole bunch of individuals loose.

Truth, as always, is somewhere in between what I see in the teams that I work with or advise or ultimately try to convince. There's still a huge amount of functional separation in those data teams. Maybe I'm in the UK now and I don't know. The teams are bigger and so there's more refuge in just talking to your data engineering colleagues. But it's just a tiny handful of at least our customers and our prospects who I've met who are doing something as radical as having an engineer and a data scientist work together. In that way, almost everyone is still running. I have a data engineering lead, a data platform lead, a machine learning engineering lead, a data science lead, some other people who might help those people talk together.

But there still seems to be a preference, at least in this market, for different teams doing different work and sometimes using radically different tools to do that. And I understand where that comes from. It can't be the future.

Yeah. So it's interesting. So I think scaling is hard. I don't think. I know one team of five people can rock it as soon as you start to scale. We now have to specialize. Our natural action is often to specialize based on the process. We become a factory where the person puts the diode together and then the next group team puts a diode onto the circuit board and the next team solder it. That factorization.

Is kind of a known process and I don't actually have a problem with that because that's a flow and lean based process. From an agile point of view, we focus on different things though. Right. We focus on how that diode's handed off to the team that put it on the circuit board. So in that case, we focus on how the engineering team hand off the data to the analytics team and then how do we deal with the uncertainty? Because in a factory we know where the diode is and we know where it goes on the board. You're not randomly choosing a new place, whereas.

And we're making a million of them a day.

Yep, yep.

Of the same thing. Yeah, yeah.

And that's not what we tend to do with data and analytics at the moment. So the key thing for me is you focus on how you're scaling, how you're specializing a bunch of those teams and then you focus on the friction between them and reducing it. But what often happens with large organizations, we're retrofitting it, we're trying to take these new ways of working and retrofit it to a bunch of people that already exist. So we start off with a scaling problem. We've got 20 people in our data team and 15 in our engineering visualization team and five BAs in the BA team and 16 data scientists somewhere. And now we're trying to retrofit a new way of working with them. And that's hard. So my answer is treat it as COG1. It's a hypothesis. Grab a couple from each of those teams, put them together, tell them that actually their goal is to find a new way of working. And to do that they have to actually build some analytical products. The analytical products themselves may work or fail, it doesn't matter. It's the way of working that they're experimenting with. That's the hypothesis. And then once that works, say, how do we apply this again and again? How do we scale it? That's what I recommend people do. And yeah, done it with data. Data teams, not so much with data teams that have analytics people in it. For some reason, we treat them as two different things.

It's amazing how long, how quickly 50 minutes goes. So before we close it out, anything else top of your mind around agile and analytics?

It's funny, one of the presentations I gave at Optimal, I think, had that title, Agile analytics, and that was maybe six or seven years ago now. I wheeled it out recently when someone on Slack at work asked, does anyone have any recommendations for how to organize.

Analytics teams, data teams, data science teams, according to agile principles? And.

The thing that.

Our prospects and our customers really want advice on is what they call operating models and organizational change. And how do they become more data driven, more AI driven. Very few of them ask us about what we've been discussing today about organizing the work of the teams and how they actually do that, which is fascinating because how people do their work is to me, the most interesting question. I think the technology has come a long way in five, 10 years. A lot of things that were actually technologically quite difficult in terms of handovers and other things that friction is reducing. And so I think that the teams that do crack those new ways of working are going to have some kind of edge and going to deliver more value. But it is interesting that this discussion we've had is not so much one that I have in the industry.

And the customers that we have. And I think that might be when you're working for a software vendor, you're talking to a kind of data leader who's making a procurement decision.

And they're looking for some piece of a technological architectural puzzle. How you do the work day to day is.

I don't know how much thought people are giving that.

Yeah. So I don't agree. It's only software vendors that cause this problem or have this symptom. So.

I look at it, I've got two examples that I can back that up with. So one is I'm still side hustling consulting with big companies to bootstrap our startup. And so I will do either an engagement where I go in and help their teams or, or I'll do an engagement where I create a blueprint. Right. So not a 500 page strategy that nobody ever used, but a smaller document that has the components that you need to get started. And one of those is the operating model. And to understand that, I typically get the team that currently doing some work in there to explain to me their data supply chain. What are the steps you take from the beginning to the end. And then we have a conversation about how much of that needs to be centralized or decentralized based on the size of the organization, the way you want to work, work. So that operating model is really, really important. You know, doing a side hustle at the moment where we're evaluating and implementing a massive modern data platform. And you know, day one I said, first thing we need to do is understanding the operating model and the supply chain. And we haven't done that. So now we are putting in tools and technologies without understanding who's doing the work and how they're going to do it. Right. That's high risk. That's COG1. The second one is if you look at all the stuff that's published, even from the, you know, the, the 2010 cool kids, they publish their platforms, they publish their architectures, they publish their code, they really publish the way they work and their supply chain. And Spotify did it and we abused it. Right. The market called it the Spotify model. McKinsey went and made it their bread and butter. And so Spotify are no longer sharing with us their way of working, which is, which is gutting because, you know, they invested a lot of money in COG1 and hypothesis testing ways of working and hypothesis testing the organizational structure of their data and analytics team and their supply chain and then sharing it with us and we did bad things to them so they no longer share. But yeah, I'm with you. Right. It's, we don't talk about that, we talk about technology. But it's not just software vendors, it's Everybody.

Yeah, it's interesting. Something you said, taught me a long time ago, was that whatever you're talking about in this data space, you can boil the work to be done down into three steps. Get the data from somewhere, do some stuff to it, and put that result somewhere that matters.

People forget that and they go off and buy some things and they've never even looked at that supply chain, which is just a set of operations of get data from somewhere, do some operation on it. Even if it's an algorithm or some kind of fancy AI, it still fits that pattern. And there's both the supply chain end to end of those operations and then within each of those operations there's more getting of data, doing stuff, putting it somewhere, and a little bit of analysis of how people currently do that and how they might want to do it goes a long way.

Yeah. And I'd probably, nowadays, I think I'd say there's five steps, right? So that's the factory, right, that's doing all the work. I'd say there's a step before and a step after. Now step before is prioritization.

You've got a bunch of constraints in terms of the people. How do you decide what work should be done now and what you're going to hold off on doing? And the last one at the end is proving the value was delivered and the action was taken. So how do you know that that investment decision you made was worth it? And those two things need to come into that workflow now. And on that note, I think we should call it a day. So thanks for your time, Sean. It's been awesome talking to you as always. And I really enjoy actually talking to somebody that's got experience, both working with analytics and hands on and with teams, and somebody that's kind of had an agile mindset from day one as far as I'm concerned. So it's been great talking to you.

Likewise. And yeah, we'll have that Data Mesh discussion one day.

We will indeed.

That Data Magicians was another agile data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata IO.