Stop Building Data/AI Products Nobody Uses! How UX & Product Boost Adoption

All right, we should be live, I think. Okay, when I open LinkedIn, it's not— I guess because there's a few seconds delay.

Yeah, it's probably a little bit of delay.

Well, I guess we don't know it yet, or we can't see them yet, but people are coming in and we're saying hi. So welcome everyone.

Welcome LinkedIn peoples of the world, unite. What are we doing? I'm doing great. I'm live from sunny Cambridge, Massachusetts, in the best part of the Maybe the best part of the year, these like 3, 4 weeks into August, early September. Super nice. So yeah.

Nice. And I'm calling from sunny, though, I mean, the sun's gonna set in the next hour or so, uh, sunny Barcelona in Spain.

Nice. I'll take that over Cambridge, Mass., 9 months out of the year.

But not this month. Yes. All right. So yeah, I guess—

What are we talking about? Why are we here, Nick? Yeah, tell us.

Yeah, uh, I'll tell you why we're here. We're here because something like 4 years ago, uh, I think it was probably still the pandemic, I was on my phone looking at podcasts trying to find something relevant to data product management and came across your podcast, Experiencing Data, and I started listening to a bunch of episodes all in one go. And I was like, I really like how this guy's thinking about things. He's putting the non-tech stuff that tends to be the reason why a lot of data initiatives fail kind of at the forefront. And that's why we're here. That's not— I don't think that's what you meant with your question, but that's how we've ended up here.

Origin story.

I do love origin stories in popular culture and And didn't work as well.

Yeah.

So yeah, we're here to talk about the importance of putting user experience design, product management, all those squishier, harder to define things than data architecture and data mesh that most people spend a lot of their time kind of talking about.

Implementation. Yes.

Yeah. Implementation. And I mean, I think in a way this is part of implementation too, like thinking about the end user, thinking about the business problem we're solving, thinking about will people actually use this, or is it just a theoretical solution that makes sense in the lab but it doesn't, right, when confronted with reality?

Right, excellent. Well, this is definitely a topic near and dear to my heart. For people, if anyone has listened to the show, you've probably heard some rhymes here.

So I guess let's start with introductions because I'm guessing if someone's listening, they've probably come across one of us or maybe they haven't. Do you want to go first? Tell us who you are. Why should we listen to you?

Yes. Well, my name is Brian T. O'Neill. My company is called Designing for Analytics. It's a company of one, happily for I don't know, 10 years, 15, I don't know, 10 years, something like that. But I help data professionals become indispensable AI and data product management leaders. So my beast, I'm trying to slay this beast, which is technically right, effectively wrong data products and helping teams avoid making those because I think there's a lot of waste of talent and there's so many smart people working in the fields of analytics and data science. trying to bring data and decision-making into organizations or through software interfaces for end users, and a lot of this intelligence from the people doesn't end up getting used. And the challenge there is requirements that often teams are building based on requirements— our requirements are what I call friendly lies. So they're delivered with good intentions, and customers or internal customers, as many of the enterprise folks probably referred to them. They mean well when they give requirements, but what we've learned in the, in the world of software product design and product management, which is where I came out of, is that you have to get behind these presenting problems, which are often articulated as solutions. I need data export. I need, you know, I need to be able to see X, Y, and Z charts in an Excel spreadsheet. I need an API to connect to this data source so I can query the number of customers that we have, and on and on and on and on. And when we learn to get behind what's behind these problems that are actually solutions, and the problem is actually obfuscated, when we learn to get behind that, we can start actually delivering outcomes with the solutions that we make. And so there are skills that can be leveraged to do that. And what I'm trying to help people do is really, how can I steal, borrow, and cheat from all the stuff that we learned building software products for money commercial software products, how can we apply that stuff? If you're new to software product management, this could be relevant to you, but particularly for people in the enterprise data team setting, adopting this product and design operating model, that's kind of— that's the solution to doing this stuff, is leveraging other skills that are not technical in nature so that you're actually producing benefits and outcomes for your customers and for the organization. And that's good for your career as well, 'cause you'll be able to say, here's the value that we're creating, here's the impact that we've had, here's how we've taken the mundane and turned it into delight for someone in procurement or someone in marketing or whoever your internal customer is. That's really what the mission is that I'm on.

Awesome. Yeah. I guess I'll take my turn. and say who I am too. Yeah. I'm Nick. I'm similar to Bryan, though for a much shorter period of time, also a company of one. I started Value From Data and AI a bit over a year ago. And my mission is similar, but I'd say I look at a slightly different part of the problem and spend more of my time there, which is the figuring out what are the problems commercially that we should be trying to build. I think that, and that those 2 things are not separate circles. There's a big, big overlap between them. But my background is in consulting and also product management. And because I've been in data my whole career, but actually I started politics, studied politics at university. Mm-hmm. I have always tended to be the person that is either officially tasked or unofficially tasked with doing all the work around the more technical specifics that the people with PhDs and computer science degrees were working on. And so what I found is there's a lot of effort that goes into, like you say, scoping out the right technical solution and thinking about it from a technology standpoint. And very often what I found in the data teams that I've worked with is you kind of take everything for granted when it comes to you as a as a priority or as a request. And for me, it's partly exactly like you say, we need to think about the end user, like what, whose pain point are we actually addressing and in what way? Yeah. But for me, it goes beyond that as well, which is when the business comes to us saying we want XYZ done, it's also partly our responsibility to help them, one, like challenge them and educate them on Why is it that we're trying to address these problems? And are these indeed the right things to be going after? And being in a data team, you have a sort of a unique superpower, which is assuming the data is not completely useless, you have the actual evidence that can be used to help prioritize where it is that the business should be investing its resources or what are the big bottlenecks that we should be targeting? And I don't know, it's funny because whenever I talk about what I do and kind of similar to you, I work with teams and either train them or help them with coaching. And when I explain what I do to some people, they kind of just go, well, yeah, obviously, or is that even a problem? And those will be my friends in consulting or maybe that come from a business management background. And usually they have a more Yeah, commercial lineage, either in terms of their own career or the people that they work with and report to. And then there's everyone else that's even like, what does that even mean? And you have to spend some time to peel the onion and get them to understand the problem of, hey, most of the time you're working on things that aren't creating value for the business, or they are and you don't even know what that value is because you're producing some data that someone takes. use it in a dashboard, maybe they make a decision which influences someone else's decision, which eventually means we sold something to a customer.

Yeah,

I think it's important to kind of make that value chain shorter and get data teams to understand what is the impact they're enabling. And if they're not enabling impact, how do you turn around and basically do something else? Don't build yet another useless dashboard.

Yeah. So we share some— definitely have some overlap. I consider you a kindred spirit. And this mission that we're on to help people produce things that matter.

It's a big mission. Like you say, it's a big beast to slay. And for me, I'm constantly reminded that you have all these gigantic, enormously well-funded tech vendors who— I'm not saying they're bad, but they're putting a lot of pressure on organizations to invest in data platforms, to invest in this kind of tooling, that kind of tooling, that kind of tooling. And there's often not as much of a counteracting voice being like, you know, whispering on the other, from the other shoulder and saying, hey, before we spend another $10 million replatforming our data, are we sure we're even working on the right things? Are we sure we even understand the end users that'll be using the outputs from all these platforms?

Yeah.

And we need more people like us that kind of have this overlap. And hopefully that's, that's what we'll get from the folks that have dialed in.

Yeah.

Before, before going just to, to the people that have dialed in, I don't know if you heard the whole introduction or part of it, but it's great to have you here. Uh, give us a comment, tell us where you're from, and also, yeah,

do you

have any, any points you want us to discuss, right? Like, part of the reason we wanted to do this live is to be able to interact. Otherwise, we could have easily recorded this at a separate time, published it, I wouldn't have had to cycle quickly to be here on time for the livestream and all that.

I'm just setting up my thing so I can see the, uh, chats. And yes, it'd be great to see where people are dialing in from and what you all want to talk about here, because God knows Nick and I can chat, right? Nick, I have a Question for you. What is the— in your experience with your clients and the people that you talk to, you host a lot of meetups, I know, like in-person meetups. There's a strong London, and I know you're in Barcelona. There seems to be a lot of in-person networking opportunities there. What is the blocker that the practitioners seem to have here? Is it I don't know how to do this work, or I know what I need to do, I know what the skills are, I know all this stuff can help me, but there's some other thing in the way, whether it's their confidence, or they just think the organization and bureaucracy is just too big to have an impact, or my team— I have a team of people, but they're not skilled for this, and they don't really want to do this work, and they're not incentivized to do this work, so change just seems really insurmountable. Like, what, what are the blockers, do you think, that are— because we've seen evidence that this is a consistent problem in the analytics, and then it was the machine learning space, then it was big data products, and whatever you call it, there seems to be a fairly high failure rate. Almost everyone I talk to, they can cite like they've lived this experience of, especially in the enterprise scale, large initiatives that don't go anywhere, dashboard deserts. Like, the problem is clearly evident that we have an issue building stuff that matters, that gets used. What is the blocker for people that actually could be in the place to make a difference here? What's stopping them?

That's a great question. And as you were kind of describing this, I was like putting a lot of conversations I've had with frustrated people into different buckets because there really are a few different buckets. For me, I'll start maybe with, I don't know, the spiciest take about a lot of data professionals is I often hear phrases like, Oh, the business just don't get it. No. Right? And I totally empathize with where they're coming from. I'm like, I understand that, you know, you're pushing for this, I don't know, big data quality initiative because a lot of problems ultimately boil down to issues you have to fix in the data because there's issues with the systems upstream or whatever else. But I'm like, that is not the— that's not the starting point that you need to have. And if your constant complaint is, oh, it's the business, you know, they're— Literally, I hear people, they're idiots, they don't understand. And sometimes it's about their business stakeholders. Other times it's about even their leadership, like the head of data or chief data officer. And if they're like, oh yeah, like our new CDO, I don't know, doesn't know Python or hasn't written code in ages or doesn't know how a random forest algorithm works. Mm-hmm. And I think I categorize this first bucket into you don't get it because the objective of a CDO is not to write machine learning models, right? Their aim is to be building alliances with the rest of the business, is to be figuring out where is it that your organization should be working with other departments to move the needle. It's not, oh, does my CDO understand how cool the model I built is? And I think with that group, There's a lot of people that just need to see a kind of mindset shift. They need to understand that, hey, if you want to succeed, if you want the business to value you and for you to be able to influence the roadmap of the business, you just need to put yourself in their shoes more, right? Understand where they're coming from. Understand why is it that they need something tomorrow and they're telling you last minute. Well, maybe it's because they're a sales rep, they're speaking to a client. really important deal that they, that's like a ticking time bomb before it can close. And the client asked for a piece of data to make the business case on their side for why they should invest in your products, right? If you understand that context, suddenly the salesperson being super annoying and asking you for data at 4 PM on a Friday isn't that unreasonable. I mean, maybe it's annoying because it's 4 PM on a Friday, But suddenly it makes sense and you can connect it to the wider objective of the business. Anyway, I've talked for a lot. That's my first big bucket. Then there's people that understand this problem, but they still don't know how to kind of bridge the gap. And for me, like, that's the kind of group I try and speak to the most. That's the group that I help with either training or I'm just like, hey, I write a lot of articles and posts on LinkedIn. I share a lot of advice there. Yeah, I think, I think those, those 2 buckets make up something like 85% of the people that know there's a problem, but they either don't know the root cause or they don't know what it is they need to do to correct it. Yeah. What

do you think? I would share your Analysis. I think a lot of that's true. I think I see a lot of maybe identity crisis. And this gets back to— I'm gonna get on my soapbox here, but particularly in the technical trades, about how we're taught in school. And school is about do this assignment, do it right, you get a grade, you get a score, and the score tells you if you're doing a good job. And how do you— what's the score if you're doing product management well? It's like downstream outcomes, metrics. It's much— it's a much harder game. And it's, it's uncomfortable if you're used to living in a space where code compiles or it doesn't, it errors out or it doesn't, I can see the problem, now it's gone. This is a very comforting place to be, particularly if you have a technical mind, because you know what that— and I know what this is like. I write enough code to be dirty and never for production purposes. But I was a web developer-designer hybrid, and I know what it feels like when you finally get it working. And it's such a— it either does or it doesn't. And there's not a lot of things in life that are so black and white. It's like, are you doing a good job being a parent? You know, there's no feedback, but there's instant feedback with technical work. There is not instant feedback with doing design and product management work in the context of data. The feedback happens over longer timescales. It's harder to see, it's fuzzy, it feels a little bit hand-wavy. But the evidence shows that technical-only skills are not enough to create change. And a lot of times, data teams in the enterprise, you are tasked with— really, when someone's asking you for an output, a dashboard, a data connector, an API connection, a model, a thing, These are outputs. What they're not stating all the time is what the benefit of the outcomes are that they seek, and we can actually dig into that. And it's not because they don't want to tell you; it's a lot of times is no one's ever asked them, and they haven't written it down and made that really clear themselves. So it's not that it's hidden; it is hidden, but it's not that they're withholding the information. This process of discovery, of understanding the unmet needs, this is. This is UX research 101, it's product market research 101, and you can treat your market— you can have a market inside of a giant enterprise organization. You can have customers and markets, and they're not necessarily swiping a credit card like a SaaS model and paying you that way, but you can play the game as if that was actually happening. And in the user experience world, we measure this instead of with financial metrics, we would look at Are— have we, have we created an improvement in this person's life or not? Well, in order to know if we've improved this salesperson's life at 4 PM, we need to know something about what it's like to be a salesperson and how they're using data in their life. And if we can't answer that question, we can't know if the delta has occurred because we don't know what current state is and we don't know what desired future state is. So we definitely can't know what the delta is. So by understanding that and caring, first of all, we have to actually care. Probably should step back to that. You talked about, you had a post or something, I think you wrote it on this for this webinar, it's not my job syndrome. And I think this is right on target because most of the job descriptions right now for data product management, AI product, they're not talking about this stuff and they're probably not ever going to talk about this stuff. But the implied need is definitely there. And if you can supply an answer to this and focus on the outcomes game, your career— the business is going to notice it, customers are going to notice it, your career is going to benefit from those things, whether or not it's in the job description or not. So if you just like doing implementation and delivery work, that's fine. We do need people to do that kind of work. I don't think that's the work you and I are talking about. I think people that realize, hey, I'm paid to code, I'm paid to build, I gave you what you asked for. It's not my fault if they choose to not use the model. It works. It's 92% accurate. That might get you somewhere for a while, but I don't think it's going to keep you, especially with all this AI stuff that's happening in terms of building, accelerating the delivery speed of the work that we're doing. I'm not sure how long this game is going to be played like that. So the next level is really to decide the change in the business, getting people to change what they're doing. This is a harder game, but if you can create solutions that help people make positive changes, that's like, that's gold.

You know, for me, part of the, the kind of craziness in so many, not even just data teams, in like IT departments and technology organizations in general, is that we've turned the engineer or the data scientist or the data analyst into just a coder. You know, you are someone that is given a Jira ticket requesting something, and your job is to just write the code that does it. And like you say, that's literally how we use LLMs. We give it a prompt, we give it some instructions, and it generates the output. But that's not why people spent 5 to 10 years at university learning maths and statistics and programming and all these other fundamentals. Right. It, you know, we've hired all these people who are expert problem solvers, right? That's what they're doing. They're using either maths or computer science to solve problems, but we're not letting them do the problem solving. We're just telling them, hey, someone's come with a request, you go and do it. And for me, that's a bit nuts. And it's one of the kind of mental shifts that once you start seeing either yourself, if you're a data scientist or a data engineer or a data analyst, or your team, if you're managing those people, it's really hard for me to unsee it. Like, for me, it's why I will almost never go to my team with just a spec and be like, code, go and make this happen, right? And even if I do, it's maybe because I don't know, we've really scoped it out, we've really looked into it, and I'm very confident that that's the spec. I'm still obviously gonna tell them, hey guys, like, this is the context, this is the problem in the business we're trying to solve. I think this is the solution we should go for. What do you think? And usually it's not even like that, right? Usually it's, hey, I just spoke to the stakeholder. This is their situation. I'm thinking this is what we should give them, even though they asked for something else, and kind of use their brainpower too to solve the problem. And yeah, sometimes if it's a super quick request, if it's urgent, whatever, you maybe shortcut the process a little bit, but that needs to be the exception, not how we do things. And this like management by Jira tickets, it for me, one, drives me crazy. And 2, I'm like, I'm gonna seek forgiveness, not permission, and just not do it anyway. Right. I'm gonna ask the stakeholder basic questions like, can you tell me why? Like, why do you need this? What is the question behind your question? And then I kind of, I think you alluded to it at some point when talking about like, how will we make this person's, like the user's job happier? I also really like defining success, right? What will it mean? How will we know that we've done a good job? Not in terms of, I asked for this dataset, you delivered me the dataset, but in terms of things like our market share is shrinking. If you can help us find the right insights to figure out why we lose customers and then we can intervene against customers likely to churn, we will stop losing market share as fast, or we will regain it. That's what success means in the context of the business. And once you ask your stakeholders, who exactly like you say, they've often not fully thought it through. They're like, oh, I think I just need some data broken down, like sales data broken down by region. And you ask why a few times and you realize that actually what they really need is to figure out why they're losing customers. It's like, well, hold on. We're the data team. We can help you actually answer the question, not just give you the data that you'll open in Excel, realize it's too much of a headache to make sense of it, or even worse, forget you even asked because by the time we give you access to the dashboard, you're busy with the next fire. For me, it's all about thinking, how can I become more of a strategic partner to the business? And because I came from consulting, this was very normal and natural because that's the only way you stay, you keep a client. Otherwise they're gonna get rid of you or go, you know, outsource to someone cheaper. But for so many people, it's a totally different way of thinking about things.

Yeah, and I mean, I'm the same being a consultant myself. I do interject, I think pulling from actually knowledge of sales, even when you're— you don't think you're selling, but anytime you're trying to get someone to do something that they maybe don't want to do, you're effectively doing sales. There's just not a transaction, a financial transaction happening, but there are skills from sales, from consulting, from product management and user experience. All of these things can be put to good use inside a data team to, to make your career stand out and to make your team's benefit really obvious instead of very fuzzy as to what are we getting for all this investment? You just asked for, you know, $50 million to replatform something, but when am I going to start seeing the results? This was the whole, you know, the data science thing a few years ago, right? Buy some PhDs, stick 'em in there, but when am I gonna see some results? And now, you know, CDO tenure seem to be, I don't know, last time I saw they're about a year, 2 years, you know? And I'm not saying it means the CDOs are doing a poor job, but it does suggest to me that businesses are not seeing the value being created here. Maybe it is and it's just opaque, or maybe it's just not being created. So there are other ways to approach this. But Nick, we haven't— there's a whole bunch of comments in here. Hi, Matt, Meghana, Stanley. We should probably maybe address some of these questions and comments. What do you think?

I was actually thinking when to make the same interjection.

Let's see.

Let me— shall we start with this one?

Sure. Time to market pressure has made AI features reactive instead of value-driven. Most companies are checking the AI box. What are your thoughts on it? Oh geez, this could be a whole webinar, but I think personally, I think we're— go ahead, why don't you go first?

I'll just give my glib answer, but it's also the short version of the long one, which is I don't think it's that different to before. It's accelerated and made it more visible. But so many organizations keep doing the whole like, oh, we're just going to build features because our competitors have done it. Was it a good idea? Is it a good idea for our users? Well, we don't know. There's X buzzword is the new hype. Oh, let's do the new buzzword. That's my short answer. I'll let you comment on it too, and then see if I have a longer answer.

Yeah, so I think, so I think most of us see this in software products. So there's kind of like, there's the, the internal, uh, use of AI inside of the business, and then there's the use of AI. And I, I gotta step back here. When I say AI, I have adopted what I see to be the majority of people's language on this. When I say AI, I'm referring to GenAI. not necessarily predictive machine learning models, that predictive analytics, all of this. The reality is that I, I feel like the massive people out there have shortened AI and they mean GenAI. So let's just get that out of the way. I'm using their speak whether I agree with it or not. So anyway, with that put aside, when we see AI and software tools right now, I think most of this is like when we saw social media interjected into every single system. CRM, stick it in there. Procurement software, stick it in there. What? Like, it was everywhere just because it could be. I think most of it is done poorly. It's, it's not improving experience. Um, it has, uh, it has potential for doing negative. And this is one of the first times where I think putting a feature in the software isn't just about either they didn't use it or it made it better. It's like you could actually make customer satisfaction worse with some of these solutions. And I think we're actually creating a mess. And it's going to probably be good for user experience people in particular, because at some point user experience quality is going to start to trump this as it becomes a commodity and accessible. And this feature is accessible to all of these companies. How we use AI at the right time to create better outcomes and value for people that's where you're gonna win with it. So that's kind of my short answer is we're kind of making a mess. Now, caveat, I think experimentation here is a good thing. I think companies should be placing bets, running experiments, playing with this new technology, exploring the art of the possible. It's the size of the bets that matter. How big are the bets that we're placing? Do we know what, as you come back to the idea of a success metric, How would we know if this bet did produce the return that we were looking for, whether it's financial or something else? And if no one has defined what that means, then your only success metric is, did we deliver some AI or not? And that's a very easy thing for everyone to look at, but I guarantee you that is not actually what the business wants to do, because that's called sunk cost. Yes, we shipped some stuff. Yeah, we shipped some more stuff and we shipped a new platform and then we moved that to the cloud and now we're going to move it over here and now we're going to do this. Those are all outputs, that's all delivery work, and someone was asked to do that work. But I guarantee you, once you get up to the P&L level, as you mentioned, that's not what the game is.

Oh yeah, I think— I mean, even when you ship all these AI features or you move to the cloud or new platform, whatever else, You are creating a business outcome for the vendor, right? Yeah. And I think for me, I keep kind of reminding, especially the people that were critical of like, hey, I think we're kind of accidentally doing what the crypto bros were doing, which was constantly look at different problems and business domains and go, oh, could the blockchain be used here? It's like, okay. But that's could it, not should it. Like, there's a lot of ways in which you could use blockchain or you could use AI to do something, but is that the optimal thing? Is it the optimal thing in terms of how fast it's going to be, how cheap it's going to be, how accurate it's going to be, how user-friendly it's going to be, yada yada yada? You know, is it good that we clutter our interface with yet another chat that actually is also so limited that it can't do nearly as much as what, let's say, ChatGPT could do? And I think for me, that's kind of the first cardinal sin. And again, we used to do that pre-generative AI times too, right? So many times you try to go, oh, there's this problem, what can we do with data? Or what can we do with traditional machine learning with a dashboard? Yeah. And for me, I've always been conscious of it. Like whenever I'm some kind of vendor, a consultancy or in a product company, It's okay. First and foremost, I need to understand that I'm here to help my clients solve their problem. But if the solutions I bring to the table cannot do that, then the responsible thing is to just say that, right? And to walk away, to disqualify them as a client. And that's what a lot of companies, I think, get wrong. I see consultancies get it wrong in the sense that they go, oh yeah, we can totally do that. We have, I don't know, cybersecurity expertise. And actually they don't, and they hire a bunch of 23-year-olds and someone from outside to deliver the projects. I go, why would you do it that way? Right? Just say, this is not a good use of my time. I'm not gonna help the client solve their problem. Let me focus on clients that need the specific thing I do. Right? And it's the same when you, even when you're in an internal team, not everything needs a data solution. Right? Like, like I said, very often actually the right solution is a better, cleaner user experience that is simpler not more complex. For me, one of the mistakes we make is that we think about what are we like, how do we get our stakeholders to say yes to things we want, to the budget requests, the new initiatives, whatever. Actually, the more important thing very often is to turn things down, right? To say this is not going to be a good investment or a good use of our time. And not because I'm being lazy and I don't want to deliver this thing for the supply chain team, but because the state of our data is not good because it's going to take 6 months to build and cost more than what we could solve even if we fix the problem by 100%. Anyway,

yeah,

like I said, we can talk about this for very, very long.

Yeah, I know a product— I think another thing with— someone was about asking, Magno, I think was asking about AI. I think AI, because it is speeding up delivery work, it means that the data product and any product manager really, but particularly in this space, I think the work of curating what work should we be doing becomes really like almost perhaps the most important thing that a product manager does is decide where should we allocate engineering and data science and all the technical resources that we have at our disposal. I need to prioritize. But that means you need to understand what are the changes that we need to make inside the business? What does the success look like for both the, the users and for the organization? And if we don't know what that means, it's really hard to do that prioritization work. And at some point you're just doing what someone asked. But I think surfacing risks is part of that thing. It's like, yes, we could make this. Here are the trade-offs: 9 months of replatforming work, 9 months of this, 6 months of this. Possible outcome is this. Do you want to take that bet, dear sir, dear ma'am? Helping understand that and looking at it. And here's the opportunity cost: 9 months on this means we don't get to all this other stuff. This is a lot of product management, is navigating these conversations amongst all these competing stakeholders. And marketing has different requirements, sales wants a different tool for this, someone else, the executive team, wants a new dashboard for something. You know, on and on and on. This, this work, this is a lot of the messy work of product.

Oh yeah. But for me, the— and you know, if someone's listening to all of this and going, oh my God, I'm so busy building pipelines or dashboards, and now these guys are saying I also need to do all this internal politics stuff, uh, a lot of this is much simpler than you might think, right? Sometimes it's as simple as asking just a couple of not even more questions, but asking some questions differently. And I'll give an example that I really loved from one of the people I worked with earlier this year. He had this like multi-year roadmap set by the top that's like, we've got this capability and you guys need to scale it into all of our countries because we're a big multinational. And before the kind of scaling for one of the countries that they were going to deploy in, he had a meeting with the stakeholders and he asked like, out of curiosity, how many people in this country are going to be using this thing that we're going to take 6 months to build? And it turned out the answer was maybe 2 people or maybe zero, whereas in other countries it was like a big user base. And so it's not that he had to convince this department of like, guys, because marketing has these other demands, do you think we can deprioritize this? They kind of just went, oh, if it's only 2 people and it wasn't a core thing of their activity, should we just take it out of scope. And it was as simple as that. And that's 6 months of work for an entire engineering team that just vanished because he asked one question differently.

Yeah.

I think it— for me, I, I like sharing this story because I'm like, just think about it. If, if you hadn't asked this question, 6 months would have passed and then you would have found out the answer when you'd have gone, why has no one logged into this dashboard? And maybe you would reach out to those stakeholders and they'd go, oh yeah, it's like we don't look at that stuff here. Yeah. So the, the— there's a lot of low-hanging fruit, and I'm always in search of that. And especially if you're in a data team that doesn't do any of the stuff that Brian and I are talking about, like that collaborative prioritization with the business and thinking about the end users, sometimes it's just asking a couple of questions differently when you're doing requirements gathering. You don't need to call it discovery if you're worried people will not like that as a term, but you should do that discovery and figure it out.

Yeah.

And over time the business understands, oh, actually these guys should be in the meetings where we talk about the broader strategy because they've got a lot of advice and good insights as to what we should be working on. Like it's a gradual process.

Yep. Yep. Let's look at some of these other comments just so some of the LinkedIn users know here in StreamYard, some of you come across as anonymous, so we don't know your names, but, uh, yeah, Matt had one. I'll read it for anyone who's, uh, vision impaired. Hi, I'm in Guelph, Ontario, Canada, working in a nonprofit context. Always curious about the minimum amount of work that will make a difference here where we're building out a whole quite simple data system with a small team and budget. What are the critical things to prioritize to maximize our chances at delivering value with analytics products?

Awesome question.

Okay. Yeah, you want to take that one? I can start.

Um, for me, the most important thing is to have at least some of the use cases for the— for that data system you're building already identified, right? Like, I, I'm very against build it and they will come. Even if you're confident that, oh, we just need to build a simple data warehouse so that we can then build stuff on top, Fine. Take a step back, figure out what are the most pressing, valuable, and easy-to-solve business problems that you can solve with that. Engage your stakeholders. And for me, also quantify the financial value. And when you do that, usually what happens is you realize that actually I don't need to build a super horizontal data platform, data warehouse, data system, whatever you want to call it. Like maybe I don't need to integrate data from all my, all the different aspects of my nonprofit because actually the most important thing is we really need to improve our fundraising and our donor database is a mess. So we're just gonna focus on donor data or like prospective donor data because we know the stakeholder we will do those outreach campaigns with. And then all the other stuff is gonna happen later, right? And so we approach it vertically. We do the whole use case, map it out, and build the, uh, the foundational data, not horizontally. But most people tend to do the latter. Um, yeah, that's my spiel.

Yeah. Um, Matt, I, I guess when we talk about the, the minimum amount of work required here, I actually want to just challenge your question here for a second because it says where we're building out a whole quite simple data system. Someone might have asked you to build a data system. I don't know if that was their ask or you're just saying this is the, the manifestation of the requirement that we got. But I can almost guarantee you that unless you're delivering this to a data consumer, like a data scientist who's building model, like a platform, the data system is not really what the organization wants from you, and it's probably not what a customer wants from you. They want some kind of benefit that's not articulated in your sentence here that they think will come from a data system. So this gets back into, again, what if, if we had a data system tomorrow, what change would we expect to see in the world, both at the organization level and at the level of the customers, that the end users, the people who are actually going to interface with this, whether it's a technical user like consuming a dataset from a platform, or it's an end user looking at a dashboard or a software application, the point is we should be able to define what that looks like. And this can happen through a series of conversations with a stakeholder, and then you might find out like, okay, the real goal here is to increase donations, and they think they need better targeting of their prospect database so salespeople know who to call. Okay, great. So this is The end user of this will be salespeople doing outbound prospecting. Now I'm going to switch, put my user hat on, and I'm going to go talk to salespeople, or more listen and observe salespeople about how they do this work now and where are they doing it wrong. Like, what's stopping them from correctly targeting prospects if the data is supposed to help them target the right prospects because they can only take 12 calls a day? and we don't want to waste 10 of them on people who have no interest in donating, then I want to see how do you go about picking the people to call today? And how, like, what's your win-loss ratio? Like, how do you feel about that? I want to understand that work context. And then ideally, in the user experience world, we would collaboratively build this solution with these salespeople. We would do research with them. We would do low-fidelity prototyping. We would show them something before it's We get feedback along the way. And partly what we're doing is also priming them that there's gonna be a change here, but you helped make it. You helped create this thing with us. We're just kind of here facilitating this change to happen so that by the time it rolls out, you've got a champion in that sales team who helped create this new thing that uses data to make their life better, which then in turn makes the business feel like we're on track, or the nonprofit org in your context. So, so success metric definition, problem space definition. Data system is not a problem, it's a solution. And then spending the requisite time observing, listening, watching users in their real world do their thing. And I'm literally talking about one-on-one time, sometimes two-on-one time, a note-taker, an interviewer watching a customer. Show me how you pick prospects to call today. It's 9:00 AM. You sat down, you got 10 calls to make. Who are you gonna call, Ghostbusters? Like, how are you gonna choose who to call? There's thousands of people in this database. Show me your process. That will tell us where could— oh my God, they're guessing. They're using alphabetical lists. Like, why are they doing it that way? Well, there's no other way to do it. I mean, I would love to do it by, you know, most recent donation made, or like zip code because that correlates with income, and— but we don't have a zip code field. So, and there's— even if we did have it, there's no way to map the people, and I don't know who my It's like on and on. All of a sudden you start hearing what their frustrations are that no one's ever asked them about. And now you have an opportunity to make that data system fulfill those needs. I don't know if that's the minimum work required. I think, I think it is. How deep you go? Just get started. I think getting started with what Nick talked about, even some of these short-term questions, just reframing some of these questions and getting out of the solution space. You're gonna find some gold there. And one of the changes that, like, when I've done training before, one of my signals that my training is helping my clients is when they come back to me and said, my customers or our clients, they noticed that we're talking differently. And they said, this meeting was so productive. Like, something has changed with our data people that come in and talk to us. They noticed the change just in the questioning that was happening, and the questioning was empathetic, and it seemed to be about them, not about getting the requirements, but like the salesperson is saying, I felt like you cared about my work and you really were trying to help me by asking me detailed questions about my life and what's in it for me. And this is empathy. This is putting our sales hat on and trying to imagine what it's like to be that person so that when we create stuff, it's for them because it's not really for us. And that's a lot of this game is none of this stuff is for us.

Oh man, there's so many things I want to kind of like pick on and build on and comment on, but I'll just keep to 2 very quick comments. Yeah, first is when I do those discovery sessions with end users, I'm always— I'm taking notes. Usually I'm drawing some diagrams to understand their process, and halfway through the meeting I'll be like, oh, let me share my screen to make sure that how I've written it down matches what you've been saying. And I find it so funny, but it's exactly for the reason you just said about feeling seen and feeling heard. Every single time I've done this in different companies with different stakeholders, the thing that everyone always remarks on is, oh my God, like you've actually been listening. And there's always a comment like that, or, oh, you drew all this out. That is And then by doing that, it also becomes easier for them to go, oh yeah, I also forgot to tell you about this step. Or no, actually this is done by Bob, not by Barbara, or whatever it might be. But you're right. I think so many people are not used to that collaborative way of working and that, hey, I'm actually here to help you. Not I'm here because I get paid 9 to 5 to come in, pretend to want to help you, go away, build something you're never gonna use because you've been through 5 digital transformations and nothing's changed. You still do your job in Excel. And it's a real step change.

Yeah, I want to get— yeah, on the question thing, just to put a period on the end of this, I want to give credit to Andy Sutton, who is in the Data Product Leadership Community with us, Nick. He's the, I think, Chief Digital and Data Officer at IDG, or I'm sorry, EDG, the Endeavor Group, which is a very large hospitality brand. I think it's the largest one in Australia. And he said, part of the challenge here is that the game of data product management is a lot more about asking questions than delivering answers. And if you've been in a data and analytics role, you're in the answers-giving business. But actually, if you're in product management, you're in the question-asking business almost a lot more. Yes, eventually we will get into solution creation and providing answers through interfaces or whatever it may be. But I think that's a really keen insight and it means part of your skill if you're moving into product here is to understand that question asking is a big part of it.

Oh yeah. And questions that let people actually give rich answers, not a simple yes or no or leading the witness to the answer that you want to hear.

Right, right. Yeah, it's— there's a genuine interest.

Do you like this thing I've spent a month building?

Yeah,

where do they— they'll have to be a very difficult stakeholder to be like, yeah, no, I don't like it.

Yeah, yeah. Well, that's a leading question too. Exactly.

I see, um, we've got— Yes.

Yeah, Hassam, uh, let's see. When you don't have direct access to customers for discovery nor usage data like non-SaaS B2B context, do you have any tips as to how to look for better understanding of the user? And what could bring value? Oh yes, I have opinions on this. You want to go, Nick?

I do very much so.

So first, I, I have a— I want to challenge you, and I, I don't have direct access to customers, so this is probably in larger organizations with bureaucracy. If someone's running a team or they're trying to adopt product operating model and you're in management, to me, this is one of the most important things that you need to do as a— if you're in management, is you need to open up the communication between makers and users and makers and stakeholders. Because if you keep having the— if you keep having intermediaries, like you can't talk to our customer service agents, but you can talk to their manager, You're going to fail because the manager really doesn't— management is giving you their biased opinion. They're oftentimes, oh, well, this is how they answer calls and this is how they log the data after the call, etc., etc. They're probably giving you an aspirational view of that, or they're giving you the way they always did it, but they don't actually know how all the line people that are out there doing this work, the 100 people in the call center, are actually doing it. They're probably not that aware of it, and they're definitely not articulating it to you in a way that would allow you to rely on that as definite fact. So getting those walls torn down is an important management thing. I do think if we position— if you don't have access, if you position the ask as being in the benefit of the user you're trying to get through. So you have a blocker, you have someone that's, that's blocking you there. The way I would try to phrase this is, hey, look at the track record. When we don't have access to customers, our track record is 1 in 10. That means 9 out of 10 projects and dashboards or whatever we make, they end up not getting used, and you're paying for that. And I don't want to waste your money building stuff that you don't need or you can't use or isn't right. And the fastest way to do that is for us to make sure that we have We understand how to build, create benefits for this other team. And so we can do it the way you asked, but there's a higher risk of us wasting time and money. And it also means your long laundry list, we're never going to get to that stuff. If you can't get through that, sometimes you do, you need to make something and you need to let, let it fail for the learning to happen. And so my game there is start small, Ship the smallest thing you can. Don't look at it as, I told you so, I knew this was going to fail. But instead, what's the smallest increment of work you can do to put it out into the world as a hypothesis? Maybe get agreement with this blocker about how would we know if this was successful or not? See if you can get in agreement with them. Like, hey, if no one logs into this tool— and this is normally a terrible user experience metric, but you could just start with that. If we launch this thing and we see basically no usage, less than 1 login per week per user in the first 2 weeks, would you be willing to let us go do some research with these customers? Because that— would that be a signal to you that this is not going well? Well, one, I don't care if one, blah blah blah, and they'll talk about the numbers and you'll come up with some— fine, pick 3, 7, it doesn't matter. The point is you now have a concrete metric By which it's not to say I told you so, it's to say good, now, now we know what our goalpost is. We're going to try to crush that by guessing what people need. Maybe you'll get lucky, maybe you won't. But now you have a metric to go back and say, hey, we've kind of struck out here. What do you think? Can we go talk to these people directly? That's— I know I didn't address all of your question, but I want to let Nick. Talk here for a second.

Yeah, I, I wanna focus on a couple of tactics for how to do this. Yeah. 'Cause my first piece of advice is to basically, and I'm not saying it just 'cause I'm Greek, but you need to take a Trojan horse approach.

Right.

A lot of the time, if you're told you do not speak to customers, right? Especially for discovery, I will do whatever I can to find some way to get access to some customer. Right? And I'll seek forgiveness, not permission. I'm not going to say like with— because I've tried it both ways. There's some people that when I tell them the story of, hey, I think I should go talk to the customer because maybe we didn't get all the requirements right. Very like most of the time I get told answers like, no, we did, just get started. No, we don't need to do it. Contract signed, whatever. And so I've started just being like, oh cool, like as part of requirements gathering, which is the natural next step or, like, you know, building out the detailed tech requirements. Um, we're just going to go talk to the users. And then I do all those things that you're talking about, right? Like the shadowing or understanding the business problem. And then when I come back with the results, um, things happen like, hey guys, um, in the initial brief you told us that we shouldn't do, um, what's the word, um, that we should only integrate with this one CRM, but actually when I went and spoke to the end user, I figured out that there's 2 different CRMs that we need to connect to, right? And there I've presented the actual finding of the discovery, which says, wouldn't it be silly if we don't integrate with both, if that's what's needed for the project to succeed? As opposed to, hey, I have a hypothesis that you guys messed up in defining what this project should be about. Yeah, that's tactic number one, like seek forgiveness, not permission. Bring the answer, not why you wanna go and get the answer. And then the second thing, 'cause I suspect with Hesham's question of not having direct access to customers, it might be because salespeople are being protective. And that's something I've had to deal with where I, you know, I go to either colleagues that are salespeople or in my last job, our salespeople actually worked for other companies that we were partners with. or that worked for a sister company. And so there was a lot of suspicion or reluctance from some of them because in a lot of sales organizations, you're basically used to hoarding, like holding onto your customers because if someone else gets hold of them, you might lose your commission. And so, but when I started understanding that, I changed how I talked to the salespeople and I also made it like informed them of some things that they weren't always aware, like, hey, if I help close this deal, you still get your full commission. We don't need to split the bonus check because that's the deal our 2 companies have made. But then again, and the second thing was again the Trojan horse thing. There were some salespeople that were happy to let me come on the call because they understood that I spoke the technical stuff much better than them and I helped them close deals. And so after a few months, it became very clear, hey, when you bring Nick on the call like these people have, we close deals. And when you don't, we're having this constant back and forth and game of broken telephone where the salesperson comes to me saying the customer wants that, and then we deliver it. And then the customer says, actually, that wasn't quite what we wanted. And there's this back and forth that literally takes 5 minutes of me going in, understanding the business problem, while knowing about our technical solution and translating back what the requirements should be. So yeah, seek forgiveness, not permission, and also understand where the blocking is coming from because it might be that it's something that you can kind of shortcut.

Yeah, I think I didn't used to think of it this way, but I think gradually and carefully understand what the resistance is because I think most people would generally agree you have a better chance of designing good solutions if you're allowed to talk to the people that are supposed to buy them or use them. So if you're not, if you're stepping in the way of that, there's a fear. And sales is an excellent example. I've seen the exact same thing there. They don't want you jeopardizing a sale. And a sale is something, especially in enterprise context, that takes place over possibly months, sometimes over a year. They don't want anything to jeopardize that. So I think developing relationships, showing that you're aware of the sales process, about their personal incentives there, and then positioning your work as being helpful to them and not something that's going to jeopardize that. And it doesn't take a lot for that team to have a bad taste in its mouth. And I have seen this before where user experience researchers have created a resistance. They either said the wrong thing, or maybe— I don't remember what exactly the commitment was, but sales got a bad taste and they just shut down because they were so worried about more damage being done. Some of this is a leadership problem because this is about people protecting their territory and bonus incentives and all this kind of stuff. But like, if you're really trying to do innovative work and you're not allowed to talk to customers, this is a systemic problem that needs that to me management needs to also be dealing with, but we can start small by understanding and talking to the salesperson about their incentives and showing that we understand and look for some low-risk opportunities to get. Talk to prospects, maybe closed lost. Say, hey, would you mind if I reach out to the closed lost? That language, that's sales language. That means people who finally made a decision and it was no. Those could be good people to go talk to. There was a reason, and you might learn something that you can feed back to the salespeople, hey, next time you reach out to— we learned a whole bunch of stuff, by the way. They actually were shopping for competitors and it was price procurement got in the way. They wanted— they actually said they wanted to buy you guys, but by the— buy us, but they didn't. You know, this is— anyhow, I don't want to get too into software industry world, but, um, I agree with most what, what Nick said.

But I mean, I've seen all those say like what— especially this scenario you just described of talking to the lost customer and realizing that It would've been so easy to win them. Yeah. I've also seen with internal stuff, right? Where it's like, oh, they hired this external consultancy cuz they thought we didn't know machine learning because we only do analytics. And it's like, no, it's the same people doing both. Something along those lines. So

yeah.

Hesham, in short, sorry, but we don't have a tip for how you can skip talking to customers. You need to find a way to do it.

What else? Do we have any other questions out there? Glad that was helpful, Matt.

We don't.

Almost at time anyways.

So yes, this was fun.

Thanks for— yes, thank you for hosting this first-time little LinkedIn Live session here. This is really good. I hope folks got use out of it. Nick, where can people like find more information about your work and your insights and all that. I know you publish a lot.

Yeah, well, first off, wherever you're watching this, so probably LinkedIn or Substack. Substack is longer articles. LinkedIn is memes and shorter hot takes. And if you live in London or Barcelona, we've got a data product management meetup that you can join that runs every month. Brian, what about you?

Yeah, home base for me is my insights mailing list. So that's at designingforanalytics.com. I also have the— host the Experiencing Data podcast, which has been around for about 7 years, uh, and that's easily findable, uh, on the website as well. So those are really the, the 2 best places to learn about my work.

Awesome. Well, Brian, thank you so much for joining, and thanks to everyone for dialing in. If you've got more questions, feel free to message us or to write them as comments. And yeah,

yeah,

hope to do this again soon.

Good. Yeah, thanks, Nick.

Thanks, Brian. Have a good day, everyone.

Cheers.