#47 Human-centered Design in Data Science

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

In this episode of Data Framed, a Data Camp podcast, I'm speaking with Peter Bull about the importance of Human Centered design in data science. Peter is a data scientist for social good and co founder of Driven Data, a company that brings cutting edge practices in data science and crowdsourcing to some of the world's biggest social challenges and the organizations taking them on, including machine learning competitions for social good. We'll speak about the practice of considering how humans interact with data and data products and how important it is to consider them while designing your data projects. We'll see how human centered design provides a robust and reproducible framework for involving the end user all through the data work, illuminated by examples such as Driven Data's work in financial services and mobile money in Tanzania. Along the way, we'll discuss the role of empathy in data science, the increasingly important conversation around data ethics, and much, much more. I'm Hugo Bauen Anderson, a data scientist at Data Camp and this is Dataframe. Welcome to Data Framed, a weekly datacamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host Hugo Bown Anderson. You can follow me on Twitter hugobown and datacamp@datacamp. You can find all our episodes and show notes@datacamp.com community podcast. Hi there Peter, and welcome to Data Framed.

Thanks Hugo. I'm happy to be here, I'm happy.

To have you here, and I'm really excited to be talking about human centered design and data science, the role of design in data science, and what data science can tell us about human centered design as well. But before we get into this, I want to find out a bit about you. What are you known for in the data community?

Primarily, I'm known for my work at Driven Data. Driven Data is an organization that runs machine learning competitions that have a social impact, so we work with nonprofits, NGOs, government groups to help them figure out a way that machine learning can help them to be more effective. Then we put that online so that our global community of data scientists can come up with the best solution to this particular problem. And then after the competition, we help the organization to use that going forward. So that's probably one of the things is just that work at Driven Data that we've been doing for the last five years. But outside of that, there are two particular areas of data science that I often talk about and I'm very interested in, and the first one is engineering best practices for data science. So I'm one of the maintainers of the cookie cutter data science project, which is project template that I hope we get some time to talk about because it's one of my pet projects, and I think it makes a big difference in our own work, and I hope it makes a difference for other people. And then the other one is thinking about the social impact that data science can have and its relationship to the larger data ethics conversation that's happening. So we just released a package called Dion D E O N that's for building ethics checklists that you can use as part of your data science project. So if we get a chance, I'd love to talk about both of those as well, because they're things that I care deeply about.

I would love to jump into all of these things. So to recap, we have machine learning competitions with social impact engineering best practices, which I think is incredibly important, particularly because, you know, there is an argument that in data science, there is the idea of best practices in general is in a woeful state and something that's trying to be a lot of people are working on correction for and bringing engineering best principles into that will be essential. And then, of course, the data ethics aspect of your work, very recently, I mean, Mike Lucidis, Hillary Mason, DJ Patel have started writing their series on data ethics for O'Reilly, where they've actually got a post, as you know, on checklists versus odes versus codes of conduct. So I think all of these things are incredibly topical. So let's just spend a bit of time to go through each of these. These in terms of your machine learning competitions with social impact. Could you just give us a couple of examples?

Yeah, sure. So I'll start with one of my favorites. And that was the first competition we ever ran. So there's a nonprofit organization called Education Resource Strategies. And really what they want to do is they want to help school districts to spend their money more effectively. So schools are spending money on things like student transportation, teacher Salaries, textbooks, and they have a wide range of operational costs. And right now, it's very difficult for a school district to say, am I paying a lot more for textbooks than neighboring districts and getting the same outcomes? Are my investments being effective? And the biggest barrier to doing that kind of benchmarking or comparison is that school budgets come in wildly different formats, and there's no standard for reporting the categories of a budget expenditure. So that I can say we're spending more on textbooks than a neighboring school, and we need to look at this. So Educational Resource Strategies gathers all of this budget information from the districts they work with and ultimately what their output is, A recommendation for how to think about the school district's budgeting after they go through this process of benchmarking that district against other districts. So the big problem is that they spend hundreds of person hours a year looking at Excel spreadsheets, reading a budget line item, and trying to assign it a standard set of categories. So, as I'm sure your audience will have picked up on that description, they have a lot of labeled data that they've generated through the course of their operations. And that labeled data is a budget line item, a description of it, the cost of that budget item, and then what category it belongs to, whether that is transportation, textbooks, extracurricular activities, administrative salaries, all of those things they've captured over their history of working with school districts. So our first competition was, how do we help this organization that really cares about the output report and not about this taxonomy process? How do we help them to automate that? So we ran a competition where people built algorithms to parse the natural language in these budgets, to look at these budget costs, and to automatically assign categories for those school budgets to those line items. And we took the output of that competition, we turned it into a tool that fits into the workflow that this organization already had. So it's saving their analysts tons of time in terms of just reading through Excel spreadsheets so that they can focus that time where they can really add value, which is about making recommendations for how those budgets can be changed.

That's great. And it seems like it would reduce so much of the manual labor involved.

Yeah, really, It's. The last time we checked in, it was saving them about 300 person hours a year to automate that process. And for a relatively small nonprofit organization, that's actually a huge amount of labor savings. And really, their goal is to employ those savings more effectively where their employees actually add value, rather than in the labeling of spreadsheets, where it's Just this task that had to happen anyway.

Yeah, absolutely. And we should mention that if people find this type of stuff pretty exciting and interesting, they can check out a lot of the competitions on Driven Data. But if they find this particular competition interesting, they can even take the data camp course that you built and that I collaborated on with, which is learning from the experts and you get to build the winning solution in the end.

Yeah, that's right. So that course will walk through not only what a baseline solution to a problem like this is, but also how the person who won the competition combined a number of interesting techniques to get to that best performing solution.

And I'm not going to spoil the punchline, I don't want you to either. But I will say that it's not an LSTM or any crazy deep learning architecture that wins the competition. So now we've talked about the types of machine learning competitions you do at Driven Data. Maybe you can tell us a bit about your thoughts on our engineering best practices for data science and in particular your cookie cutter data science project that you maintain.

Great. Yeah. So my background is in software engineering. And so one of the things that I think about while I'm working on data science projects is how software and data science go together. I think there are some important differences between the processes in that data science tends to be more open ended, tends to be more of a research and development effort, but it's still the fact that a lot of what we do in data science is at its core building software. Even if that software exists in a jupyter notebook or in our markdown file, it's still a piece of software. That and a lot of the best practices that come out of software engineering can be employed to improve those products. And so the cookie cutter data science project is what we think of as the first pass at standardizing our process to make ourselves more effective. And that's to have a consistent layout of what a data science project looks like. So if you were to look at a web application in Django, which is a Python web framework, or in Ruby on Rails, or in Node js, all of these are different programming languages. But anytime you build a web application in one of those frameworks, you have more or less the same structure. What that means is that anyone who's a web developer can go into a project like that and have some expectation about where they would find certain kinds of code. The code that talks to the database usually lives in one place. The code that talks to the front end usually lives in another place. Those expectations make it very easy to work together and collaborate on projects. The cookie cutter data science project idea is to bring that kind of standardization to our own data science work. We have a defined folder structure where our data lives. And we have a set of folders for raw data data that has some processing but is in an interim state and then processed data. We have a particular folder structure for where we keep our jupyter notebooks or R markdown files that are built in the literate programming style. And we have another set of folder structures for data processing pipelines that may exist as scripts and then ultimately may get refactored into something like Python package. And because of this consistency, it's. It's very easy for us to move from project to project and pick up something and remember sort of where we are, how to reproduce something. And because we work with lots of different clients and lots of different projects, this means that anyone who works on the team can jump into any project without having to spend a lot of time figuring out what happens where.

That's great, and I find all the details very interesting. But as you've hammered home, the idea of actually having a. An overarching, overall consistent layout to data science projects and a system of best practices, I think is incredibly important. I presume this actually plays a role in your approach to building the package for data ethics and ethical checklists of having something that you can carry across projects. And if there are biases involved or challenges, they're systematic in the sense that they won't be induced by a human working on the project. They'll be in this structure as a whole. So you'll be able to, you know, that will become aware. You'll become a parent of that.

Yeah. So I think the two are related in that we really spend a lot of our time building tools for people who are data scientists. A lot of times they start out as tools that we're using ourselves, and then we open source those tools so that other people who are working in data science can use them. So that's how the cookie cutter data science project started. And that's really how this ethics checklist package Dion started, started as well. So the idea there was. There were a lot of conversations around data ethics that we found very compelling from a standpoint of really seeing where things had gone wrong in the process and feeling like ourselves. We were vulnerable to some of these things that could go wrong. It's very, very hard to have perfect foresight and to understand exactly what can go wrong in any given circumstance. And because we kept Seeing these examples and feeling like, would we necessarily have caught this in our own kind of work? We wanted to have a more actionable way of engaging with that data ethics conversation to make sure that we didn't fall into some of these traps that just exist in the work, that if you're focused on methods, if you're thinking about data, you can get into these really technical aspects and not have a chance to pull up and look at the ethical implications. And so we wanted to have a really process driven way of engaging that conversation for our own projects. So what this package we built does is it generates a data ethics checklist that's really framed around the data science process. So it starts with the data collection phase and, and there are a set of checklist items that ask you questions like, do the participants in your data collection give informed consent for the data that is collected? That's one example of an item on the checklist. And what we've done is we've taken each of these checklist items at different parts in the data science process and we've mapped them to real world examples where something has gone wrong. So we've got this collection of, of news articles and academic papers that explain when data science projects have hit ethical implications and the problems that have arisen. And there was actually just a really great article about Amazon really trying to build a resume filtering algorithm. And they get an unbelievable volume of resumes for any position that they open up. And, and they had the belief that they could use all of their historical data to train an algorithm to identify the top candidates that were applying for a given position. Now, as a sort of framing that may seem from a data science perspective, like it makes a lot of sense in that we've got a long set of training data and we want to be able to replicate this with an algorithm. But they just actually shut down the team that had been working on this project for years because they found that the algorithm was biased against women. And in particular it wasn't saying, oh, is this person a woman? And they've indicated that on their application. Now I want to use that and discount their application, but it was using things that were a little more implicit than that. So in particular, if the applicant had attended a women's college, then their score went down and they discovered these problems with their algorithm and disbanded the team that was working on this project entirely because they couldn't get it over this, this bump in the process. And so they're still having humans review all of these resumes that come in and so this is just sort of a classic example of something that seems like a great setup for a machine learning problem in the abstract. But if you don't think about how it's affecting your outputs that aren't just some measure of accuracy, it can go really, really wrong.

And these are the types of things that Dion would ask you to check for.

That's exactly right. Yep. And so it goes through the data science process, from data collection to data cleaning, to exploratory data analysis, to actually building models. And that's where this would come in. And then actually it's got a section on deploying models. So when the model is deployed, what are the questions that you should ask? And really, the goal is not to have all of the answers about what's right and what's wrong in a checklist. Given all of the different domains people work in, that's really an impossible task. But the goal is to take people who are already conscientious actors that want to be doing the right thing and, and make sure they're having the conversations about the way that things can be misused. So really, the workflow we see for the tool is you generate a checklist using the tool and then you submit a PR to your project that says, hey, here's our ethics checklist. Let's make sure we talk about each of these items as a team. And it's really about spurring that team discussion to make sure you've considered the particular implications of your project and you've made the right decision about it.

And so we'll include links to the cookie cutter project and the Dion package in the show. Notes. I think this, you know, you discussing data ethics in terms of thinking about the variety of stakeholders really dovetails nicely into our conversation about human centered design in data science and why it's important as a prelude to human centered design. Though, I'd just like to ask you a quick question about the role of empathy in data science as a whole. So I'm wondering, what is the role of empathy in data science for you?

That's a great question. So I actually feel like empathy is a term that has started to pop up in the data science conversation as a core skill of a data scientist. And in my mind, empathy is just one way to get at a particular kind of approach. And that approach is to be problem focused rather than method focused. And what I mean by that is we should start as data scientists that are really in a professional service role. We're providing a service to different parts of a business or different parts of an organization. We should start with what the problem is that we're solving and understand the context for that problem, rather than saying, hey, who's got an NLP problem that I can solve with an lstm? Or who's got a computer vision problem where I can try out the latest neural network methods and get a really cool result. Because if you start methods first, a lot of times you end up with a solution that's not going to be really useful in the context in which it operates. So when we talk about data science and empathy, what we're really saying is that you should empathize with how your data science output will be used. You should empathize with what's the problem we're solving. So when we talk about empathy, I think that's one way of getting to a perspective that's problem first rather than method first.

And any concrete examples spring to mind.

Yeah. So I think that for me, a good example is we worked on a project that is trying to automatically identify species in videos. Species of animals, that is. So there's a research organization that has these motion detecting cameras that they set up in the jungle and they try to record videos of chimpanzees, but they get a lot of videos of other animals as well. And instead of sitting there and watching all of these videos and saying, which one has a chimpanzee, which one doesn't, we're helping them to build an algorithm that to automatically identify the animals that appeared. And we actually ran a competition around this last year. If you're curious, you can look@drivendata.org and see the results of that competition.

And what was the name of the competition?

Competition was Primatrix Factorization.

Stop it.

Thanks for asking. I know, that's brilliant. We really care about our data science puns, and that's one of my absolute favorites.

We're talking about the data science puns. We have to mention naive Bs as well.

Yeah. And to be honest, Hugo, with your accent, I wasn't even sure if you were saying bees or bays. So it works even better in Australia.

And Naive Bees, of course, is currently being turned into a series of Data Camp projects as well.

That's right, yeah. So the Naive Bees competition was to build an algorithm to identify honeybees from bumblebees. And we're working on a set of data Camp projects to help people work through that problem, to give them that first exposure to a computer vision task that fits into a classification framework and look at the more traditional methods and Then move on to deep learning neural networks and convolutional neural networks.

Okay, so after that puntastic interlude, back to empathy, the role of empathy in identifying primates, back to the role of empathy.

And really this is about going back to the context and understanding the context in which you operate. So we were working with this team of ecologists and biologists. They spend a lot of time in the field setting up these cameras, capturing data, watching videos, and then writing papers about what they see in the videos. And the output that we ended up working on after the competition was an open source package that let you run from the command line predict here are my new videos. And it would output a CSV with each of the videos and a probability that a certain kind of animal appeared in the videos. And we were pretty pleased with this output. It'd be super useful for us. And the first thing we heard from the team we were working with is we can't even get this tool installed. I can't get XGBoost to install on my machine. I'm having trouble getting the version of TensorFlow installed. I'm having trouble getting GPU drivers installed. And so all of this stuff that feels like second nature to us as data scientists sort of blinded us to the context in which this tool was actually going to be used. And it's by ecologists that aren't used to all of this complex machinery around the packaging of data science tools that can make it really challenging to use the latest methods. And so that's just a really concrete example of a place where we weren't doing the right thing up front to really understand that context and make sure we built something useful. We were building something that we knew would be useful for us.

I got to make clear that I'm sure a non trivial proportion of like working expert data scientists have a lot of problems getting XGBoost installed occasionally as well.

Yeah, if someone wants to take on the initiative to improve the XGBoost installation experience, that's a really valuable project that someone could do for the open source community.

We'll jump right back into our interview with Peter Bull after a short segment. We're back with our favorite storyteller. Justin Boyce has been telling us about probability distributions and their stories. Hey, Justin.

Hey, Hugo.

Normally I'd ask you to tell us another story, but I have some questions for you, so I'm just going to ask.

Great. Ask away.

Last time you were talking about the exponential distribution. It's the first continuous distribution you've talked about and you gave the time between the rare events of no hitters in Major League baseball, which can be described as a Poisson process, as an example of an exponentially distributed random variable. As a reminder, a no hitter is when a team allows no hits by the opponent in a baseball game. But then you told us the time between no hitters in units of games played. If I remember correctly, you told us the longest time between no hitters was 6,000 some odd games. But isn't the number of games discrete? And the exponential distribution describes a continuous time.

Very good observation, Hugo. I could almost hear the air quotes when you said time between games. That's right. The time between games is continuous, but number of games is discrete. So strictly speaking, we should use a discrete distribution to describe the number of games between no hitters.

So we can't use the exponential distribution then?

No, we can. And as I explain why, I'm going to introduce another distribution with a new story. Here goes. The number of successive failures of a Bernoulli trial before a success is geometrically distributed. So you can think of a Bernoulli trial as a game. A failure is not getting a no hitter, and getting a no hitter is a success. So the number of games between no hitters is geometrically distributed.

So not exponential, but related to the exponential.

Right. Strictly speaking, not exponential, but the geometric distribution is the discrete analog of the exponential distribution. It has the same shape as the exponential. It's just defined at discrete values. So we can approximately use an exponential distribution to describe the number of games between no hitters. It's called a continuous approximation. And in this case it is a very, very good approximation.

Ah, so now we have connected another couple of distributions. The geometric, which is discrete and it's analog. The exponential, which is continuous.

Exactly. If you think about it, any digital measurement of time results in discrete data. For example, if we have timing data from a marathon in units of seconds, we strictly speaking have discrete data. But the error in treating the timings as continuous is not large.

So how about giving us another distribution connected to the exponential?

Sure.

Here's a new story. The amount of time I have to wait for the arrival of two Poisson processes is gamma distributed. Or more generally, the amount of time I have to wait for n Poisson processes to arrive is gamma distributed.

So the exponential is a special case of the gamma distribution where n equals 1.

Exactly. Or you could say that the gamma distribution generalizes the exponential and the gamma distribution gets more interesting. As N grows, it has a peak, unlike the exponential distribution which just decays away I should note also that N need not be integer.

And so where do we see gamma distributions?

Like all distributions, they pop up all over. Now, gamma distributions come up a lot in my field of biology. A common example is the time between spiking or firing of a neuron after it spikes. There's a refractory period and a few processes have to happen to have it fire again. Now, they're complicated objects, so not all neurons have interspike intervals that are gamma distributed, but. But many of them do.

Thanks, Justin. Your stories have really been making my neurons fire. And thanks for that intro into the geometric and gamma distributions and their stories. What distribution and story are you going to tell us about next time?

You know, I think we're overdue for the granddaddy next time we'll talk about the normal or Gaussian distribution.

I can't wait. Let's have story time again. Soon. After that interlude, it's time to jump back into our chat with Peter. But first I wanted to let you know that Driven Data is currently hiring. If you're enjoying this interview, you might be interested in working with Driven Data. Currently, the team is looking for a software engineer who loves the idea of building Python applications for social impact, and we'll put a link in the show. Notes we here at Datacamp have collaborated with Driven Data and Peter on several initiatives, and they've always been a joy to work with. So let's jump into human centered design and why it's important in data science, because I think probably a lot of our listeners wouldn't necessarily think of design principles as being something which would play a huge role in the data science process. So maybe you can tell us about human centered design and why it's important in data science.

Great. Yeah. So human centered design is a way of framing the design process and it's really related to other terms that are in this field that you may have heard of, like design thinking, the double diamond method, and design sprints. Those are all sort of popular terms that people may talk about. And it's really referring to the same set of ideas, which is about having a design process. And human centered design in particular is the one that we're most familiar with through our work with an organization called ideo.org so IDEO is one of the leading human centered design firms. They helped design the first Apple mouse. They have a long history of being designers, both from an industrial design perspective, but also from a digital design and then eventually service design perspective. And so they have a really long history and track record of working and using these design tools to spur innovation. And they spun out a nonprofit arm, IDEO.org that works with NGOs to have the same sort of results. So we partnered with that organization to look at digital financial services in Tanzania. And just to take a step back, that's sort of the context that we're working in, is with this team of human centered designers and what that human centered design process looks like. And I'll give you the overview first, and after that we can dig into the details of that particular project, which I think are pretty enlightening for how data science and design work together. So the big picture is that human centered design is about starting with what's desirable. So there's a perspective that the best solutions to a problem are desirable in that someone wants to use them. They're feasible from a technological perspective, and they're viable from a business perspective. And the best solutions sit at the intersection of that Venn diagram. You know, you're not having a good data science conversation until you're talking about a Venn diagram, right?

I was waiting for the Venn diagram.

So this particular Venn diagram is desirable, feasible, viable. We want the intersection of all three of those things.

And of course, a lot of data science work maybe will start with feasible, like the newest cutting edge technologies and the coolest, most efficient algorithms and that type of stuff, right?

That's exactly right. And I think that that is one of our tendencies as data scientists that I see in myself all the time, where I'm getting excited about lots of these new technologies and I want to find ways to use them. And so the trick is just to find the balance for really solving a problem we're using that is appropriate. And the human centered design process sort of starts from this perspective of what's desirable to a user. And it gets there by moving through these three phases of the design process. The first is inspiration, and inspiration is about going and observing the people who will be users of your end product. So in the case where you're a data scientist, let's say that your job is to create this report that's emailed out to your executive team once a month. So what you would actually do is you would go and talk to the people who get that email and you would say, hey, when you get this email, what do you use it for? What does it go into? Do you, do you say, okay, I need some top line numbers from here that I put into slides? Or is it something you read to get context, that then you say to the People that you manage, we need to change things X, Y and Z. You would go and you would actually talk to the consumers of your data science process to see how does it fit in to the bigger picture. And so the inspiration part of the phase is really about going broad, brainstorming and trying to get inspired by everything you might see around you. It's not about, let me see the data and get inspired by what's in my data, it's let's get inspired by everything before we even think about the data. So that's the first phase. The second phase is ideation. And what this means is trying to come up with particular solutions to a problem and then testing those really quickly. So one of the core concepts here is having the lowest fidelity prototypes possible and getting real user feedback on them. So it might be the case that you're working on a model to do some classification and ultimately it's going to be like this big complex machine learning system that's deployed in the cloud. But what you might do first is, let's say we're working on this honeybee bumblebee problem. You might just say, okay, here's a spreadsheet of probabilities for each of these species. What would you do with this? That's sort of My Lowest Fidelity 1. Take the most basic method, take the most basic output and say, is this useful? And then you take that and you learn from that. So the ideation phase is about these iterative cycles of learning from low fidelity prototypes that slowly and slowly build fidelity around them. But it sort of helps to keep your project going in a direction that ensures that the output is going to be targeted at a real problem, that it's actually going to be useful. And that as you come up with new ideas throughout that process, you can see which of those are good ideas and which ones aren't.

And it keeps you honest. Right. In the sense that you're not going to end up building something which is useless or going down the entirely wrong path.

That's exactly right, yeah. So, I mean, I seen so many times that even work that we've done where you build a dashboard that no one ever looks at, and everyone thought the dashboard was what they wanted, but that's not the right tool for the job. People really cared about answering one specific question and just if they went to a webpage with a thumbs up for yes and a thumbs down for a no, that would have been even better than the dashboard you created.

So in this dashboard case, instead of building the dashboard, perhaps the inspirational ideation phase would involve drawing the type of figure on a whiteboard or on a piece of paper that the dashboard would show and say to the stakeholders, hey, is this actually what you want?

Yeah, that's exactly right. Or even mocking it up in PowerPoint or using Microsoft Paint to make a little prototype and say, hey, is this graph something that you would need? How would you actually use this in your process? So trying to get at not just saying, hey, do you like this? Is this something you want to see? And more, how would you use it then? Because that question of how you actually use something will change people's answers. I think in a lot of data science work, it's very clear that if you ask, hey, do you want to see this? Hey, do you want to see that? Hey, do you want to see something else? People say yes to all of those questions. How could you not want to have all of the information that you could have? But really the question of how would you use it helps you to narrow down on the things that are going to be valuable and not create this information overload.

So tell us about the third phase. We've got inspiration, ideation, and the third eye is.

The third eye is implementation. So implementation here is not just, okay, you finished it, now go build it. Implementation is actually a continuation of this process. So to go from prototyping to actually piloting a solution. So we think of prototypes as being small scale, low fidelity, something we do with a couple people to get some feedback. We think of implementation as, okay, how do we become more data driven about this decision that we're making now? Implementation is about picking a pilot cohort, a set of users that will actually consume this, and then saying, okay, here's the version we're working with now. Here's a higher fidelity prototype that we have. Let's put it out there with a particular user group and let's do some real testing of if this solution is working and solving the problem that we want to solve. And so implementation is this piloting phase to get to the point where not only do you have a lot of great anecdotal and qualitative evidence that you've built up from these discussions, but. But you're also starting to get this quantitative evidence for how what you built is changing the metrics that you care about.

We'll jump right back into our interview with Peter Bull after a short segment. Let's now dive into a segment called Studies in Interpretability. I'm here with Patter Coyle, machine learning engineer and one of the core developers of the open source statistical modeling platform PyMC3. Great to have you on the show, Patter.

Thanks for having me, Hugo.

So Patter, you've got very strong opinions on interpretability and fairness in machine learning. That's one of the reasons I like you. Can you start off by telling me what interpretability and fairness are?

Interpretability is telling you why a model makes certain decisions. Often we don't just care what a model predicts, we care why it predicts something. Fairness is making sure the predictions are unbiased and not discriminating against protected groups, implicit or explicit. An interpretive model can tell you why it decided that a certain person is not worthy of a credit card. And for a human it becomes easier to judge if the decision was based on a learned demographic or racial bias.

Thanks, Patter. So why do you think this is becoming a significant concern for data scientists?

I think if you look at the trends such as general data protection regulation or GDPR and change in consumer attitudes, we as data people need to invest time in understanding interpretability and fairness. The key terminology in GDPR that concerns us is the right to explainability. And that means that in specific cases a black box model might not be sufficient. And by black box model I mean one that is not interpretable. For example, you might want to know why a model predicted cervical cancer in a patient and in particular explain that to a subject matter expert.

Now those are really big topics. Can you talk a bit more about fairness specifically?

Well, it's worth talking about what's happening in the research community. So specifically. In the past few years we've seen tutorials at NIPS on fairness and machine learning. These will be in the show Notes I think it's very important to point out this is a sad fact about humanity, that humans are prejudiced. You just need to look at how few minorities and women are represented at the C level of Forbes 500 companies, for example, to see that. In fact, Morris Hart, a researcher into fairness and machine learning, points out that in the financial space, automated underwriting increase approval rates for minority and low income applicants by 30% while improving the overall accuracy of default predictions. So is possible depending on the rules and the process implemented and what data is collected to make less prejudiced decisions with algorithms.

One of the concerns that we see in the media and society at large is the idea of trust in models. What does this mean for data scientists practically, especially in terms of fairness?

I think for us to have impact as data scientists, we need to have empathy, an attitude of move fast and break things doesn't work in the healthcare space and especially doesn't work in cases where you might be impacting human beings lives like models for recidivism. So we shouldn't dismiss these concerns. I think we should invest some time in learning interpretive models. In fact, my field of expertise is probabilistic programming and I've got a course of this online if listeners are interested, called Probabilistic Programming Primer. These models allow you to both interpret the results, you get uncertainty quantification of what you're modeling, and you can also incorporate domain knowledge fundamentally. Also, the generative way that models are built in probabilistic programming is less black box than machine learning methods such as random forests.

So what does this mean practically for data scientists?

Practically, I think it's very important to talk to humans and think of the regulatory aspects of what you're doing with your data. For example, when I worked with credit risk models, we weren't allowed by law to use certain features such as risk. I think that part of maturing is data scientists is accepting the legal or ethical framework that you act within. And also think if this model got leaked to the press, would I be embarrassed about it? I think that's a good sniff test patter.

Thanks for that introduction to studies in interpretability.

Anything I can do to help listeners. Thanks for having me.

Time to get straight back into our chat with Peter. So you and I have discussed a really interesting project you worked on previously, which I think illuminates all of these steps really wonderfully. It's a project where you're looking at digital financial services in Tanzania. So maybe you could tell us a bit about this project and how human centered design actually played a pivotal role for you.

Great. Yeah. So this project, I'm going to step back and sort of give you the, the context that this project is done in. So for a lot of people, your money has always been digital. And what I mean by that is that when I opened my first bank account, my parents brought me to a bank in middle school and said, you're going to have a bank account, you have to be responsible for your finances. Now I gave that bank some money and they wrote down on a piece of paper how much money I had. And that amount of money wasn't physical cash that I was holding, that was actually in a database that the bank had that was purely digital. And that's my native experience of what it's like to interact with money is to have what is really a digital representation of that currency. Whereas in lots of places, bank infrastructure is not very good. It's expensive to build banks. There are security risks of moving physical money around. And in many countries, it's the case that people do most of their interaction with money in pure cash. And that means when they want to save money, they have cash. When they want to spend money, they have to get enough cash to buy something. And that can be very limiting in terms of your ability to save money over time, your ability to get loans that you might need to buy particular things. And so there's a belief in the international development space that one of the ways in which we can help lift people out of poverty is to provide them access to digital financial services. So the same digital experience that people have had growing up, where they put their money in a bank, how can we provide that without having to build all of that physical banking infrastructure? And one of the approaches to this is to have a digital wallet on your phone that's associated with your mobile phone account. Mobile phones have been one of these leapfrogging technologies where people who didn't even have a landline now have access to a mobile phone. So the mobile phone providers are now starting to offer services where you can actually save cash on your phone. You can see, say, okay, I've got a wallet with $10 on it, and that's just associated with my phone. And so there's this transition from a world in which you just work with cash to a world in which you have some digital representation. And for people who haven't had the experience of always having a digital representation, you have to build trust in that system. You have to make sure that the digital financial services actually fit the needs that this community has. So that's the big sort of project background and context is how do we increase uptake of these digital financial services that are providing for people in these environments that have very volatile incomes, some sense of stability for their money, some access to loan infrastructure, some access to savings infrastructure. So that's the context. This is a project that's funded by the Gates foundation to try to understand what levers we can pull to help engage people and give them access to these tools. So one of the particular things about this digital financial services system is that you need a way to exchange your digital currency for cash. There's always going to be some sort of transaction that's like working with an ATM. However, ATMs are relatively expensive and there's a lot of physical infrastructure you need for an ATM to work. So what happens in places where they have digital financial Services. And this also goes by the name mobile money. So I'll use those interchangeably. You have what are called mobile money agents. These are usually people that have a small shop somewhere that are selling home goods, snacks, drinks, and they also become mobile money agents. What that means is you can go to them and say, I want to trade five digital dollars for $5 in cash, or I want to take $5 in cash and put that in my digital wallet. So they're the interface between physical cash and digital cash. So the focus of our project was how do we make that interaction between agents and customers, one that can help build trust in this mobile money system? And that direction was driven by this human centered design process. So when we went out, the first thing we did was talk to people who used mobile money and people who didn't. We went and sat in a number of markets, we watched people buying things, and we asked them, why did you buy that with cash? Why did you buy that with mobile money? Do you have a mobile money account? What's your experience been like?

And it sounds like in this process as well, then you're getting qualitative data as well as quantitative data.

That's right. And really a big part of what you do is try to gather that qualitative data as well. So we've been gathering this qualitative data through these interviews, but we also got mobile money transactions from one of the mobile network operators for a full nine months. And it was tens of gigabytes of data, all of the transactions, hundreds of millions of mobile money transactions in the country. Our goal was to combine that very rich data source about real behaviors with behaviors that we heard about with these qualitative experiences that people actually had. And so one really great example that I think just highlights the value of human centered design is the fact that we kept talking to agents and saying, what are your biggest struggles with mobile money? How do you see it fitting into the larger context of your business, where mobile money is one of your revenue streams, but you also sell other goods? And what we kept hearing from these mobile money agents is, well, it can be really tricky to predict how much money I'm going to make on mobile money transactions, because I earn some commission on each transaction. But it's really opaque what those commissions are. And it's very hard for me to predict on any aggregated time frame for some given week or some given month, how much money I'm going to get in commissions. And this was particularly interesting to us because we as data scientists have been digging into this Huge treasure trove of data thinking. Oh, this has all the answers in it. This is going to be amazing. We can find so many insights in this huge range of transactions that we have. One of the things that we realized after doing these interviews is we didn't know what the commissions for an agent were. That was not data. That was in the data set that we had. We had the amount of the transaction, but the portion that then an agent got as a commission was calculated totally separately by some business logic that existed in another application.

And even if you did know how much they got, you may not know whether that was a lot or little or how that affected them on the ground.

Oh, absolutely, yeah. And so just the thought that we could have known what was valuable to agents by looking at the data set and figuring out these patterns. That data set didn't even have the most important variable to the agents that we were working with. And we wouldn't have learned that if we didn't talk to them and just tried to learn things from the data alone. So one of the big things that we try to do in all of our work and I really encourage all data scientists to do, is to go out and observe how your data gets collected. So if you're working with IoT data in a factory, actually go to the factory floor, watch how things happen. If you're like us working with digital financial services, go watch people make transactions, see what actions in the real world correspond with something in your data. Because that perspective changes how you think about the data itself. It changes where you trust the data and where you don't trust the data. And so I really encourage people to get away from their screen, step away from their desk and go watch the data collection in action. And I think in nearly every case you can go do this and it will have a transformative effect on how you think about that data.

I think that's really important, particularly as we live in an age when people may be first get associated have experience with data science, online competitions, for example, such as yours, platforms like datacamp, getting tech data or getting data online and not actually thinking a lot about the data generating process.

Yeah, it's very easy to just start with the data and say, okay, what's in here? But until you really understand that data generating process, you won't know to ask what's not in this data that I might care about or what in this data is not reliable. For example, we saw a lot of these mobile money transactions fail because of network connectivity, for example. And for some of those transactions we wouldn't have seen that in the data. If the network failed, the transaction ever went through, it doesn't get recorded in the database. So understanding the limitations of the connectivity and how that affects the experience is something that we can only even start asking about. How do we measure those transactions when we've actually observed them?

So we're going to need to wrap up in a few minutes. Peter, I was just wondering, are there any other aspects of the project in Tanzania that you wanted to discuss?

Yeah. So I just to share one other example of where the human centered design approach I think really made a difference. We were looking at the times of day which a mobile money agent was busiest. So when were people coming to trade cash for digital currency or do the opposite? And we took the data and we looked at it for days of the week and times of day. And we built this really beautiful heat map visualization that you can think of as a checkerboard where each of the squares is either lighter or darker based on how many transactions you have. We made it interactive so that you can hover and you can see how busy for a different region is a different time of day, is a different day of week and really get a great sense for the patterns and mobile money use that happen.

And it's also colorblind, human friendly.

It absolutely is. We did build it using viridis, which is colorblind friendly. And if you're not thinking about that for your visualizations, you should be because that's a little bit of human centered design.

And speaking as a colorblind individual, I'm red, green, colorblind, as you know, 8% of human males are.

So you appreciate viridis even more than the rest of us that think it's a beautiful color map.

I can't get enough of it.

Well, I think it's a really compelling and beautiful color map, which is one of the reasons that we loved this visualization, which is one of the reasons the people we worked with loved this visualization and how interactive it was. But none of the agents that we were working with had access to a computer. They weren't sitting at a laptop and they wanted to look at a dashboard that had this beautiful visualization meant that wasn't going to be useful to them. So what we ended up actually building was a text based visualization that was essentially just a bar chart where it would say M for Monday and then it would have three capital I's in a row and then it would have T for Tuesday and it would have eight capital I'S. In a row. And by building these text based visualizations, essentially bar charts built out of characters, we could actually give a data visualization experience to these agents that were working on feature phones. And that process of taking something that we think of as an amazing data science output, this really compelling interactive visualization and putting that visualization into a context where it can actually get used is one of the transformative experiences of that project for us where we started to think about, okay, what's the context for all of our output? Not how do we make the most amazing data visualization.

Yeah. And it's so reliant on the knowledge of what actual technology, what phones humans have on the ground, Right?

Yep, that's absolutely right.

So Peter, as a final question, I'm wondering if you have a final call to action to all our listeners out there.

Yeah. So I think there are in my mind four core activities of human centered data scientists. And I think we should all be human centered data scientists. And the first one is go to the field and observe the data being generated without understanding what a row in your data set means, without actually observing that happening, without knowing what gets captured, what doesn't, what happens when something goes wrong, you'll be very limited in the output that you can have. And also if you do go and do that, you'll be so much better positioned to ask questions that matter of your data. Without talking to those agents, we wouldn't have asked that commission question of the data set. So going to the field observing data being generated is item number one. Item two is design with not for by iterating on prototypes. So this process of constant iteration, conversation with people who will actually be using the output, getting their buy in on the decisions that you're making, means that it's going to be something that's useful when you finish the project. Not I worked for three months. Is this good for you? Oh no, it's not. Or it requires some major changes. It's how do we keep that process tighter in sync so that we're actually building things that are useful. And we do that with really low fidelity prototypes that we're constantly testing. The third is to put outcomes, not methods or outputs first. So that's really saying what is the outcome we care about. In our case it was the increase in adoption of digital financial services. That's what we cared about. And in particular we thought we could do that by improving the tools that mobile money agents had. So our goal was to say, okay, the best outcome is for mobile money agents to be making more transactions. That's what we want to measure and it wasn't how do we do the most interesting dimensionality reduction on this huge data set that we have? The fourth item is to build consensus on metrics for success. And I think this is one of the most difficult but most important ones is you need to define up front what success means and you need to get buy in from everyone on that being successful. And I think this is one of the things that people assume they've got the same goals if they're working on the same project from the get go. But until you have that explicit discussion about what success means and what those metrics are, you won't be optimizing for exactly the same thing. So those I think my call to action really is to take those and try to build them into your process as a data scientist. Other than becoming a human centered data scientist, thinking about your users and using a more collaborative process, I would encourage people to come check out a competition on drivendata.org got a lot of interesting social impact projects happening there. Or to check out one of the open source projects that we had as part of this discussion. That's Cookie Cutter Data Science and Dion, the Ethics checklist package.

And of course both of those projects and engaging in competitions and all the other great stuff you do on Driven Data will help any budding data scientist or established data scientist to doing more human centered data work as well.

Yeah, that's the goal.

Peter, it's been such a pleasure having you on the show.

Hugo, thanks for having me. I love chatting as always.

Thanks for joining our conversation with Peter about the importance of human centered design in data science. We saw the need for applying software engineering best practices in data work and the importance of ethics checklists along with Driven Data's Cookie Cutter Data Science and Dion projects which respectively step up to these issues. We also saw how the process of human centered design is a manifestation of the principle of empathy for users in terms of iterating on data products, for example, that they both want and find useful. Peter provided several illuminating examples including Driven Data's work with Mobile Money financial services in Tanzania to make the interaction between agents and customers one that can help build trust in the mobile money system. Starting with what is desirable is key as opposed to what's feasible and then rapidly iterating through inspiration, ideation and implementation with the end user along the way. And don't forget, if you like this interview, you might be interested in working with Driven Data. Currently the team is looking for a software engineer who loves the idea of building Python applications for social impact. We'll include a link in the show notes. Also get pumped the heck up for next week's episode. A conversation with Angela Basa about managing Data Science teams Angela is Director of Data Science at iRobot where she leads the team through development of machine learning algorithms, sentiment analysis and anomaly detection processes. IROBOT are the makers of consumer robots that we all know and love, like the Roomba and the Braava, which are respectively, a robotic vacuum cleaner and a robotic mop. Angela will talk about how to get into data science management, the most important strategies to ensure that your data science team delivers value to the organization, how to hire data scientists and key points to consider as your data science team grows over time, in addition to the types of trade offs you need to make as a data science manager and how you make the right ones. All this and much, much more. I'm your host Hugo Bown Anderson. You can follow me on Twitter Hugo Bown and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com community podcast.