#15 Building Data Science Teams

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Drew Conway, world renowned data scientist, entrepreneur, author, speaker and creator of the Data Science Venn Diagram. Drew and I will be talking about how to build data science teams along with the unique challenges of building data science products for industrial users. How does Drew now view the Venn circles he created? Those of hacking skills, mathematical and statistical knowledge, and substantive expertise when building out data science teams. Stick around for this and much more to set the scene. The first half of the show will focus on what data science looks like today through the lens of the evolution of the Data Science Venn Diagram and the unique place data science holds in an industrial setting. The second half will use all of this knowledge to focus on data science team building and recruiting. Hugo I'm Hugo Baun Anderson, a Data Scientist at DataCamp and this is Dataframe. Welcome to DataFrame, a weekly datacamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter hugobown and datacamp@datacamp. You can find all our episodes and show notes@datacamp.com Community Podcast hi Drew, and welcome to Data Framed.

Hey Hugo, it's great to be here.

It'S great to have you here, and I'm really looking forward to chatting with you about how to build a data science team, along with the unique challenges of building data science products for industrial users. But first, I'd like to find out a bit about you. What are you known for in the data science community?

Right, so a long time ago, I was probably known for being one of the earliest bloggers in data science. So as the story goes, when I was first admitted to graduate school at nyu, I was really excited about finally having the opportunity to speak publicly about the interest that I had and some of the work that I was doing. So I started this little tiny blog called Zero Intelligence Agents and really just used it kind of as a public notebook and things that I was working on, you know, code that I was writing, things that were interesting to me, and then eventually kind of combined that with very early social media days on Twitter and found that there was this small community of folks who were, you know, writing and tweeting about the interesting data stuff that they were doing, even predating a kind of generally agreed upon term of data science. And then of course, the thing that I'm actually most known for in the data science community is the data science Venn diagram, of course.

And then you were, you're heavily, and you've been heavily involved in the NYC data science community as well, right? Yeah.

And so the, you know, probably the thing that, that I'm most proud of in terms of my contribution back to the data science community has been my ability to, or at least my concentration from being in graduate school at NYU and then, and then building companies and building teams in New York is, is really kind of planting a flag for New York City as a great place to scientist and a great place to do data science. Yeah.

And you actually have a strong argument that New York City was a place where data science was being done even before data science was a discipline, right?

Yeah, that's exactly right. I mean, you know, we think about what the anchor industries have always been in New York and you know, you can take tech out of that and just think about the financial services and banking industries, the media industries and the advertising industries. And all three of those are really data centric. And so I think what, what we see now in New York City is, is that it's, it's sort of always been this place where data and the robust analysis of data has been central to business and to profit making. And now with technology and its dissemination and movement into other verticals, those industries themselves, you know, financial services and media and ad tech have become a big part of what makes New York City unique. And, and then beyond that now, I think with the university system and the, the amount of startups that are in the space, New York has become a really great place to do this kind of work. I think there's a bunch of other things that are really unique about New York that, that tend to enhance that, one of which is just the geography of the city. Right. And you know, it's easy to always make comparisons to the west coast, but you know, New York City is a, is a tiny little island and we're all, we're all crammed on it. And if I want to go have a conversation with you, I can just jump on the subway, go downtown, and there you are. And if I want to go to a meetup, I can go there, I can go speak to a professor at a university, and everything is within, you know, five or six square miles of where I'm sitting right now. And I think that really has changed how people in, in this city can, can do work.

That's right. And there's such a strong sense of community around data science here.

Agreed. And I think again, part is just kind of the culture of New York. I mean, people, for better or worse, we're loud, we're brash, we like to talk about what we're doing. And that means that a lot of ideas get shared. It makes all the difference in the world.

And you've spoken to a number of topics there that we'll get back to your early days as a blogger, really contributing to the evolution of data science and defining it initially as a career, the role of community data science in New York City as well, and also the famous Venn diagr. So I thought maybe you could tell us a bit about the Venn diagram. I'm sure you don't want to talk about, I'm sure you've been talking about it for years, so I'm sure you don't want to talk about it too much. But where it came from and where you see its evolution has gone since then.

Yeah, sure. So the origin story of the Venn diagram is actually intimately tied to the data science community in New York City. You know, about, I guess a little less than 10 years ago when I, when I first arrived in New York, I kind of inserted my. To what was then this again, kind of nascent community of folks in academia and industry and startups who were doing this work. And so eventually they're formed a kind of, I don't know, almost working group of folks who every month we would meet for a potluck brunch at the top of the New York Times building in the R and D floor, which is all the way at the top of the New York Times building, and just sit around on a Sunday morning and kind of talk about what, what this data science thing was. You know, we would have topics around, you know, how would you think about teaching it? And you know, names that are, that I think are now basically associated with data science are almost venerable names were there just thinking about it. So, you know, folks like Mark Hanson and Chris Wiggins from, from the academic side and Hillary Mason And Mike Lakidis on the, on the commercial side, you know, I was there.

And when was this.

This was back in, you know, 2009, 2010, having these morning conversations. And sometimes they're mostly just for fun. I mean, we were all friends, we knew each other from various walks of life, and we would just come together and chat. And so one day we were having this conversation around, what is data science? How would we think about defining it? What are the requirements to be a good one? And we had this wonderful conversation. Chris Wiggins and Hillary Mason were leading the chat. And I walked away from that, that discussion with a whole bunch of ideas in my head as to, you know, well, okay, this is, this is what these guys think, and here's how I might interpret this. And so the following week, this is when I was still in graduate school. So on one particular class, I sat all the way in the back of the lecture hall and just opened up my laptop and started kind of thinking about how I would define data science based on the ideas that had been discussed at this potluck breakfast, which ultimately led me to, you know, firing up my, you know, open source gimp Illustrator and creating what is now the data science Venn diagram. And I went and then ultimately wrote a blog post about it that went as viral as a data science post could go viral in, you know, circa 2010.

And we'll definitely link to that blog post in, in the show notes. And of course, in the middle of that Venn diagram, you have, have data science, but maybe you can tell us the, the things that revolve around it that, that are necessary skills. Yeah.

So, you know, the central part of the debate I think, that we were having back in 2010, and honestly, it seems like many folks are still having, although it's shattered into many more dimensions now, is what are the constituent pieces that some person should have if they want to actually be a data scientist? And so I broke this into three big groups. One is you have to be competent in using and developing software, what I refer to as hacking skills. And what I really meant by that is this is not someone who's a professional software engineer. Hacking skills mean someone who is, you know, able to fire up the command line, can manipulate text, knows how to work with a scripting language so that they can produce repeatable, maybe shareable and reproducible pieces of code that could be used to analyze data. You know, again, there wasn't a sense of professional application. It was just, do, you know, some stuff, can you actually code enough to be able to build kind of an MVP of something. The other piece of it was kind of the academic side. So if you're going to be building these things, you should have some real kind of grounding in the statistics and the mathematics that go into the models and the methods that you're using. If you don't have that, then you may simply be kind of pointing a very powerful technical weapon at data and not actually know what's going on. And then the third piece, piece, which I think ultimately becomes kind of the glue that brings it all together, is what I call the substantive expertise or really kind of subject matter expertise. And this has nothing to do with your skills as a coder or your competency as a statistician, but more, do you know how to ask good questions? Right. Because at the end of the day, and again, thinking about this back in 2010, ultimately what I was observing out in the, in the kind of intellectual marketplace, so to speak, is that there tended to be a lot of people, or most people were good at coding. And a lot of people had or could get training in statistics and math, but they didn't really know how to ask good questions. And if you don't have a kind of point of view on a problem or point of view on a data set, then you're kind of starting with nothing. Because no matter how much data analysis you do, if you're asking the wrong questions, you're kind of just, you know, treading water. And so we combined all three of those to create data science. And of course there's the, the kind of secondary overlaps that occur between all of them. And I think the one that for a lot of people was, was most satisfying was, you know, people were trying to make this distinction between, well, is data science, machine learning, and what's the difference? And so what seemed obvious to me is if, you know, if you have hacking skills and you know about statistics and math, you put those together, that's really what machine learning is and certainly was those many years ago. And so, you know, we kind of built that. And I wanted to balance that with what I viewed again at the time as a, as a PhD student, as what kind of traditional research is. Right. So if you have this methodological grounding in statistics and math, but you also have substantive expertise, as always, in a political science department. So people who have, you know, who are working on American politics questions or looking at international relation and conflict questions, they know a lot about those subjects. And oftentimes they can apply specific mathematical models to try to estimate what some of what they're seeing in data, but that's not data science. That's traditional research. And so I kind of had this overlap of traditional research. And then maybe the other overlap that was, that was, I don't know if it was controversial, but people had a lot to say about it was this idea that if you had substantive expertise, that is, you knew, you knew a subject well enough to ask good questions, but then also were able to kind of hack and write code and get some answer, if you didn't know the statistical and mathematical grounding in what those answers meant, then you're in the danger zone. I often refer to this as kind of, you know, you knew enough to be dangerous, and that's kind of the worst place to be because then you could create very misleading results. And I was trying to use that as a guard against what, what I would hope would be folks not, you know, putting data science on a path to, you know, snake oil.

Yeah, absolutely. And I actually, the danger zone is, is very interesting to me. I love that you put an exclamation point at the end of it to draw even, even more attention to it. But I think the fact that the idea that people can, especially with, you know, the, the raging success of all these new fantastic APIs that allow people to fit and predict a variety of models after importing data without necessarily knowing a lot they're. They're using it is actually incredibly dangerous.

Yeah. And I think, you know, there's, there's, there's a whole bunch of dimensions to this. Right. I think, you know, take, let's take it first from, like, even from a commercial perspective, the, the thing that I found most interesting about how data science tools and even platforms and applications have grown over the course of the last, you know, five to 10 years. For a while there, there was this real attraction to, say, building data science in a box tools as like, you know, take your data set and stick it into this tool and it would, it would predict for you all the possible outcomes for something. And like, voila, there you have a, you have a, you have a model. You can put that model in production, and it's great. And so these tools were targeted people that had substantive expertise. Right. So it's like, I can build this tool and I can go sell it to someone in an insurance company, and that insurance company will have better actuarial tables. It doesn't matter that the, you know, the folks in the insurance company might not exactly know what my random forest is doing and why it's making those distinctions. It just matters that they're getting better results. And I remember being around and hearing about a lot of those companies and quite honestly at the time thinking it seemed like a reasonable idea, but having this kind of sinking feeling that they were building tools that were kind of in this danger zone realm. And what my observation has been since then is that the reality is that a lot of those tools don't really fit into a real use case. Right. They're sort of in this valley between two real use cases, one being, okay, you have no hacking skills or methodological skills and your primary tool is like Excel and you're good at making charts, but you don't really know how to build things. And so you have a specific kind of service oriented need. And then there's the other folks who actually are really good at the substantive stuff and the mathematical stuff. And so they need really granular tools to do their work. And so they're the ones that are going to ultimately probably learn R or learn Python and become data scientists, but there's no, nobody really in between. And so ultimately I almost saw tools being built that fell into that danger zone and ultimately didn't have a whole lot of success.

Yeah. And I think that also speaks to the fact that pursuing, let's say, accuracy or model performance at the expense of other, other qualities is also inherently dangerous. Right. And now we're seeing kind of a rise in a desire and need in society for machine learning, interpretability, which brings us back to more substantive expertise.

I think that's, that's a really good point. You know, there's these things tend to ebb and flow and so there's, there's kind of this natural, I think, early attraction to black box tools because in a lot of this I think follows almost from a lot and sometimes almost the negative downside of the Venn diagram or, or folks kind of viewing these holistic definitions of things as being these kind of almost unicorn like individuals. And so what that does is it casts a shadow over the discipline that says, okay, well only a very specific kind of person can do this. Somebody that has all. And if you don't have that, well then you need a very specific tool to do it. And of course, you know, the history of data science and the history of many technical craft is really, you know, is not about finding one person who does everything right. It's always about finding a group of people who, who know about, who know a lot about some parts of it and can work together. And so I think there's Always this natural early inclination to say, okay, I can build, I can build this tool that does this thing really well and I'll go try to find somebody to use it. And again, I think the, the results in the market have not been, have not been great for those approaches, for sure.

And you've actually preempted my next question in some sense. I was going to play devil's advocate and ask you whether you thought the entire Venn diagram was a danger zone in itself in terms of its potential for being misinterpreted and the search for the unicorn.

Yeah, I think, you know, certainly that that was, I guess it is still true. If, if there's one thing that I wish I could have been clearer on when I introduced it is that it wasn't, you know, it's called the data science Venn diagram. It's not called the data scientist Venn Diagram. And I think a lot of people, you know, when they looked at it, they said, oh, this is, these are all the skills I need to hire for if I'm going to hire a data scientist. And really the idea of the Venn diagram is that this is what the discipline is. Right. And so if we think about other disciplines like software engineering, we don't think that there's a canonical software engineer that does everything right. And again, I think it's the same for data science. And unfortunately, in those days, you know, and to some extent still today, I think people view these kind of holistic definitions. And as, you know, the Venn diagram, I think, has become useful shorthand because it's, you know, it's an image, it's easy to share, but there's still a lot of pixels and ink that gets spilled trying to holistically define what the, what this career path is. And ultimately, I think that's, that's, that's in some sense a waste of time because we sort of know, we know how this movie is going to end. We've seen it many, many other times. And so we should be thinking about it in the context of teams and how people work together.

Yeah, absolutely. And that's something we'll get to the, the future of, of data science. And I will, you know, make it clear that your, your Venn diagram doesn't say data scientist in the middle. It specifically says data science. And in not necessarily just a unicorn. I think you had a great slide at Jared Lander's conference nyr, which had a unicorn with a cat with a laser gun on it or something. Is that right?

Yeah, yeah. And I wish I could credit whomever the artist was that created that, because I think it's wonderful. Yeah, I mean, you know, if I recall from that talk, you know, what I was thinking back on then was. Was actually part of these early days and, you know, kind of of right at the turn of the decade, in kind of 2010, 2011, where I was in New York City, I was having lots of conversations with people at various companies, from early stage startups to Fortune 500 companies. And when they heard that I was a data scientist or that I could help them find data scientists, the punchline of that joke is that is who they thought they were meeting with this cat riding a unicorn with a handgun and a flamethrower or something. And of course, we know that that's not tr. And the further we get to a professionalized discipline of data scientists, the further that becomes true.

I couldn't agree more. And something that I'm looking forward to chatting about later in this episode is about how you'd go about building a data science team from this Venn diagram.

Yeah, me too. I'm looking forward to it. Yeah.

But before we get there, I'd like to know a bit about what you do these days. What do you spend most of your time doing?

I do almost no data science, in fact. So I'm the founder and CE of a company called Illuvium. What we do is we build data analytics products for men and women working in complex industrial settings. So what I do most of the time is I think really hard about what their problems are and in particular, how we can build tools that help them better leverage data to make decisions. What that means is I spend a whole lot of time listening to our customers and asking them those kinds of questions. Of course, I also spend a whole lot of time listening to my team, you know, my teammates, and answering their questions and learning about what kind of techniques, you know, particularly on the data science side, they think would be most applicable to solving these problems. And then I also have an opportunity to speak to folks who might want to think about working for us. And so I still do a fair amount of recruiting and thinking about, you know, how to best explain what we do to folks and how to get them excited about working with us.

Fantastic. And are you currently hiring?

Oh, yeah, yeah, yeah, yeah. That was gonna. That. That'll be my call to action at the end of the episode.

Okay, fantastic. And so it actually sounds like in. In some respects, you're acting as an interface between the substantive expertise of the industries that you're working with and the hacking, hacking skills and, and mathematical and statistical knowledge.

Actually that's a great way to think about it. You know, when, when I founded Alluvium, there was, there was no, no question in my mind how little I knew about the day to day lives of someone, you know, working in a, in a, in a power plant or working in an oil refinery. But in is in fact, one of the core values at Alluvium is, is about learning and learning firsthand. We call seek the firsthand. We always want everyone in our company to think about how they can go out and through first hand knowledge, learn about something new and in particular learn about how our customers do their work work and, and so that substantive expertise in, in how industrial operations work, how they, you know, what kind of data gets generated, how that data, how that data generation process gets instrumented, who the actual people who are, you know, standing on the front lines making decisions with that data. Who are the people standing in the control room who are observing that data and you know, who are the folks back in the headquarters building making business decisions from that data? We want to seek and learn about all of that work so that we can go about building products that actually support them in their day to day.

Yeah. And if I recall correctly, your webpage says that a lot of data scientists will put on hard hats and go out there in the field.

Oh yeah, and not just the data scientists. The whole team gets out there. We have yet to get Alluvium branded hard hats, so we're often relying on our hosts to provide us with them. But it's probably one of the most exciting parts of the job. A year ago, we went to the large recycling center out in Brooklyn here in New York. Was fascinating to see how the city of New York handles its waste and how they try to improve the efficient use and recycling of that. And then this past winter we went to a robotics consortium in the Navy Yard and learned about how they're using robotics for art and for industry and for startups as well. And I think learning has become such a core part of how we do our business that I'm always excited to get a chance to go out and see how people do their work.

That sounds like an incredible opportunity. And when you said you haven't got Alluvium branded hard hats yet, I just wondered whether you've tried to put Alluvium branded laptop stickers on them at any point.

It's a good idea. The nice thing is that I guess we could just Maybe you're preempting me. Or we could just buy the hard hats and then just put the stickers on. That's the easiest thing to do.

Yeah, exactly. So I'd love to talk a bit more about Alluvium and I'd like to kind of motivate my question by quoting you or paraphrasing you. You've said that much of data science is a stack of tools developed to deal with big data and design for the web. But what can data science do for non digital industries? So that's really to frame my question, which is what are the major challenges that you're trying to solve with your work at Alluvian?

Yeah. So, you know, to kind of go back a little bit to that context. You know, when I founded the company, I was, I coming off of having worked at a, at another startup here in the city, which, which was, which was a consumer health company. And, and we were building a product where we were trying to kind of use real time streaming physiological and telemetry data to help people understand kind of their, their overall health. And, and part of what really attracted me to that opportunity is, is actually working with streaming data from the real world. In the early, early part of my career, I'd worked in the, in the, and I dealt with a lot of data from sensors, whether it was telecommunication sensors or measurement and signals out in the field, and then using that and combining it with highly unstructured data like text reports or images of maps and things like that. And one of the things that really stuck for me from that early experience is that even in those days, in those days, I mean, kind of mid-2000s, when a lot of what we think of now as this kind of commoditized stack of big data tools really didn't work well for dealing with that, that we, we just had to develop a lot of ad hoc methods for dealing with that data. And so fast forward to where I was working prior to Alluvium. I returned back almost a decade later to find that it's mostly still the same that we, and by that I mean we as a kind of technology and data community had seen that there was a ton of value in weblog server files and search results and clickstream data and all of these things that were, were produced by and used within kind of digital platforms. But we hadn't thought a lot about how do we, how do we do the same sort of stuff with data that's generated outside the web. Right. It's sort of like these, these physical Systems are so complex and there's so many things that are hard to observe, and we also have poor ways of measuring them. And we also don't have good software tools for dealing with them. Right. Sort of like the example that I like to say is it's still really hard to predict the weather and, you know, more than a day out. And part of the reason for that is, you know, the Earth is an extraordinarily complex system and we don't really have good ways of measuring it. We certainly don't have good ways of measuring it and doing analysis on it. So when, when I had the opportunity to start thinking about my own company and the kinds of problems that I would want to solve, the thing that I realized is that this technical problem was still highly present, that there was just not a good way of doing kind of distributed, real time, unsupervised learning from data from these kind of physical sensors in any unified way. Right. If you had multiple assets across multiple physical locations and you wanted to have a kind of uniform view of how all those things were operating, and you want to do that learning without any training data, how would you think about doing that? And so that was kind of the technical spark for Alluvium. And then ultimately the, you know, the founding spark, kind of the commercial spark for it was was realizing where that problem was most acute. And so kind of again, reaching back to this kind of seek the firsthand idea. I just got out and started talking to folks and quickly it became clear to me that the industrial space, which has for hundreds of years been in a highly data driven set of industries, whether it's the oil and gas industry, the manufacturing industry, both for discrete and process manufacturing, all of which were really good at collecting data, but none of which had really matured in what they would refer to as kind of their digital transformation. Right. These are processes and systems that are still in many cases highly analog. And even when heavy investments have been made in generating data, there's just not a lot of good tools for doing anything with them. So those two ideas kind of slammed together and we got to work.

Could you give us an example or a case study of some of the work you've done?

Sure. So probably the best example that I can think of that we're sort of, you know, able to talk about right now was actually an early, early pilot that we did with the New Orleans Police Department. So this actually ultimately ended up not being a path that we decided to take commercially, but in the early days we were really interested in what Our technology could do, say, in inside a vehicle. You know, modern car is basically a motorized computer, and so it generates a tremendous amount of data, but you want to be able to have both a kind of local view of how that vehicle is operating and then kind of a global view. And the way that we talk about that at Alluvium is through this idea of stability. And stability kind of forms the central, not only language that we use around our products, but in some sense, kind of of the core value proposition of the business. We want to provide our customers with a view of the overall stability of their operation. And then when those things change, we want to be able to quickly alert and guide an operator to where in a system that instability may be coming from, so that they can very quickly make an evaluation of that and ultimately take an action if they need to. And so in the case of the police department, we built a prototype and a pilot for them based on vehicle operation. And the idea was we wanted to be able to show how police vehicles were operating in the city and actually putting software inside the vehicles, on the vehicle laptops to stream data from the OBD sensors, the onboard diagnostic sensor, which has a huge amount of information that you can draw from it to give a kind of global view of the stability of, of the vehicles out in the field. And so we built that. Ultimately we decided that there was much bigger opportunity in plants and factories, and that's where we focused our attention. But we were able to build this prototype and this pilot for them where we were able to see changes in vehicle operation, how that changes stability, and ultimately produce some real interesting insights.

Great. So what essentially we're also talking about is not data science standing alone by itself, but actually building data products as well.

Yeah, and that's kind of the whole gig. Right. I think we have a particular point of view at Alluvium that data science, machine learning, AI, whatever you want to call it, it only takes you so far. Right. Ultimately, if you're building a product that is there to support someone making a decision, then you need to think about where is the point in which their knowledge, their context, their expertise needs to take over and ultimately make some decision adjudication based on what you're presenting them. I often when I'm introducing what we do to say folks in the industrial space or potential customers, we talk about this tension between data discovery and then data reasoning or reasoning about data. And so put yourself in the role of an industrial engineer who's standing inside an oil refinery. They are basically beset on all sides by this kind of wave of information. I think I forget what the exact statistic is, but I think, you know, the average oil refinery will produce more data in a day than, you know, all of Twitter, right in the same time period. And so if you're that person standing there and your job is to mitigate any, any problems and to track how this process is working, there's just no possible way that you, you as an individual, or even a highly competent and highly trained team of, of, of mechanical and industrial engineers could, could, could, could do that high dimensional math problem in their heads or, or, or even use tools to do it. But a computer is really good at that, right? A computer is really good at taking in lots of information, performing lots of, of analyses on it and, and applying lots of dimensionality reduction methods to that data to try to identify, you know, what are overall or systematic changes in it. And so we believe that well designed data tools should really be pulling the cognitive responsibility away from this data discovery to data reasoning. Because computers are really bad at reasoning about data. They don't really know why something changes. They might just know that it does change. But a person, particularly a highly trained industrial engineer, knows exactly why something might be changing if they're presented with the right information at the right time. And so we think about what is the equilibrium point or the perfect optimal point in which we can kind of hand off an automatically generated finding about these kinds of changes to an operator who can quickly move through that information and make an evaluation or take an action and then based on that action, have the system learn from that and get smarter and get better at identifying important changes versus changes that aren't important. Because at the end of the day, we want that experience for that human to be as good as possible. Because we really want to respect their time. Because, you know, and just the final thing I'll say in this is that, you know, our customers are a little atypical for data science products because they don't really care about software at all. Right. In some sense, software is like the thing that they have to do or the thing that they go to when they really need help of a very specific kind. But their job is to run a plant. And that is a, that is a physically intense job, not one that typically requires staring at a computer screen screen for very long. But when they do look at a computer screen, we want that to be a really high value interaction.

Exactly. And software is a tool to help them answer questions and to get deliverables.

Exactly. And it's in particular, it's a tool that they, that they tend to be pretty skeptical of. Right. So you have to really be able to show value quickly.

Yeah. I mean, they've been doing this longer than software's been around. Right?

Exactly. Yeah. And I, you know, one of the things I talk about with the team is, you know, if even generously, if you think about. All right, well, what is the history of kind of, of modern big data as we think about it today? You know, it's roughly 10 years old, maybe 15 years old, if you assume that, you know, Google and Yahoo were developing these things for years before they were released. But folks who've been working in the industrial space have seen hundreds of years of technology revolution change the way that they do their businesses. And so our little drop in the bucket barely makes a wave.

After a short segment, we'll jump right back into our interview with Drew to use everything we've just discovered to focus our attention squarely on Drew's approach to data science team building and recruiting. Let's now jump into a segment called Rich, Famous and Popular with Greg Wilson, who wrangles instructor training at datacamp. Hey, Greg.

G'. Day.

So, Greg, what do you have for us today?

Well, as a follow up to our discussion last time about using empirical methods to guide the design of programming languages, I'd like to see someone use data science to find actual design patterns in software. If you haven't run into the term before, a design pattern is something that comes up often enough to be worth giving a name, but isn't precise enough to turn into a single general purpose library. The term originated in architecture, for example. I think most people know what a porch is, but if you try to pin down a precise definition, it turns out to be surprisingly slippery. Similarly, programmers use terms like pipeline or plugin in more or less the same way to describe more or less the same high level concept. But there are enough different ways to turn those concepts into code that there's never going to be one definitive implementation that everyone uses.

So how can data science help?

Well, data science is pretty good at finding patterns, so I think it would be cool to see if we could use clustering techniques to find patterns in the way code is structured and used. You see, the design patterns we have now are all the product of experienced programmers on eyeballing their code, which is great as a starting point, but, you know, different experts will see different patterns or put particular instances into different clusters and so on. If we throw the tools we have against hundreds of thousands of source files Will we find the same patterns in how classes are structured? What if we look at traces of those programs execution? Will information about when objects are created and how they call each other's methods and so on give us more insight?

I can see how you would get source code from somewhere like GitHub, but where will you get traces from running programs? I mean, I'm all for helping science, but I probably wouldn't let you install something on my computer to look at what I was doing.

I don't think we'd have to go that far. There are enough software projects out there now with decent test suites that we could look at how the code runs itself and we could start small. If you look at the work of people like yermasajaniemi, there's actually a surprising richness in how single variables are used. A loop index isn't the same as a state flag, which isn't the same as an accumulator. But you might need dynamic analysis to tell them apart. I don't know. And I think the only way to find out is to have someone take a crack at it.

Aren't software engineering researchers already doing this?

Some. But the work I've seen has taken the hand rolled categories as a given rather than trying to validate them or discover new ones. And I think we'd learn a lot by having fresh eyes.

Thank you very much, Greg. If anyone in the audience is interested in giving this a try, please get in touch. We'd love to hear from you. Thanks Greg, once again and looking forward to speaking with you soon.

Thanks, Hugo.

After that interlude, it's time to jump back into our chat with Drew Conway. As we discussed earlier, your work that you personally do is for a large part of it is building and managing data science teams. So I'd like to really get into that now. So hypothetically, I'm going to give you a million dollars to build a data science team. How would you spend a million dollars to build a data science team and what would the team look like?

You know, it's interesting. The amount of money is obviously great because I could hire a bunch of data scientists, maybe not as many as I would hope with a million dollars, but you know, the first thing that I always talk to folks about when they're thinking about building a data science team is do you need data science?

Fantastic.

Do you actually need a data scientist? Right. I think there's a lot of companies that are data companies that confuse that for I'm a data science company. Right. And so there's lots of great Examples, you know, some of the ones I like the best. There's, there's a lot of opportunity in the world for taking old, difficult to navigate data sets and making them easier to navigate and easier to draw conclusions from. From. But does that need data science? You know, do you need a model? Do you need some prediction or some classification to be good at that? Probably not. So do you actually need to hire data scientists? And so the first thing that I think about is, okay, do we actually need that now? Okay, if we've convinced ourselves that we do, then the first question you need, the next question you need to ask is, you know, what, what, what kind of data science is important to my company? Am I a company that wants to do research and, and develop things at the cutting edge? And, and, and I'm willing to put a tremendous amount of resources and a tremendous amount of risk into commercializing academic pursuits? Well, if that's true, then, you know, there's a certain kind of data scientist that you would want to think about hiring and, and building a team around. Right? Folks who actually have experience in independent kind of basic research around methodologies and data, you know, might look more like a statistician than an engineer. And so thinking about that is really important. On the other side, and this is probably more relevant to, you know, your listeners or certainly is more relevant to folks in business. Is, you know, is data science core to your product? Is it the thing that actually gets people using it and buying it and coming back? Is it that prediction? Is it that classification? Is it that finding that you can provide them with data that makes them use the product? If that's true, then you want to think about hiring for a different set of skills. You do need software engineers. You do need folks who understand how to collaboratively build software and know that there's always tension between kind of purity of results and functionality of something in a production system. And then, you know, then the place where I'm happy to be talking even more is then, okay, let's think about building a recruiting process that actually supports the answers to all of those questions.

I'd love to talk about that more in just a minute. I do think the initial question, does the company need data science? Is incredible because I was actually speaking with a data scientist at Google a while ago who said to me an analogy that stuck with me. She said, I might mess up the analogy slightly, but she said, said a data scientist to a company is like a tiger to a drug dealer. And I said, I said, tell me more.

And right.

I want to know more. And she said, well, if you're a drug dealer and there's another drug dealer down the road who has a tiger on a leash, you're going to get a tiger on a leash. And she says, similarly, if you're a company and your competition has data scientists, you're going to want to get data scientists without actually thinking about whether I need one to build up my team or to, you know, meet whatever metrics I'm trying to meet.

I like that a lot. I also need to get a tiger now.

Absolutely. The other thing I found interesting about what you were saying is that we discussed earlier that the substantive expertise in your line of work at Alluvium comes from your customers and also your data scientists on the ground that they get out into the field along with everyone else in the company as much as possible. How do you think about hiring around the other parts of the Venn diagram? I mean, presumably you don't require that all your, all your data scientists have a, like, a really strong background in, in the mathematics and statistics from the academic view, nor do they have, you know, computer science degrees.

Yeah, we, we take our. You know, what I often say to folks, and this is certainly true for Alluvium, is, you know, your recruiting process is the first product that you're likely to build to, you know, to kind of MVP and completion. You know, if you're hiring someone to do a job, then the recruiting process should reflect as closely as possible what, how they're going to work in that job every day, or at least a close approximation of it. Right. The recruiting process is a naturally asymmetrical event. Right. You and your team, if you're bringing someone in to interview them, is going to get a ton of information out of that person as to how you think they may perform based on the questions that you ask them. Right. That's entirely one direction, the other per, you know, the person that you're bringing in to recruit. Depending on how you, how you, how you build that recruitment product, we'll get little to no or a lot of information about what it's going to be like to do this job with you. So if I have a recruiting process that starts with, you know, a puzzle, moves to a whiteboarding session, and ends with a culture fit interview and maybe some live coding that has, that reflects absolutely not at all what my job would be like when I get to get to work. So what we do at Illuvium is we try to build the whole process, start to finish, basically, as you might Imagine going from zero to a piece of code deployed into our production system as a data scientist through the whole process, right? So the first set of the kind of first round interview is mostly a get to know you a little bit, learn a little bit about what interests you, learn a little bit about why we might be interesting to you. In particular, what is it about this sort of unique intersection of industry, data science and product building that's really exciting to you? And then, you know, a little bit of a technical screen just to just have a little bit of a baseline to ask, okay, you know, what kind of tools do you like to use now? What are some examples of problems that you've solved with them? And can you walk us through, you know, maybe a piece of software that, you know, you put into production or if you're just coming out of an academic position, you know, a piece of software that you used in a, in a paper and how that process went. And then we have a, you know, a kind of take home coding exercise. But the coding exercise reflects exactly the kind of problems that, that we would use. And so, you know, in our case, we actually, you know, put a lot of effort into building the exam. It's, you know, we even built our own streaming service. It's kind of a stylized problem, but it's based on a real problem that we worked on having to do with wind turbine data. So we created a streaming service that streams and emits real data from a wind turbine. And you're asked to do some simple exploratory analysis and then try to predict some values coming off of that wind turbine. Then you submit that as a pull request to a GitHub repo that we give you as part of this exercise, then we as a team will look at the results of that. If we like what we see and we're interested to learn more about you and how you might fit in, we, we invite you for an on site interview. And the very first session in that on site interview is actually a code review of what you did. Because if you're working for us and you submit code, your code will be reviewed. And if you're a data scientist, that code will be reviewed both for the actual technical code itself, but also the methods you used. And so we like to be pretty skeptical with folks as to the choices they made, even if they are perfectly reasonable and make sense to get a sense of why someone chose to use one method versus the other. And a perfectly acceptable answer is this is what fit into the, into the time allotted for the exercise. And I think, you know, if you're a professional soft, you know, professional data scientist and you work in a business, that's a perfectly reasonable answer. And so, you know, that's, you know, just one example of how we do that. And then the next, the next meeting that you have is we talk about how, you know, how you might think about kind of expanding this into a larger project. You know, this is a kind of toy example of something. So how would we actually think about building this into a product that was designed to do this? And we actually have folks at the company play the role of a customer who would be asking questions about this product and then a software engineer at the company who would think about productionizing the system. And then we do have our own kind of culture fit interview and we talk about our company values and how someone might think about what matters to them in working at a company. And then we mostly just look, listen, we, we bring folks around and let them ask questions of, of me or of anyone else in the company. And we really try to, you know, give someone a sense of, by the time they've left the interview, they really know what it would be like to walk in day one and start working. And the last thing I'll say on that and, and really not, you know, not, not to kind of go through at length at the process. But if we do ultimately get to a place where we want to extend somebody an interview, I think it's really important that people know what they're going to be working on when they get here. Right. If you're hiring someone, then you can't articulate in at least bullet form the work that you want them to do when they get here. Why are you hiring them? Do you not have work for them to do? So we actually will send someone with their offer letter a list of tasks. We'll say, here's the stuff we'd like you to work on in your first week when you get here. Here's the things that we think you'll probably grow into working on in your first, say 30 or 45 days. And here are some big projects that we'd really like you to be part of in say the first quarter or four to six months of you working here. And that's really worked well for us that that kind of end to end process is a. This is what it would be like to work at Alluvium.

Well, it sounds like you've developed a very thoughtful approach to recruitment and spent a lot of time perfecting it as well. There are A lot of things that, that spring to mind there that are firstly the, the fact that you have code review, that's, that's fantastic because not enough places do.

Right.

But also the fact that you get people in to, to talk about it and the fact that communication is such an essential aspect of the process. The recru. Communication like see how people talk in terms of culture fit, but talking about code, explaining their own code. And also the point you made of explaining code around the constraints that you have in a business, whether it be time or finances. And time is, you know, our most important resource. The last thing that really sprung to mind when you explained the recruiting process is how much empathy there's involved towards people applying for, for the job. The fact that you're not only attempting to discover a lot about them, which you do in this process, but that they're actually discovering whether this is the right job for them. So what are the biggest challenges facing data science as a, as a holistic discipline these days and as a career path?

Yeah, so, you know, we touched on it a little bit before. I think the, you know, there, there, there remains this challenge of this kind of myth of the unicorn or the rock star or the superhero. I think, think that it is a persistent problem. And in fact, I may be somewhat biased now because I, you know, I work in, in a set of verticals that are, that are sort of coming, coming of age to a certain extent or at least entering their kind of adolescence in thinking about data science. And so they're kind of starting from a position where a lot of more mature industries were, you know, five more years ago, where they were really focused on finding a specific data scientist. But that is a problem. You know, we don't want people focusing on finding a perfect person who hits all of these different dimensions, I think. You know, so I think the one that actually affects all industries kind of equally is there really isn't a kind of theory of management or theory of product development for data science yet. You know, on the one hand we have lots of really good theories of product development for, you know, software products. You know, we have Agile and we have, we have Scrums and we have, and then some companies do it with, they have waterfall releases and they have all of these, you know, there's lots of great tools and lots of great theories of product development that you can use if you're building a piece of software. We really lack that for kind of professionally designed data science software product. There's lots of competing opinions, there's lots people will Borrow things from, from different theories of product development and they experiment with them and that's really important. And I think it makes sense, sense that we're at that kind of experimental phase. But I also think we've, we've sort of been through this process long enough that there's a lot of people who were, who have been data scientists for the last four or five years. And so the question is, you know, okay, what do we do next? What do data science managers do? What do they care about? How do we measure their success? How do we measure the success of their teams? We don't have, we don't really have a good set of metrics and a good theory for that. And that only serves to hurt us us because we, we don't want to lose folks and we certainly don't want companies and industries to be, to lack success because we didn't spend enough time thinking about how do we, how do we bring up and make the next set of data scientists teams most successful.

Thinking about professional development in that light is really interesting because I've noticed this in other aspects of data science as well, though, in terms of professional development and support for junior data scientists, for example.

Yeah, actually that's a great point. Right. I mean, I, I don't actually have the exact data on this, which is sort of shameful given the context. But, you know, you almost never see a job opening for a junior data scientist. Right. Explicitly defined. You know, you could probably do a scrape of indeed. Or any, any other greenhouse or any of these websites. And my guess would be a huge majority of data science posts would explicitly require, you know, three to four years experience. Experience. So what does that mean? How do we, you know, how does anybody get started? Does everybody have to be an intern or does everybody have to have some academic credential that is roughly equivalent to that experience, even though we know in practice that it's not. That's really problematic. And as I even think about, you know, this is, this is a mistake that we've made at Alluvium. And it wasn't until more recently that I sat down with the folks on the data science team and started talking about this that we really decided to break this up and recognizing that despite the fact that on paper someone might not be, you know, ready to contribute at a high level immediately, there's a tremendous amount of value in bringing in someone who's really smart, has an aptitude for this stuff and can learn directly from us how we think, how we think that that process should work and I only say that I wish that, that more, you know, folks who are hiring data scientists would think about that.

Yeah, that's right. And it seems as, as we've discussed at length, that alluvium is a place where you're learning on all fronts, constantly.

Yeah. Yes. For better or worse.

So what does the future of data science look like to you?

I think the future is quite bright. I think, you know, the, the future of data science in, in some sense may follow the path of lots of other technical careers. You know, we've, we've mentioned it a few times. I think, you know, to take it, to take a different example, you know, how many folks do you know are running around these days with the title webmath? Right. How many webmasters are there out in the world who are kind of the single points of failure for some big enterprise website? Hopefully very few, but there's still lots and lots of folks who are running out with the title data scientists who are kind of the single point of failure for all data analytics in an organization. And so for the future to be successful, we have to start breaking that up and thinking about what are the core competencies of a company that has data and data analysis at the core of its product. Obviously, data science is going to be part of that, but we have to figure out, you know, again, what, what do you mean by data scientists? What kind of work is that person doing? I think there's also a huge amount of work to be done in terms of, you know, development tools and again, development methodologies for doing that. And you know, there's a, there's, there's emergent titles like Data Engineer and people who are explicitly focused on making sure that the data scientists have the right data at their fingertips at the right time to be able to ask and answer those questions that make a difference for the business. But I think there's also this kind of emergent and related set of skills around DevOps and development ops for data sciences. What does it mean? How do we keep data driven systems running and how do we instrument them and measure them and make sure that they're working properly? How do we actually know that that one model in production is meaningfully performing better than another? And who actually builds that stuff? Right, because it's not a data scientist, it's somebody else. And then as we already mentioned, you know, who is the person who's actually managing this team and what is their career path look like and how do we measure their success?

So as a, as a final question, I'd like to know if you have a final call of action for our listeners.

Yeah, so, you know, I'll say if, if the. If the topics that we mentioned in terms of what we work on at Alluvium or that interview process sounds appealing to you, you know, please drop us Alluvium IO careers. We're actively hiring for data scientists, both at the kind of mid and senior position, as well as for the entry level folks, and then a whole swath of other opportunities for backend engineers and DevOps engineers and product engineers. So that's definitely one. And the other one that is if you're in the very beginning of your career or if you're not even ready for a career yet, you're a college student or you're thinking about making a career shift, take one thing that I think I did really well when I was in your position and just start kind of writing and talking publicly about what you're doing. A lot of folks talk to me and say, oh, you know, I don't like public speaking or I don't like putting myself out there. You know, I'm kind of introverted. I think the best hack for an introvert is to actually get yourself out there because then you don't have to spend a lot of energy going and talking to people because they'll come and talk to you. So, you know, start a blog, get out, you know, volunteer to speak at a meetup. Maybe it'll be a little scary the first time you do it, but it'll give you a sense of kind of what the opportunities are and, and you'll build a network and you'll meet people and I think you'll have a lot more success.

I couldn't agree more. And you'll notice how your approach and your ideas and your conceptions change when you try to formulate them to communicate to others as well.

Absolutely. Yeah. You know, sort of old, old platitude is you don't really know something till you have to teach it. And there's sort of a baby step to that is you don't really know something that you have to present it at a meetup.

I like it. And we'll put a link to your. The Alluvium careers page in the show notes also.

Great. Thank you.

Awesome. Drew, it's been an absolute pleasure having you on the show.

It was a pleasure. Was mine. Hugo, thanks for the great conversation.

Thanks for joining. Our conversation with Drew about how to build data science teams along with the unique challenges of building data science products for industrial users. We saw Drew's vision of building data science teams as a set of individuals who collectively cover all aspects of his data science Venn diagram and can communicate across it to go from data to insights and actionables. We also got tremendous insight into Alluvium's recruitment process, which reflects the job itself as much as possible. On top of this, we saw just how much Alluvium's work at building data science products for industry. Industry requires a combination of well hewn data science stack tools and new methodologies to deal with streaming data that is higher in volume than that created by Twitter on a daily basis. Make sure to check out our next episode a conversation with Mara Avrick, Data Nerd at large and tidyverse development advocate at RStudio. Mara and I will talk about exactly what it means to be a data nerd, the role of data science in sports, data for social good, good civic tech, and the role of data science paradigms such as the Tidyverse in the data science ecosystem as a whole. Do not miss it. I'm your host Hugo Bown Anderson. You can follow me on Twitter Hugo Bown and DatacampataCamp. You can find all our episodes and show notes@data camp.com community podcast.

It.