#40 Becoming a Data Scientist

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible.

It just works.

Power your data and AI transformation journey on datacamp.com.

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Renee Teit, a data scientist at higher ed analytics startup Heliocampus and creator and host of the Becoming a Data Scientist podcast. Renee and I will be discussing the many possible ways to become a data scientist, the common data scientist profiles, and how to figure out which ones may be a fit for you. We'll also dive into the fact that you need to figure out both where you are in terms of skills and knowledge and where you want to go in terms of career. Rene has a bunch of great suggestions for aspiring data scientists and also flags several important pitfalls and warnings. On top of this, we dive into how much statistics, linear algebra and calculus you need to know in order to become an effective data scientist and or data analyst.

Boom.

I'm Hugo Baun Anderson, a Data Scientist at DataCamp and this is Data Framed. Welcome to Data Framed, a weekly Data Camp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host Hugo Bown Anderson. You can follow me on Twitter ugobown and Data Camp at Data Camp you can find all our episodes and show notes@data camp.com community podcast for those interested, We've also got a special offer this week for Data Frame listeners the opportunity to try Data Camp yourself. All you need to do is email sales datacamp.com that's sales datacamp.com with a subject line podcast and redeem your free two week trial. Hi there Renee and welcome to Data Framed.

Hi Hugo, Great to be here.

It's great to have you on the show and I'm really excited to talk about all the things we're going to talk about today, the podcast that you worked on for so long, the idea of becoming a data scientist and your journey and process there. But before that I'd like to find out a bit about you and maybe you can tell us a bit about what you're known for in data community?

Sure. Well, I think I'm known for the podcast that you mentioned. It's called Becoming a Data Scientist. And I interview people about how they got to where they are in their data science journeys and whether they consider themselves to be a data scientist. And I plan to start that back up soon. So I think that's what I originally kind of got known for. But a lot of people also follow me on Twitter that may or may not have been an original podcast listener. So I have a Twitter account called Becoming Data Sci, and my name on there is Data Science. Renee. And I try to help people that are transitioning into a data science career to find learning resources and inspiration. I built a site called datasci guide.com which collects learning resources and people can go on there and rate them. And I hope to eventually, you know, make that into learning paths and things like that. I have a Twitter account called New Data Sci Jobs, where I share jobs that require less than three years of experience. And I try to share articles about learning data science and getting into this field to help people transition in. And then on top of that, I share my own data science challenges and achievements and try to encourage and inspire others so they can kind of watch what I do. And I'm really happy, especially in the last year, I feel to see a wide variety of people with different educational backgrounds that want to enter this field. So I intend to help them become data scientists too, because I think the broader the background of people in this field, the better it's going to get. So I guess that's what I'm known for the podcast and Twitter account for the most part, for sure.

And I think one a wonderful through line there that, of course, we're very aligned with at Data Camp is lowering the barrier to entry for people who want to engage with analytics and data science. And one of your wonderful approaches, I think, you know, you stated that on the podcast you'll even ask people you have on about their journey, but whether they consider themselves to be data scientists, kind of what this term means and how their practices apply to it. So it kind of demystifies data science as a whole, which can be a very, I think, unapproachable term with a lot of. A lot of gatekeepers around as well. And I think the work you do, you know, is very similar to how we think about our approach at Data Camp. So that's really cool.

Great. I definitely aim for that.

So how did you get into data science initially?

So this is My favorite question, because this is what we talk about the whole time on my podcast. So hopefully I don't run too long, but I will give a detailed answer. So I've worked with the data my whole career. You might call me like a data generalist. Right out of college I went to James Madison University in Harrisonburg, Virginia where I still live. And I majored in something called integrated science and technology. So it was a very broad major. It gave more breadth and depth in a lot of topics. We covered everything from like biotech to manufacturing and engineering to programming. But you kind of get a taste of everything and find out what you like and don't like. It had a lot of hands on real world projects. And one thing we learned in that in the programming courses in the ISAT program was relational database design. And this is something I had never done before then. But when I was in the class I realized, hey, I'm pretty good at this. I get this, it makes sense to me. And so right out of college I started doing that type of work. So I was designing databases, building data driven websites, and like designing forms and reports to interact with the data. I did a lot of SQL and helped design a reporting data warehouse and building interactive reports where people can interact with the data. And I did some analysis on that. So I wanted to take my career to the next level beyond that. And at the time I thought that a master's in systems engineering would fill in a lot of the gaps in my knowledge. So in my undergrad program I didn't have a lot of depth in math, for instance, or coding. I just had some introductory classes. So this program had, it was at the University of Virginia and it had simulation and modeling courses, optimization, statistics. And at the time I was kind of afraid of the math and I had to take linear algebra at the community college in a summer course to even qualify to apply for this master's program. And this is eight years after undergrad, so. So I should have known that it was going to be more math intensive than I originally thought. But I found out that like the title of each of these courses in the systems engineering program is kind of like a code for another type of math. So it was very math intensive, but I needed that. And that's something that I wouldn't have learned as much on my own if I'd at all. Self directed learning.

I have a question around that, which of course I get a lot as an educator, which is to be an effective data analyst or data scientist. How much linear algebra do people need to know.

I think it's good to understand the basics. It gives you a sense of what's going on behind the scenes of those algorithms to understand how data is being transformed and processed. However, if you're really going to be like an applied data scientist and not so much like a machine learning researcher, you don't have to really know all those intricacies. I'm glad I got a background in it, so I understand how these things work, but I don't use those skills on my day to day work. They're like packages that, you know, abstract all that away. So I don't have to be doing those type of calculations on a daily basis as a data scientist. So I would say it's good to get a grasp of it and feel like you understand the concepts, but you don't need to like have a mastery of the actual computations yourself. I mean, that's what computers are for. They can do a lot of that for you.

I agree completely. And I do think, you know, there is a lot of anxiety around learning these types of things, linear algebra and I suppose multivariate calculus in particular. And I do also encourage people to push through a bit and persevere a bit because, you know, a big part of the challenge is the language and notation.

Right.

Like a lot of the concepts aren't necessarily that tough, but when you're writing like a whole bunch of matrices and that type of stuff and get pretty gnarly pretty quickly.

Yeah. I still like shudder when I see certain depictions of like all, like the, you know, like you said, with multivariable calculation calculus and a calculus that's done in a matrix. And it just looks so overwhelming and the notation still gets me. So. So I feel that. But I'm glad I understand the concepts behind it, even if I still shudder every time I see those.

Yeah.

And you can have like some crazy notation that really what it is referring to is like the direction of flow along a surface or something like that. Like something that intuitively is quite easy to grasp, but we've got like this heavy archaic notation around it.

Yeah. And it's not even consistent. So I was in a program that had like professors from different, you know, departments at different universities, and my husband is a physicist. And there was a course where I was just really struggling with this particular type of computation and the notation. And he looked at it, he was like, you just learned this last semester? And I was like, I've never seen this before. He said, no, it's the same concept. It's just different notation. And that's when I really started to understand, like mathematicians and engineers, for instance, might use different notation for the same thing. So, yeah, it gets complicated. And now I do think if you're going to become like a machine learning researcher or go into like a PhD program, or you're developing things around the cutting edge of data science and really like pushing forward the field and building algorithms that other people will use, then you need to really understand that stuff. But if you're mostly applying algorithms that are already built, you don't have to get as in depth. But for statistics, I do think you really need a solid statistical foundation. So I would kind of say the opposite. Everybody that does data science really needs to understand basic statistics.

Well, great. So what then happened in, in your journey while or after you did this program?

Yeah, so while I was in the program, the Data Science Institute got started at uva and I had been hearing about data science everywhere and I, I kind of wanted to switch into that program, but I couldn't without completely starting over. They, they kind of moved as a cohort through their program. So I found out that I could take a machine learning course as an elective. And so I started taking that just because I wanted to know, like what it's about and how close is it to what I'm already doing. Because it felt like my whole career up to that point was kind of leading towards data science and I had never heard of it. So in this machine learning class, it started with a lot of that math and it moved really fast. And I'll be honest, I bombed that midterm. I really thought I was going to, like, fail out of the course, but I decided to keep going because the first half of the course was the math and the second half of the course was, you know, the coding and applied part of it, which is what I was looking forward to. So I thought, well, even if I get a bad grade, I want to learn what I'm supposed to learn in this course, so let me stick with it. And so like you said, with that abstract symbols and things, I was having a hard time even like understanding the textbook. But then the last part of the course, we had been building these machine learning algorithms from scratch. And by the way, all the examples were in C, but the professor let us use whatever coding language we wanted to. So I started picking up Python. At that point, I didn't have a very good grasp of C. I had mostly done Visual Basic, NET up until that point and SQL, and I didn't know Python at All, but I figured that was my chance to learn it. So I kind of learned Python as I went as well, which is probably part of the reason I struggled in the class. But by the end we had this project. So by then I kind of got Python and I kind of got what's going on with machine learning. And I was going to school part time while I worked. So I asked my manager, can I use this data that we use at work to apply it to this project that I'm doing in school? And he said yes, and that was fine. And so what I did, I was working in the advancement division at jmu, which is basically the fundraising arm of the university. And so for my project, I predicted which alumni were most likely to become donors in the next fiscal year. And so the professor loved it and even mentioned that maybe this is something I could publish in the future. And I guess that project outweighed my performance in the math portion of the course because I ended up getting an A in that class, which just.

That's incredible.

Blew my mind. But I was like, okay, now that it's kind of confirmation that this is something I should be doing.

Absolutely. And I just want to flag that before you go on that you've actually made an incredible point there, which is that you didn't do a project kind of in a vacuum. Essentially you were working on data that was meaningful for you, meaningful to your employers, and actually gave some insight into something important to a bunch of stakeholders there.

Yeah. And it took away like, you know, in class we have pre prepared data sets and they were all just lists of numbers. They weren't even like kind of related to the real world at all. You know, the professor chose those data sets because the answer would come out a certain way. And so diving into something that was unknown, that no one had really looked at before and in our, at least at our university, and finding some insights that I could share and you know, actually make a real world difference, that, that tied it all together for me.

In a learning experience as well. Working on something that means something to you and interests you is so important.

Oh, absolutely. I always encourage people to find data sets that are interesting to them and, and use them throughout their learning journey. Cause it's really, it keeps you interested when things get tough. And also you'll understand the output better if it's something that you've, you know, had a background in or even interested in. Like if you're into sports, you know, use a sports data set because you'll have a better sense of whether the output of your model even makes sense in context of sports.

And I always say, if you. A lot of people wear fitness trackers these days and they can get their own data with respect to exercise and sleeping patterns and that type of stuff, and they can quickly do a brief analysis or visualization of stuff that's happening physiologically for them.

Yeah, that's an awesome idea and definitely something I would encourage.

Awesome. So what happened next in your journey?

So, for my last class. So most of my program that I did in grad school was online. It was synchronous, so I was like, actually watching lectures over the Internet that were live. And there was a class there, but for the last semester, I commuted to campus, which was an hour for me. So I started listening to a lot of data science podcasts because I knew at that point I'm interested in this thing. So back then I was listening to, like, partially derivative and talking machines and the O'Reilly data show. Linear Digressions, Data Skeptic. And so I was just absorbing all of this data science information, and I knew that this was what I wanted to do. And so as soon as I graduated, I started diving into books about data science and teaching myself what I needed to know to get a job in this field and move on from. At the time, I was a data analyst and I wanted to move into being a data scientist. So, yeah, that's, that's what I, I did next. And then I applied to a bunch of different jobs that, like, at the time, I was just getting comfortable with data science. So I didn't want necessarily a data scientist job, but I wanted to make sure it was a job that was moving in that direction, because the job I was in wasn't giving me a lot of opportunities to really exercise these new skills and do machine learning on the job. So I knew I was good with designing analytical reports, I knew I was good with SQL. I had this new master's degree in systems engineering, but I wanted to grow into a data science role. So I started applying to a bunch of different jobs that partially involved data science, but they had components that I knew I already had the skills to provide value in. And I didn't get any of the first several I applied to, but I was starting to learn by doing those interviews what they were going to ask and what gaps I had in my knowledge so I can go back and learn more. And at the time, there were two different startups, one on each side of the country that apparently needed that type of generalist that could do Both the backend data engineering and SQL stuff and you know, move into the predictive modeling side. So I got two offers at the same time. They were both for remote roles that were like a combination of data analytics and entry level data science. I didn't have to do whiteboard interviews or coding interviews for either of them, which was nice because that part I don't think I was as good at at the time. But they needed somebody with my background and my experience with databases and someone that was good at communicating with the stakeholders. So I think that helped me stand out and I think we're going to talk a little bit more about that later.

Absolutely.

But one of those two job offers was with people I had worked with before. I worked at Rosetta Stone as a data analyst and a lot of the people at this startup had come from Rosetta Stone. So I was more comfortable with that one and, and took that one and have been able to build my data science and machine learning skills on the job. So that company is called Helio Campus and we work with university data. And I can tell you more about that if we're interested. But I've been in, in that role for about two years now as a data scientist.

Fantastic. And that, that's telling that the project you did, what did involve alumni data initially, right when you were first learning?

Yeah. And so at Helio Campus we've kind of, it's extended me into a new domain. It's still university data, but we work a lot with the student success data and admissions and things like that. So I guess I'll give a little brief overview of the company. So at universities they have databases that are like all kinds of data that you might not even think of when you're applying and enrolling at this university. So there would be a system for admissions and applications. There's usually a separate system for enrollment in courses and faculty. And then there's another system that they have for payroll and financials and then they'll have another system for the fundraising and alumni information. So they have all these different databases across campus and the leaders want kind of a big picture look at the students trajectory through this whole experience of applying and then going to college and then becoming alumni. And so to get metrics on that whole system, you have to combine that data. So we combine it into a data warehouse and we have reports in tableau that point at that data, so we have some canned reports. And then my job is to then work with the end users to do analysis that's not already built to answer questions they have about the students and to do some predictive modeling. One example is for the admissions team we have. We'll take a look at all the students that have been admitted to a university and try to predict how many of them will enroll or which ones might kind of be on the borderline of the type of students that sometimes enroll and sometimes don't. They might need some extra outreach in order for the school to get their attention, or students that need additional financial aid, for instance. So we've helped them get some insight by doing predictive modeling into what their student body looks like and what type of students they can expect to come to their university and what trend we expect in the future for. For their enrollment. So that's just one example of many different aspects of what we do at the universities at Helio Campus. But that's the kind of work I'm doing now.

That sounds like very interesting and fulfilling work, particularly with your kind of deep interest and mission as an educator and investing in learners.

Yeah, definitely.

We'll jump right back into our interview with Renee Teet after a short segment. And now we've got another installment of Statistical Distributions and Their Stories with Justin Boyce from Caltech. Justin, have you got some stories for us today?

I have a tantalizing story today. It's the story of the binomial distribution. As I mentioned last time, when you can match your experiment to a story of a distribution, you know how your outcomes should be distributed. So knowing these stories is really empowering. Before I tell the story though, I want to mention that this idea of matching a distribution to a story is not my own and has appeared in many books on probability and statistics. My favorite, though is the book by Blitzstein and huang and the Stat 110 course at Harvard associated to it. I put a link to that in the segment notes.

Ah, yes, that course is famous. Videos of every lecture are online now. Please tell us a probability story.

Okay, if you remember the last time we talked, we discussed the Bernoulli distribution. The story of that distribution is as the outcome of a Bernoulli trial is Bernoulli distributed. Recall that a Bernoulli trial is an experiment whose outcome can be encoded as 0 or 1. In the lingo, we say that getting 1 is a success and getting 0 is a failure.

And the canonical example being a coin flip. A possibly biased coin flip, right?

Yes. Now let's extend that to another story. Say I flip a fair coin 20 times and I count the number of heads I get. The number of heads is binomially distributed. More generally, if I perform N Bernoulli trials, each with a probability P of success, the number R of successes is binomially distributed.

So the binomial distribution has two parameters then?

Yes, the probability of success P, just like the Bernoulli distribution had, and the number of trials n. So some people.

Get a bit confused and think that the number of successes or heads R is also a parameter, right?

Yeah, that's a common misunderstanding that I have seen with my students. The number of successes is something that is observed. It is not a parameter, but a random variable, a quantity that can vary meaningfully from experiment to experiment. How big or small R is depends on the parameters N and P. Okay.

So we have the story what might be binomially distributed then?

A classic example is a box of light bulbs. Say I buy a box containing n light bulbs produced by a factory known to produce defective bulbs with probability P. The number of defective light bulbs in my box is binomially distributed. You know, I love biology. So here's another example. Say I have N cells and each is going to divide. To divide, it must copy its DNA. So there are N opportunities to get a mutation. The number of daughter cells in the next generation that have a given mutation is binomially distributed.

That mutation example does not seem as obvious as the light bulb one. Can you expand on that a bit?

Well, think of each DNA replication as a Bernoulli trial. The success is acquiring a mutation and failure is not acquiring the mutation. The probability of getting a mutation is P and there are N total replications and therefore N Bernoulli trials. This is the skill mapping your problem to a story.

Great, so you know it's binomially distributed. What can you do with that?

Well, once you know the distribution, you can start to compute expectations about outcomes. You know, the probability, mass function, the mean, the variance, etc. You're empowered to do statistical modeling. Now, there is one more thing I want to point out. The Bernoulli distribution is a special case of the binomial distribution. And you can tell that from the stories. The Bernoulli distribution is just a binomial distribution. When the number of trials n is one. You'll find that distributions are often related to each other as special cases or limits. And that is especially clear when you know their stories.

Great, so now we've got two related stories. I really hope we can hear more from you soon.

It would be my pleasure.

After that interlude, it's time to jump back into our chat with Renee Teep. It was fantastic to find out once again about your journey to becoming a data scientist. And something that of course you do is insist through your podcast, through a lot of different media, that this is only one journey, right? That you know everyone's journey, particularly to becoming a data scientist. There are a lot of different paths and there isn't a one size fits all approach to becoming a data scientist. And that before actually deciding on a path, people need to figure out both where they are and where they need to go and connect those points somehow. So what I'd like to know is what questions do aspiring data scientists need to think about when figuring out where they're starting from on their journey?

Yeah, definitely. So that's actually why I started my podcast, because I was listening to all these other podcasts showing what cool stuff data scientists were doing, but none of them had focused on how did they get there, what, you know, what did they do. And so I started asking questions and one of the things I realized that you have to assess, no matter which different educational background or career background you have, is your starting point. So the kind of questions you need to ask to map out your data science learning path is like, have you coded before? What language have you coded in before? Data scientists typically learn R or Python, often need to know SQL. How comfortable are you with the mathematics and statistics and do you need to brush up on those things and get some refreshers? Or maybe you need to take it to the next level from where you're at. Have you ever presented a report based on data? So have you done an analysis in a professional setting before? Have you ever answered questions with data? These are like the basics that you need and then you're going to probably be working in a particular domain. So within that field, do you know the lingo? Do you know what kind of data related career paths there are in that domain? So how you might focus in your data science learning to target one of those career paths. So you might want to like talk to a data scientist in that domain or, you know, analysts in that field and get a sense of the common questions and state of the art of what problems are they working on, what are they asking? So you get that language. So it's kind of this baseline of all the different parts of those common data science Venn diagrams that you see of how many of those pieces do you need to still need to work on to fill in? So, so you're just assessing your, your starting point and then next you'll look at where you want to go so that you know how to Map out that learning path.

Yeah. So to recap, essentially we have coding chops, whether you can program what languages, comfort with maths and stats, then communication skills and actually presenting, I was going to say data based reports, but I really mean reports based on data and then domain knowledge. And I think these are definitely very important aspects of your own practice to analyze when figuring out where you're starting from. And then of course, as we both said, you need to have an idea of where you want to end up. Now this may be a relatively amorphous, changing, vague notion, but what are the typical data science profiles that we've seen emerge that people can end up as?

Yeah, so as you mentioned, like data scientists can mean a whole lot of things. But I've noticed that there seems to be these groupings of specialties within data science. So there's like an analyst type of data scientist. These are people that are usually working with like end users or leaders or other people in the business. They're understanding the kind of questions that can be asked and figuring out how to convert those questions into data questions and determine do you have the data available to answer those questions and doing the analysis and then presenting the results and probably developing data visualizations for those kind of things. There are the engineer types of data scientists that are doing a lot of the back end work, the coding, working with databases and data warehouses, probably doing some of the feature engineering, working with big data systems and technologies that can handle massive data sets, building those data pipelines that support the analysis. Then there's what I mentioned earlier, the researcher type of data scientist, that they're improving those cutting edge algorithms and developing new tools and techniques. So that's a different focus of data science. So I'll say that most people end up doing some combination of these things, but you end up specializing either in like the analysis part or the engineering part or the research part. And my current role, I do a lot of the back end engineering stuff because I have that background, but also mostly focusing on the analysis tasks and communicating with people at the universities, the institutional researchers and decision makers that are going to use the results of what it is that I'm doing.

Yeah, great. So we've identified the three archetypes, the analyst, engineer and researcher as endpoints or at least career paths, knowing kind of the ways we need to think about where we are and knowing where we can end up. What are paths that you would recommend? What do recommended paths look like? Essentially?

Yeah. I'm hoping to formalize this more in the future with information I'm gathering at DataScuide, but it really depends on the individual. So that starting point that you assessed, the ending point of where you want to end up at and what are you comfortable teaching yourself or taking courses in learning online, deciding if you need to go back to school. I do think it's a myth that you need a PhD to be a data scientist. I don't have one. A lot of data scientists I know don't have one. So I would say go back to school if there's something like there was for math for me that you would be uncomfortable teaching yourself and you really need someone else to help you understand like the fundamental concepts there. Talk to someone that has a similar background as you and has become a data scientist. Or, you know, find people on Twitter that seem to be following paths that you like and you want to follow that. And then do that project based learning like you talked about, like finding the data set that has the information you're interested in, whether that's, you know, sports statistics or political data or geospatial imagery or medical data or entertainment data. Like there's so many different types of data out there that you can find something that's really interesting to you and ask a question that you can answer with the data and then learn whatever techniques you need to learn in order to answer that question. So I think project directed learning is really valuable, but that exact path and what resources you use. I have a really hard time like recommending any one thing because different things work for different people. Though I would recommend keep trying different things until you find out what works for you. Like, don't get discouraged if you pick up a book that a lot of people say is popular and great and you don't really get it and it's not sinking in for you. Just try something else. Don't give up and say, oh, I'm not cut out for this because this popular book doesn't make sense to me.

Yeah, there's a lot of great advice in there. Something I haven't thought about a lot beforehand is talking to someone that has a similar background, essentially finding people like you. And I think this is really cool because after you've done the work of identifying where you are and where you want to go or where you'd like to be in whatever time frame you're thinking, I think it's easy to forget or to think that there aren't people like you out there and that you're alone in this, this journey, particularly in a field that's moving so quickly. So to find people at different points in their career who are like you, that type of community, you know, to advise or be a mentor or mentee later on, these types of things is an incredible idea.

Yeah. I think another thing that I just thought of that ends up being difficult is just even orienting to the terminology. So even when you're out there looking for someone like you like, there's a lot of weird words that are used in data science that can be confusing at first and you don't really know, like, is that person doing what I think I want to do? So I have an article on my blog about how I use Twitter to do this, so podcasts like yours are great for that. Like, learn. Just hearing people talk about data science and learning, like, what kind of things data scientists have to think about. And so when I was ready to move into this career path, I got this book. It was called Doing Data Science by Kathy o' Neill and Rachel Schutt. And that was great for me in terms of getting an overview of, like, the big picture of what is this stuff and what do I need to learn and what are some of the basic terms? And it pointed you at other resources to learn. So, yeah, just orienting to, like, how people even talk and what the. You know, what matters in data science. And maybe there are things that you actually know already, but it's called something else by data scientists because data science is kind of a combination of fields that have already existed for a while. So, yeah, just learning that terminology and listening to data science and watching them on Twitter and reading articles to figure out what you don't know yet is important first step.

And so in terms of this journey of becoming a data scientist, can you suggest any. Any learning tasks for beginners?

Yeah, I would say build a report like you were saying, maybe use your own data with, you know, from a Fitbit or something like that. Like, just explore a data set and do some basic statistical summaries and then practice communicating those results. And so as you learn, you're going to be using different tools and techniques, but you want to make sure that the outcome is always understandable. And so see if you can bridge that gap as you go. And actually, I think when you're learning is a great time to do this because this one, it's fresh and new to you as well, so you can bridge that gap between the technical analysis and using that information to make decisions and talk to people that are less technical to get the point across. So constantly, like, blogging is a great way to do this talking to friends or people in your field is a good way to do this and just explaining the analysis you did, but in a way that just makes people comfortable that you know what you're talking about and then makes that information usable without getting into too much of the nitty gritty of like, statistics behind it, for sure.

And I do think working on data sets that are relevant to you is so important. And the Titanic and IRIS datasets don't count even if you think they're relevant to you.

I agree.

We need to move away. So I think you dispelled very importantly the myth that you know you need a PhD to do this type of stuff. I'm wondering what other potential pitfalls or warnings you have for people along the way on their journey.

I think there's some misconceptions about how much you need to learn, and it's a pitfall is that it's really easy to get discouraged when you're learning. So there are so many topics under this umbrella of data science that you can easily get really like overwhelmed and not know where to go. Especially with self directed learning, you have to kind of balance learning enough to qualify for the type of job you want, but then not over planning it or overdoing it to the point where you're, you're starting to feel totally off track and psyching yourself out and feeling like you're never going to make it. So in a talk I gave, I talked about it like you're planning a trip and so you could plan it out turn by turn and print out the directions and know exactly, you know where you're going to turn and what it's going to look like at each of those turns. But you still want to have your GPS handy because if you run into unexpected traffic or road closings, you got to route around that. So at some point you're going to feel lost in your learning or like you've totally hit a roadblock. But instead of giving up, you might just need to go back and find other resources to get you like more comfortable with the topic before you move forward again or decide, do I really even need to learn this? Maybe you can totally skip that part and come back later when you have a better understanding. So instead of just getting stuck and waiting for things to kind of clear up in front of you, just be prepared to reroute. So there's a whole lot of different paths to a data science career and just be prepared to change course. And also I think a lot of people look at those Terrible job postings that are like a wish list of everything that company could ever want a data scientist to be able to do. And they're basically describing a whole data science team in one job posting. And people think that they need to learn all of those things in order to get that job. So I would say no. Learn a few key things really well. Practice applying that knowledge you have to real world problems so you have experience, like overcoming challenges that you're going to encounter on a real job. And that will also help you have a story to tell in your interviews of how you overcame trouble and ended up having usable results in the end. So I guess what I'm trying to say is don't derail yourself and don't feel like you have to learn everything you've ever heard of in data science in order to be a data scientist. None of us know how to do everything. You just have to know enough of the basics that you feel like solid in that understanding and confident that you could pick up other tools and techniques as you need them. So I would say learn the basics and then learn a couple specialty items that might set you apart or are particular to the field that you're trying to get into. And also the communication skills are really important too, not just the tools and techniques.

Absolutely. And to build on that, something that you hinted at earlier is get out there and do some job interviews as well to find out what the market's like and what interviewers want and ask them questions to figure out what gaps you may have as opposed to learning in the abstract what you think may be needed out in the job market.

Yeah. And it can be discouraging not to get a job, but I remember, you know, once I did get a data science job, looking back and saying all those ones that I didn't get, they weren't right for me anyway, so why should I feel bad about not getting them? You know, I wasn't right for the job or the company wasn't right for me. And so, you know, once I found one and it was the right fit and I feel good about it and I like my job. So looking back, you know, I realized there's just times when it really gets, you know, frustrating or depressing if you keep getting turned down. But there's so many different kinds of data science jobs out there. I think everybody can find one that matches their skills, even though it might take a while.

Yeah. And I do think it's incredibly discouraging and horrifying to, you know, not get a bunch of jobs In a row advice I give, which I definitely don't necessarily I find it difficult to take myself though, is that you only need one hit. You're looking for one hit out of a bunch of opportunities. And the ones that don't work out can be really incredible learning experiences as well. That doesn't make it any less brutal to be turned down. Right?

Yeah. It's not until after the fact that you look back and realize, like, how much you learn and how valuable those, you know, rejections were.

Yeah, exactly. So talking about what employers are looking for, I think one thing that we can forget about when thinking about data science in the abstract is that a lot of the time it's used to solve business questions. And you have a great slide that demonstrates how data analysis and science can be used essentially as an intermediary step to get from a business question to a business answer. That. So this movement from a business question to a business answer is factored through data science. So I'm wondering how key this concept is to your understanding of data science as a whole.

Yeah, I created that for one of my first data science talks in order to illustrate, like, what I think the data analysis process is. And I got such good feedback on it and people really like it. So I go back to it a lot now. So for anyone that hasn't seen it, it has four little phrases with arrows between them. So it starts with business question and goes to data question and then to data answer and then to business answer. So I'll go through each of those. So for the business question, I don't necessarily mean like a sales and marketing kind of business, but like a domain question, something that a decision maker in your particular field or business might ask. And so your job as an analyst is to convert that into a data question. What data is required in order to answer it? Do we have it available? What related questions might we have to answer first to get to that one? And what type of analysis needs to be done to get us to a usable answer? Right. Then you have to do the analysis. So that's the data answer piece. So this type of analysis will depend on, like, what kind of field you're in, what's your role and your skills, what data is available. So the type of analysis differs, but basically to turn that data question into a data answer, you're doing analysis. Then you have to take the results of that and turn that into a business answer. So there's very few people out there that will want to hear your data answer. You have to be able to Communicate that in terms that a non data scientist can understand so they know what the data is telling them and can use that information to make a business decision. So you have to be able to convey like statistical results and uncertainty in business terms and explain what your analysis like means and does not mean so it's not misused. So a report, when we talk about building a report in the real world, the end result is usually not some sort of statistical readout with model evaluation metrics. It's like a presentation of the results that clear and usable by people that are not data scientists.

Absolutely. And I do think to keep in mind that we're always attempting to answer business questions or develop business insights in this context is incredibly important. So I want to shift slightly. We have a lot of aspiring data scientists and learners out there and I'm wondering what's your take on where people can learn like particular places people can learn the skills and knowledge necessary to become a data scientist?

Well, like I said, I have a hard time giving specific recommendations because it's so personal. But I've heard great things about Data Camp. Of course, it's actually the highest rated course system on Data Sci Guide. So people that use Data Camp seem to really love it.

That's great. I'm personally I'm a huge fan of Data Camp. Whether there's any bias involved here and.

I'm not saying that just to suck up, it's really, people love it. Also there's Dataquest, there's Khan Academy for some of those basic skills. There are lots of books out there. People tend to really like the O'Reilly books and there's some other favorites. So again, I hesitate to give specific recommendations just because they vary so much. So people can tweet me if you looking for a certain, you know, resource that will get you started from where you're at. And usually like I retweet that and lots of people that follow me will help answer. So it's really kind of a personalized answer. But there I'll just say there are a ton of resources and it's also, it's easy to get overwhelmed by the resources. So don't be afraid to ask to find what might be best for you. And then if someone recommends something and you really don't like it, don't feel bad about that either. Like just move on to the next thing. So yeah, I mean my site, Data Sci Guide, I'm trying to collect those reviews from data science learners so we can get a sense of what did you need to know before you use this resource because that tripped me up a lot when I was learning is there. There weren't clear prerequisites for certain resources and I would, I would start out real excited and like, yeah, I'm getting it. And then, you know, five lessons in be totally overwhelmed and wanting to give up. And I think that's this dangerous. So, yeah, talk to people that are like just ahead of you on the learning path maybe and find out what helped them get to, you know, for that first over that first step from where you are to where they are. And maybe not reach out to like, people that are already working as data scientists, but other data science learners.

We'll jump right back into our interview with Renee after a short segment. Now it's time for a segment called Programming Topic of the Week. I'm here with Emily Robinson, a data scientist on the growth team at datacamp. Emily has just launched a data camp course called Categorical Data in the Tidyverse and I wanted to have her on to discuss it because it's so exciting. I for one, have spent a bit too much of my professional and personal life wrangling categorical variables. Hi, Emily.

Hi, Hugo.

Emily, can you do me a favour and define categorical variables for our listeners?

Sure. Categorical variables are variables that fall into a pre specified number of groups. For example, survey responses to a question about your country of origin would be a categorical variable. Technically, categorical variables aren't ordered. If they are ordered, they're called ordinal variables. For example, a question on income that asks if your income is between zero and $10,000, $10,050,000 or more than $50,000 would be an ordinal variable. There's a preset number of answers and they have an order. While ordinal variables may have numbers in them, they're not numerical variables. The values of numerical variables are just numbers. You can do mathematical operations like taking the mean or max, which you wouldn't be able to do with this ordinal income question. Often people refer to ordinal variables as categorical, so we'll do that here.

All right, so what do our listeners need to know for when they encounter these categorical variables?

Well, it used to be that dealing with them in R could be very frustrating. R has a special way of representing categorical variables called factors. One problem I often had was making effective visualizations. If you're plotting an ordinal variable, you want your graph in the correct order. On the other hand, you may want to make a graph of average income by occupation, in which case it looks best if you order the occupation axis by average income. Or maybe you have 60 categories that you want to reduce to the 12 most common, putting the rest in other to fit into one graph. I swear I could never remember how to do any of these, and even when I found a solution on Google, it wasn't always easy to implement.

Ouch. But you said used to be. Does that mean there's a better way now?

Yes. In 2016, Hadley Wickham came out with a new package called Four Cats, which is all about working with factor variables. It is part of the Tidyverse, so it works well with other data analysis packages like ggplot2, bplyr and tidyr. It solves a lot of the headache of working with factors, and all the functions start with fct, making them a lot easier to remember.

That's awesome. What resources do you recommend to learn more about working with factors?

Depends on what you want to do with them. To get started, there's a chapter in R for Data Science, an introductory data science book by Hadley Wickham and Garrett Grolimund on factors. If you're interested in doing statistical tests, datacamp just launched a new course on inference for Categorical Data by Andrew Bray, an assistant professor of statistics at Reed College. If you want to learn more about the history of factors in R and specifically why when you read In Data, R defaults to making all strings into factors, check out Roger Peng's a professor at Johns Hopkins. Post Strings as Factors an unauthorized biography Emilian McNamara, assistant professor at University of St. Thomas, and Nicholas Horton, professor at Amherst College, wrote a paper called Wrangling Categorical Data in R. They compare multiple methods for doing the same task, and spoiler alert, the Tidyverse comes out ahead in terms of the compactness and robustness of the code. That's part of the reason I chose to use the Tidyverse in my new course.

Excellent. Thanks, Emily. And I can't wait to check out your course. Time to get straight back into our chat with Renee. So something we've been talking around, Renee, is Twitter, which can be an incredible resource for aspiring data scientists. So maybe you can tell me a bit more about that.

Yeah, so in addition to all like the books and courses and tutorials, like I mentioned earlier, I really use Twitter a lot to get the lingo of data science, and there are these great communities on Twitter and you can usually use them by searching for certain hashtags, so I'll give you a few of them. For Python people, there's PI data py ladies py4ds p y the number 4ds for people learning r, there's rstats and r ladies, a hashtag r for ds. So these are all hashtags you can search. A lot of those have Slack channels too. So there's like a data science learning club, so Slack channel that some followers of mine started a while back based on my podcast learning activities. There's a Slack called Data for Democracy. For people who want to get into political data. There's a hashtag for data ethics. So I'm sure there's similar groups like these on other social media like Facebook and LinkedIn. But I'm mostly on Twitter, so I have a whole blog post about using Twitter to learn data science. And if you start searching for hashtags related to what you're learning, you'll usually start finding the kind of leaders or the hubs in these communities. And you can learn a whole lot just by following them. And then if you ask a question and use that hashtag, you'll usually get an answer. So it's pretty cool.

That's awesome. And we'll link to your article on how to use Twitter to learn data science in the show notes as well. So for learners, how will they know when they're ready to actually be a data scientist or start interviewing?

Yeah, so I think people are ready to start applying for jobs before they feel fully ready to make that jump. So don't wait too long to start looking. And like we talked about, like, doing those interviews is really instructional as well. But I'd say that you're ready when you're confident enough with those basics. So you know how to do exploratory data analysis and do some statistical summaries. You know that basic feature engineering, how to get a data set into shape that you can use for machine learning. You know how to do some of that data pre processing and cleanup. You can build a good report and a data visualization and communicate the results. Maybe you've used a few basic, commonly used machine learning algorithms like logistic regression and random forest, so you're confident enough with these basics that you know that you're not going to be totally struggling on the job. But once you feel that you have that solid understanding of, like how machine learning works and you can apply it, you probably want to also add in a few specific techniques that will make you stand out. Either something you feel like you're good at, maybe you're really awesome at building, like pretty visualizations that are easy to read. Maybe you're really good at that back end data engineering stuff. So something that you can say is your specialty when you're applying for the jobs, but you don't need to check off the entire list of every algorithm and every tool and technique out there. I've interviewed for jobs that included skills that I already had throughout my career and I was confident with, plus some skills that I was still picking up. So if I knew that I could understand what people wanted and I was confident enough that I could pick up those new tools and techniques along the way, then I realized, like, I got a job before I thought I was ready and, you know, at least I hope and I've been told that I've done really well there. So a lot of stuff you can pick up as you go if you have the basics down. So, you know, don't feel like you have to be an expert in every area like nobody is. So start applying and you'll get a sense for what it is that you still need to learn in order to get a certain type of job. But yeah, don't wait too long.

And I think the field is so vast and there are so many techniques and new techniques emerging all the time that if you try to be as comprehensive as possible, you'll always feel there's more stuff to learn and you'll never get out.

Yeah, you're going to be learning on the job no matter how advanced you are when you apply. And there's a huge demand out there right now for people with data skills. So even if you get kind of a transitional data analyst type of role, you might not have the title of data scientist right away, but, but if it's a role that offers you the possibility of doing some machine learning, yeah, you can grow into that as you work.

So I want to shift slightly. And recently you gave a talk called Can a Machine be Racist or Sexist? And using this question you posed as a jumping board, can you speak to what you consider the biggest ethical challenges facing data science and data scientists as a community?

Yeah. So we could do a whole episode just about this and I'll connect you with some people that I think would be great interviews that could talk extensively on this topic. But, but the main purpose of me doing that talk was to get people to understand that even though you're using these mathematical algorithms and computers to get a result, that doesn't mean that things produced by data science are unbiased. So there's so many ways that bias, maybe you'd say racism or sexism. And I'm talking about the systemic kind. So not somebody yelling a word at somebody on the street, but historical racism that's baked into systems. I have that master's in systems engineering and I think I've always been kind of a systems thinker. So I picked up on this quickly and I was trying to share it with other people. And you can link to my whole talk for all the slides. I really struggled to cram in all the examples I wanted to give because there's really so much to learn here. With machine learning, you're really doing pattern matching. That's what those algorithms are doing are finding patterns in the data, which is a lot like stereotyping. So you have to be aware of what data is going into making those decisions and make sure you understand the model outputs. And it's not completely a black box where you don't understand why a particular decision was made by the model. When people's lives are being affected and biases can be introduced at every step along the way in this development process. So the data could have been incorrectly recorded in the first place. It might not be representative of the full population. It might be a limited sample. And you're training your model. Assuming it's generalized, it's going to generalize and it might not. Your data could contain historic biases. Like, for instance, crime databases are only going to contain records for crimes in areas that are policed. So if a crime at a certain location isn't observed or isn't recorded into the system by the police, an algorithm you train on that is going to think there was no crime there and make predictions accordingly. So it's just you're encoding not what's happening in the real world necessarily, but you're capturing what people are capturing about the system that you're looking at. There are certain techniques that can amplify bias. When you're doing your pre processing and model training, there's a question of what are you even optimizing for. So, for instance, YouTube has this problem where they're optimizing for viewing time. They want your eyeballs on their ads. So if something is like particularly crazy or creepy or exciting, people are going to watch it a little longer. And so those videos that are like really extreme will bubble up to the top and be recommended to more people because when you watch them, you might be fascinated by it and watch longer. And so it can kind of radicalize people. They people might get to the point, especially kids, I think, where you can't necessarily separate the truth from this fiction that's like constantly in front of you because that fiction is exciting and interesting and makes you watch longer. So what you're optimizing for and what kind of effects that could have is important. How do you even decide when to stop optimizing or if the results of your model are good? That's a decision that requires a human input. How do you know if the results of your model are being used properly and it's not being misused or misinterpreted? There's people and people making decisions at every step along the model development process. So you can't say that, oh, it's automated and computerized. There's no bias involved. There can be bias introduced at every single step.

And a lot of these issues are cultural. Right. As well, that as a community of data scientists, we're only now really starting. Well, there's been work done on it previously. I don't want to dismiss that. But we're really only starting to think collectively about how to approach these problems now.

Yeah, definitely. Yeah. And it's a culture of. Of how the company is run, and it really takes us data scientists making decisions about what we're willing to do as well. And so much of this, like, the models are being built under pressure for deadlines and being rolled out, and you might not even know how it's being used in the end. But. But just being aware of the impact of these things that we're building is important. And I love this quote from Susan Ettlinger in the TED Talk that she gave. She said, we have the potential to make bad decisions far more quickly, efficiently, and with far greater impact than we did in the past. So we're really just, like, speeding up these decisions. We're not necessarily making them better unless we make an effort to do that. So we have to make sure that as data scientists, that we're not causing harm. And we're in high demand right now, so we're lucky we have some choice. And what kind of businesses we're willing to work for and what kind of products we're willing to contribute to so we can make a difference in our future and hopefully make it a little less dystopian than the entertainment world imagines or that we can imagine just by being aware of this and making conscious decisions of what we're willing to build.

I couldn't agree more. So, Renee, do you have a final call to action for all our listeners out there?

Yeah. So I know there's a lot of people that listen to these podcasts that are just getting into data science but some people have been like lurking on Twitter for a long time, listening to podcasts for a long time, reading books. And so my call to action for them is like, dig in, find a data set, start working with it. Tweet me at becoming datasci. If you need help, I'll connect you with an online community that can help get you started. So don't delay actually working with real data. And then my call to action for people that aren't new to data science is I would encourage you to read up on the data ethics so that you understand how the work that you do in this field can affect real people's lives. And there's lots of great books out there now. So someone remind me when this episode comes out and I will tweet a list and share a bunch of books that I've collected that I've either read already or they're in my Kindle waiting to be read. Because I'm really interested in this topic and it important to me and I think it's vital for people in our industry to be well aware of. So that would be my call to action for people that are already data scientists.

Fantastic. Rene, it's been such a pleasure having you on the show.

Great, thanks for having me, Hugo. I've been listening for a long time and it's exciting to actually be on here.

It's great to have you on, particularly because I've been listening, was listening to your podcast for so long, so it was a really fun experience.

Great.

Thanks for joining our conversation Conversation with Renee about all the possible paths to becoming a data scientist. We saw the importance when learning of working with real world data sets that are relevant to you and that it's helpful to know the basics of linear algebra, but far more useful to know more about basic statistics and how to think about data. We saw just how important it is when starting out to analyze where you are in terms of skills and knowledge and to have an idea of where you want to end up and then to connect those two points in your path to becoming a data scientist. Whether you want to be a data analyst, research data scientist, or the engineer breed of data scientists, the basic archetypes Rene identified these principles are the same. Renee also warned against being overwhelmed by the potential plethora of techniques and concepts you may want to learn and to know that you'll always be learning in this field. There's so much out there. Also remember that you'll likely be ready to start interviewing before you think you are, so jump in there. Also make sure to check out our next episode, A Conversation with Alan Downey. Alan is a Professor of Computer Science at Olin College and the author of a series of free open source textbooks related to software and data science. Alan and I will be speaking about uncertainty in data science and how we as humans are not always good at thinking about uncertainty, which we need to be in such an uncertain world. Should we have been surprised at the outcome of the 2016 election? What approaches can we as a data reporting community take to communicate around uncertainty better in the future? From election forecasting to health and safety, thinking about uncertainty and using data and data oriented tools to communicate around uncertainty is essential. All this and more in next week's episode. I'm your host Hugo Bauen Andersen. You can follow me on TwitterGoBown and DataCamp@DataCamp. You can find all our episodes and show notes@datacamp.com community podcast don't forget that. For those interested, we've also got a special offer this week for dataframe listeners, the opportunity to try datacamp yourself. All you need to do is email salesdatacamp.com that's salesdatacamp.com with the subject line podcast and redeem your free two week trial.

It.