#34 Data Journalism & Interactive Visualization (with Amber Thomas)

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Amber Thomas about data journalism, interactive visualization and storytelling. Amber is a journalist engineer at the Pudding, which is a collection of data Driven visual essays. We'll discuss the ins and outs of what it takes to tell interactive journalistic stories using data visualization, and in the process, we'll find out what it takes to be successful at data journalism, the trade off between being a generalist and a specialist, and much more. We'll explore these issues by focusing on several case studies, including a piece that Amber worked on late last year called How Far Is Too An Analysis of Driving Times to Abortion clinics in the U.S. this, along with much of the pudding's work, revolves around an essential current question, particularly with President Trump's nomination of Brett Kavanaugh to the Supreme Court, which could have lasting impact on the fate of Roe vs. Wade and the future of abortion legislation in the United States. I'm Hugo Bauen Anderson, a Data Scientist at DataCamp and this is Data Framed. Welcome to Data Framed, a weekly Data Camp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host Hugo Bown Andersen. You can follow me on TwitterGoBound and Datacamp. Datacamp. You can find all our episodes and show notes@datacamp.com community podcast before we get into the interview with Amber, I wanted to mention that Jonathan Cornelison, our CEO and everybody's favorite co founder here at Data Camp, has an article called the Democratization of Data Science in the current issue of Harvard Business Review. As you faithful listeners are interested in the democratization of such skills across workplaces and society, I wanted to let you know about this thoughtful piece that also provides context around the motivation behind what we do here at datacamp. In Jonathan's words, data science isn't just for data scientists anymore, if it ever was. I'll include a link in the show notes and you can also find it by googling the Democratization of Data Science and How HBR or Harvard Business Review for those interested. We've also got a special offer this week for Data Framed listeners, the opportunity to try datacamp yourself. All you need to do is email salesdatacamp.com that's salesdatacamp.com with the subject line podcast and redeem your free two week trial. Hi there Amber, and welcome to Data Framed.

Hey.

Hi. Thanks for having me.

Such a pleasure to have you on the show. I'm really excited to hear about your work at The Pudding today, data journalism, data storytelling, interactive visualization, all of these things. And I'd love to jump in and hear a bit about the Pudding and what you do there.

Yeah, definitely. The Pudding is kind of this amorphous Internet thing. We refer to ourselves as a collection of data driven visual essays. Other people have referred to us as like an online magazine or some people call us a blog, which I feel like we're not quite a blog and we're not quite a newspaper. We're kind of this weird thing in the middle. But basically what we do is we go out into the world and ask all sorts of questions, generally questions about things that people are already talking about. We joke that a lot of our stories are things that people would argue about over beers. So like a friendly argument about something. And we try to add data to that discussion or to that argument and then tell the story in a fun, interactive, visual way on the web.

That's cool. And I love that you described the Pudding as, you know, something amorphous and a weird thing in the middle, because essentially, from my perspective, the Pudding is something that has arisen relatively recently as a result of many of the new technologies we have. Right?

Yeah, absolutely. I mean, it's wouldn't be possible without a lot of the interactive tools that we use and the programming languages and all of the data that we have access to and the ability to analyze all of the data. You know, none of, none of the stories that we tell would be possible without all of that, like, computing power and those languages. So it is a new thing. And I think it started January of 2017, so it's like about a year and a half old.

And I do think this idea of interactive visualization, particularly with platforms and websites such as the Pudding, it's very interesting because it changes the whole paradigm of storytelling and journalism in a sense. Right. Instead of having thousands of words that people need to read and navigate in order to extract meaning, allowing people to interact with visuals and figures and that that type of stuff allows people to move through that space of data themselves. And I think this is something that we'll. We'll speak more about later in this. In this talk.

Yeah, absolutely. And we try to make our stories as visually driven as possible. So we actually will often like write a story and then go back, cut out a bunch of the prose and make sure that the visuals are telling the story. And of course there's still prose to give you like background information and some other insight. But yeah, we try to let the interactives and the Visuals be the actual driving force of the story.

That's actually something I've admired in much of the Pudding's work, is that prose will come up interspersed through plots or when you hover over something to add more information as opposed to directing the information flow.

Absolutely, yeah. It's kind of like bonus content rather than the content itself.

Great. So what do you do at the Pudding?

So just as the Pudding is kind of an amorphous thing, my role is kind of like that as well. My title is Journalist Engineer, which is kind of a weird collaboration or amalgamation of different roles that I do there. And this is the case for most of the people that work at the Pudding, but. But basically I have the ability to do all of the things for a story. So everything from coming up with the idea to collecting data, analyzing data, designing what the visuals should look like, writing the story itself, and programming the interactive viz for the web. So that's kind of why the title is a little weird. There's like the data analyst part and the journalist part and the sort of front end engineer part and the designer part and all of these pieces. So that's what I do there, all of the things.

And actually, as you know, your colleague and founder of the Pudding, Matt Daniels, has a great medium piece called the Journalist Engineer, which kind of elucidates a lot of these things. And we'll link to that in the show notes.

Yeah, definitely. He actually wrote that before we landed on that title for our roles, but we just kept coming back to the Journalist Engineer. We kicked around a bunch of other titles and just decided that one worked best.

It makes sense. But as you said, there are so many moving parts in this type of position. You mentioned web developer, data analyst, designer, journalist. And I'm just wondering what was your trajectory to pick up all these moving parts so that you could actually work in this role? How did you get into it?

So I took a very winding path to get to where I am now. So by training I am not any of those things. So by training I'm actually a marine biologist. I went to college for marine biology and chemistry. I went to grad school, have a master's degree in marine sciences. I worked as a research scientist for several years. So my background is in like academic science in terms of picking up all of those skills. I actually picked up some of them in my work as a scientist. So I started learning to do data analysis and statistics in the R programming language when I was in grad school and I used it on and off throughout my work. As a researcher, it really depended on who I was working with and what tools they were using. But, you know, data analysis and experimental design and all of those things that I still do today, that all came up in my work as a research scientist or as an academic. The communication of complex things also came up a lot. So, you know, when you're a scientist, it's really important that you can communicate the results of your research, whether that's in an academic setting or in a conference talk. You know, when I was working as a research scientist, I was at an aquarium, and so I was studying the animals that lived there, and I often had to communicate my research to children. So I got, I think, pretty decent at, like, explaining really complicated topics to anybody who is listening to me, which comes in really, really handy when that's your sort of job. The only difference now is that I'm writing it down instead of talking about it. Yeah, so I did a lot of that. I worked briefly. I had started this small science communication service where I worked with scientists to translate their research into something more accessible to the general public. And then I worked with an animator to animate that story into something that the general public could consume in a more exciting way. So all of that kind of came into play in sort of transitioning into this current role. The rest of it I have learned on the fly. So I started learning JavaScript and D3 programming specifically about a year and a half ago because I had an idea for a visualization that I wanted to use that I couldn't for the life of me figure out how to do in R. And so I just started teaching myself D3. And when you do that, you kind of learn HTML and CSS on the way. So I knew a little bit of that. But I have learned a bunch more as I've continued working in D3. And, yeah, so a lot of it I just kind of have picked up as I needed it.

Yeah. And it looks as though D3 is part of the bread and butter of how you work at the pudding. Is that safe to say?

Yes, definitely. That was the most steep learning curve when I started working with the pudding, because I had only been using D3 for like a month or two when I reached out to them about freelancing with them the first time. And they were really receptive to helping me out when I got stuck with D3. And I was really motivated to learn it. So. Yeah. And now I use it all the time and I still spend a lot of time on Stack Overflow.

Yeah. So I'M glad that you've said that. I'm glad that you said that. It was a steep learning curve because D3 is, I think, incredibly powerful. But it's also well renowned for being pretty tough to start off with, right?

Oh, absolutely. And you know, I, because I'm an R user, I was used to ggplot, where you can make a bar graph or something in like two lines of code. And in D3 that same bar graph would take like 50 lines of code, or maybe not quite that many, but it was way more than two. And I think that's because I had this sort of disconnect in my mind that d3 was a charting library in the same way that ggplot is a charting library. When d3 is really this, it can do a bunch of other stuff. Like it can make charts, but it doesn't have to make charts. So it gives you the ability to control every pixel on your screen, which is really cool. But that means that there's a lot of information that has to go into telling it where every pixel belongs and how you want it to be presented.

So when it takes so much work to write the D3 code, why is it then the state of the art or the most powerful or why is it kind of the standard in your line of work?

I think because it's so powerful, especially when it comes to interactive things, you can do so much with D3. And again, because it's not really. It doesn't have default charts. There's no bar chart function that you can just feed stuff into. If you want to add bars, you have to program it to add rectangles that's bound to your data that you've fed into it and all of these things. Because of that, it's incredibly customizable. You know, we kind of joke that all of our data, viz things are like bespoke. Like they're all custom written code and sometimes that's overkill. So we do on occasion for static graphics, make stuff in R and then sometimes we'll clean it up in Illustrator or other, you know, static design things and then you'll have just like a JPEG or a png, like an image on your website. But sometimes it works well to make it in D3 and have everything just like natively on the website instead of an image embedded there.

Yeah, exactly. And it was developed actually at the New York Times, right, By Mike Bostock.

Yeah. And I think it was only in 2011 that it came out. So it is still pretty recent, even though it's grown like crazy in popularity.

Fantastic. So I'm so excited about getting in and hearing about some of your projects and stories that you've worked on at the Pudding. So maybe you can start off by telling me about one or two of them.

Sure. So I've worked on quite a few stories now at the Pudding, and I did mention that my job entails doing all of the parts of a story, but for any given story, I'm not necessarily doing all of the pieces. A lot of our work is collaborative, where somebody does one piece of it and then somebody else does another piece. So this first example, I worked on parts of it, but this was a huge collaborative effort. So the story is called How Far is Too Far? And it basically focuses on how long it takes people to drive from where they are in the US to their nearest abortion clinic. This story was very sensitive, and it's one of our stories that has a little bit more of a political tone to it, but it's a story that we thought was really important to tell.

Yeah. And particularly at a time when these things are changing and legislators are altering this type of landscape that may affect a lot of people on the ground.

Right, Absolutely. And that's kind of where the general idea for this came from. So there's these laws that specifically target abortion clinics. And in the state of Texas, these laws were actually causing a bunch of clinics to close. And so there was a group that brought this to the, like, circuit Court of appeals in Texas, saying that when these clinics closed, patients that need to access these clinics now have to drive over 150 miles to their nearest clinic. And one of the judges replied, like, do you know how long that takes in Texas at 75 miles an hour? And that really got us thinking about, like, well, I mean, that's not a very good, like, feedback to that problem. But that does bring up an interesting point that the thing that really affects people is how long it actually takes to drive from where they are to where these clinics are. And a lot of other research projects we had seen around this focused on the distance, like, as the crow flies, when the driving distance is what really affects people. So we broke up the country into these hexagons and. And looked at how long it would take you to drive to the nearest clinic from the center of each of these hexagons and from the center of any city with a population above 50,000.

So, and I recall that in the first figure, there's a slider which allows me or the user to have a look at the Distribution of cities and places geographically where people can reach an abortion clinic in under a certain amount of time and change that and see how that changes with respect to the time that I. That I'm wanting to know about.

Yep, exactly. So we wanted to let people really explore the data themselves. So when we talk about something being a really long trip, I'm originally from Connecticut, where anything over like an hour feels so long. But I know people from other parts of the country where like a five hour trip is what they consider long. So we wanted to allow people to really experience this data in ways with whatever frame of far distance they have. So yeah, that first chart really lets you restrict the data and change how long of a drive you want to.

See and how does the story evolve after that.

So the next thing we wanted to really drive home is that we can say that you can reach a clinic from wherever you are in say, a two hour round trip drive. But the thing is that not all clinics treat all patients. So depending on how far along they are in the pregnancy, they may or may not treat a specific patient. So we made another graphic that is again looking at the whole country with all of these little hexagons, and we showed how the round trip time to the nearest clinic changes the further along in a pregnancy you get. So the further along you get, the, the fewer and fewer clinics will actually be able to work with you.

I won't give out too many spoilers, but there are quite dramatic differences there.

Yeah, absolutely. And that's something that was really surprising for us. And that's why we wanted to include this. When we started exploring the data, we didn't expect there to be quite as drastic differences as there were. So for this graphic, there's again a slider that allows you to change how far along in a pregnancy a patient is. And we have it autoplaying. So if you just scroll through, you'll see that map animate.

What's your impression of what this interactivity gives to the reader? Does it empower the reader or what's your take on this?

I think so. I think it really allows the reader to make the story their own. So again, if they don't really think that an hour round trip drive is far, they can decide how far they want to look at, like which distances they are most interested in looking at and how far into a pregnancy they are interested in getting this story from. So they can kind of change the lens of the story a little bit to be a little bit personalized. And we do have a table in the story as well that, you know, it shows you the cities with the longest round trip travel times, and it geolocates you. So it'll give you the cities with the longest travel times, but it'll also show you your city for reference. So, again, we're really trying to make these stories personal to the reader so that it drives home the point a little bit more. And so adding these interactions and some of these subtle elements really helps to make a story personal for whoever's reading it.

That's fantastic. And that actually provides a really nice segue into another piece of yours that I think is wonderful, which is the Greetings from Mars, which actually locates me or whoever's looking at it to compare Martian environments to where I am.

Right, Absolutely. So this one is a story that I actually did do all the pieces. So the data comes from the Curiosity rover, which is, of course, on Mars, and it collects a bunch of data, but some of the data it collects is about the weather in its current environment. And I was really excited to find this data, and I didn't know what to do with it. And then I started thinking about what if the Mars rover didn't understand what postcards were for and thought that postcards were just like when you go on vacation and you say, the weather is great and I wish you were here. And the Curiosity rover was telling this story through postcards because it just wanted to tell you all about the weather using postcards, because, of course, that's what postcards are for. So this story, when you scroll through, it is literally postcards that flip over as you scroll. And Curiosity tells you all about the weather on Mars, and it uses the weather where you currently are as a point of comparison. And so when I'm looking at it, like, right now I'm in Seattle, and it says, it looks like today in Seattle, the weather is partly cloudy throughout the day. And. And it gives you, like, a range of temperatures. And. Yeah. So it walks you through. And this one updates every day. So the data is the most current weather on Mars and the most current weather in your area as well. Same. So this one is very personal for.

The reader and I actually for the listener. And remind what I. What I said to you earlier. I was like, I thought it was great, but it said, it looks like today you're in Hoboken. I'm very close to Hoboken, but it gives me the temperatures in Fahrenheit. And I said, as an Australian, I'd love them in Celsius. And your response was of course if I was in Australia I would receive them in Celsius, but because I'm in the US I don't Right.

And it's actually pretty funny. Shortly after we published the story, somebody gave us similar feedback that they wished the story was in Celsius, and I asked where they were and I believe they were in Sweden. So I was confused as to why it wasn't working. And it turned out that they had their location services disabled and so we had a default location for the story as New York. So it was giving them information as if they were in New York, which was of course not what they were expecting, but it was functioning the way we wanted it to. So we did our best to make it as personal for as many people as possible. But sometimes, you know, we can't control for everything like that.

We'll jump right back into our interview with Amber Thomas after a short segment. Time for another blog Post of the Week segment. Hey there, Spence.

Hey Hugo. Today we're taking a look at a recent really thoughtful post about hiring a data scientist by Mikhail Popov, who was recently in the position of needing to fill a data scientist position himself on the team at the Wikimedia Foundation.

Awesome. I know several companies that rely heavily on Mikhail's post for their recruiting pipelines. What are his recommendations for hiring data scientists?

So the first step that he recommends is, first of all, just deciding what kind of data scientist you actually want. There's a lot of potential job descriptions that might fall under the title of data scientist, and it's critical to think really hard about which skill set you actually need. And that's not always what you think you need at first glance.

So what are Mikhail's recommendations when putting together a data scientist job description?

Well, he references several resources that came in handy when attempting to put together the most inclusive job description possible. The way that you craft a job description has a really large effect on the quality and diversity of the applicant pool that you're going to get. So it's always worth doing, taking the time to do thoughtfully.

And what else can you do to get a more diverse pool of applicants?

Well, so the most important thing is to just meet candidates where they are. So don't just post the job opening where you would tend to look, mikhail points out. R Ladies has one great example. If you're interested in attracting women data scientists, for example, go to them. The blog post contains an example of what their listing actually looked like, so check that out as a great example the next time you're Putting one together for yourself.

What's next after putting together your job description?

Well, Mikhail isn't a really big fan of live coding on a whiteboard for data scientist interviews, so he recommends replacing that kind of thing with a take home analysis task. That's just way more reflective of the kind of environment a data scientist will actually be in when they're doing the job. This gives candidates the opportunity to do background research, to figure out their way around the data set, and to use whatever tools they're most proficient in. Putting together a case study from some real problem that your team has solved recently is pretty easy to do. And it also comes with the benefit of giving you a lot of signal into whether or not a candidate can solve your specific types of problems.

What about after the candidates pass a take home? What was Wikimedia's strategy for interviewing?

This is pretty cool. So they split their questions up into five different categories. Three of them are pretty self explanatory questions about data analysis, about statistics and then about machine learning. Interestingly though, first they asked what the most important qualities of a data scientist are to the candidate, which sheds a lot of light onto that candidate's philosophy when it comes to actually doing data science. And they posed one more set of questions that I really think more people should really be asking questions about our ethical obligations as a data scientist. Again you can check out the blog post to see exactly what sorts of questions they used. But Wikimedia did one more thing that I really want to highlight because I almost never see it being done in practice. They actually created a matrix ahead of time that they used for evaluating the quality of the answers. This takes a little bit of extra time to do up front before you even get to the interviews but, but it really helps you and your team be completely objective about candidates performance and their answers to questions.

Thanks Spence. Well listeners, there you go. If you're hiring a new data scientist soon, be sure to read the post. We'll link to it in the show notes. After that interlude, it's time to jump back into our chat with Amber. And so you said something that I found very interesting that you did all the things on this piece as opposed to working in a collaborative environment. I'm wondering if there's anything very different when you're doing it all yourself about the process. I can imagine it's more frustrating but.

Yeah, yeah I think, I think it's, it's kind of interesting when I'm working on stuff on my own and when I say on my own like I still do rely on the team a lot, and I did have an editor for this piece to, like, bounce some ideas off of and things like that. So I was getting feedback throughout the process, But I find that I think through things out loud. And so when I'm working with somebody, sometimes that process is smoother or faster because I have somebody else to bounce ideas off of. And we work together on figuring out the story. When I work on projects by myself, sometimes I get really caught up in one part of it. So, like, I will be content to analyze data for weeks and never move forward with the rest of the story. Or I get really excited about one piece of it and then it makes sense in my head. But as soon as I tell it to someone else, they're like, ooh, that doesn't really make sense. And I have to take a step back and start over. So I've gotten better about it of at least being aware that I do that and looking for more feedback throughout the process when I work on projects by myself now. But yeah, it's a little bit different. Cause you just put your head down and move forward on this story and hope that it ends up in a good spot for sure.

But you're absolutely right. I have this all the time where I think something I have done is a great idea. And as soon as I start to explain it to someone else, even before they tell me it isn't, I'm like, oh, wait a second. That doesn't sound like.

Exactly. And that's actually what happened with the Mars story. My original thought, once I found out that there was all this weather data, I was like, oh, I'm going to write welcome to Mars, and it's going to be like a welcome packet for people who just moved there. And I was super excited, and I mocked up this whole thing. And as soon as I told it to the team, they were like, but, Amber, if people were moving to Mars, don't you think they would know what the weather was like before they left for Mars? And I was like, oh, yep, you're 100% right. It was just something I hadn't thought of because I was so excited about the data and the idea. And so it needed a. It needed a reframing for sure.

And that's true, unless it's one of those mystery vacation packages. But I wouldn't expect that to be. That'd be a horrible mystery vacation.

Yeah.

Like, seriously, I need to be back at work now. So there's one other piece that we discussed of yours that I think is so wonderful. It's the makeup shades piece. So I thought maybe you could tell our listeners a bit about this.

Yeah. So this actually was a story that came to us from a freelancer. So we do work with a lot of freelancers at the Pudding and and this was one that I had stumbled upon on Twitter. So this illustrator named Jason Lee was trying to come up with a better color palette for skin tones for the characters that he was illustrating. And he had written a blog post about using like the skin tones from emojis and given feedback from people on that. He decided to look into the shades of makeup that are being offered because makeup brands, you know, they have a stake in their colors actually matching people's skin tones. So he wrote this like little blog post about making a new color palette. And I read this and was like, this is amazing. I want more of this. So I reached out and asked if he would be interested in expanding the story for the Pudding. And he was really excited and he brought on another person to help us with the story. And so for this one, basically the idea came down to these days, foundation shades and makeup shades. They are all coming out saying that they have these diverse shade ranges. So, you know, Rihanna came out with this brand called Fenty Beauty last year and it had 40 shades when it launched and it was this huge thing. And now a bunch of other brands are like rushing to get up to 40 shades. And we wanted to see if all 40 shades are created equal across all of these brands. Like, just because you have 40 shades, are you actually making the colors diverse enough for the people that need them? And so it's this kind of exploration through the color shades that are offered through different brands here in the US And Japan, Nigeria and India. And Jason did most of the data collection and story writing and I did a lot of the, like, design of the graphics. Jason and I went back and forth a lot on how the graphics should be designed and shown to the reader to help illustrate our point as well as we could do it. And then I did all of the front end, like programming for it.

Great. So I won't say too much about it and we'll include a link to it in the show notes. But I do think it's, it's quite wonderful how you've demonstrated the distribution, the relative distributions of shades where, you know, where the highest density of shades are in certain products in certain countries. And there are some quite, you know, ones that make sense post knowing it, but some quite surprising results in there with respect to countries for example, in India, the result there makes sense after knowing the fact, but it is also surprising at first sight, I think.

Absolutely. And that's the feedback we've gotten the most from readers is that India was surprising and it was super shocking for us too. So that's always kind of interesting when you're still surprised by what you're finding.

Absolutely. But in particular, the way you show the frequencies and distributions is very, very nice visually. So I won't say anything more because our listeners should go and check these out straight away.

Awesome.

And look, I really, of course, and I really want to thank you before we move on for, you know, this has been a difficult task to describe interactive data visualizations in words on a podcast, and I think you've done incredibly well. So moving on, I'd like to know a bit about process because, you know, in data science, and science in general, a common paradigm of the process is to start with data and let it lead you to the hypotheses and then questions. But in what you do, you start with a question.

Right.

So how does this different approach change things for you as a practitioner?

I think it changes a lot of things, but primarily it changes where we find the data. So before I was working with the pudding, when I was just doing personal projects on my own, I would go and just find a data set and analyze it and look for just interesting things. But now basically the stories that we end up writing that end up being the most interesting are things that started with a question. So again, we're trying to come up with these questions that people are already discussing, questions that people are maybe having a friendly argument about. And we try to answer that using data, which usually not always, a lot of the times means we need to collect the data ourselves or scrape it from some source or. Yeah, it's usually not like a data set that's already existing and on the Internet and cleaned and ready to go.

Is there a challenge with the fact that you think you've got a correct or interesting question and the data, once you find it, doesn't really give you the story that you were hoping for?

Uh huh. Yep. So yeah, that happens a lot. That actually happened with the very first story I proposed. I was really excited about this story and I thought it was going to be great. And I collected all this data that was, you know, like I had to scrape a website and so it wasn't like a data set that already existed. And once I started analyzing it, it just wasn't interesting. And sometimes we're able to Recover, and we'll end up answering a different question than the one we set out to answer. So this is where we fall kind of back into line with what's more typical. Like, once you have the data, you know, you can start looking for other things because you might find surprising insights that you want to talk about either in addition to, or instead. But yeah, that first project I tried working on, every angle I could think of just was not. Not coming across the way I wanted it to. So sometimes that means we end up just like, killing a story and we just don't write it, which happens sometimes. But, yeah, I think of it a little bit as, like, publishing negative results in science. It just doesn't happen very often because basically all you're saying is, like, what we thought was going to happen didn't, and that's the end. And there's all, of course, huge debate about whether or not that should be the case and whether or not we should publish uninteresting things. But at least for the pudding, we try not to publish things that we don't think will be interesting.

Yeah, absolutely. And of course, in. In basic science research. But there's an argument that there should be more negative results being published.

Of course. Exactly. And I mean, that comes down to, you know, having people not repeat projects and experiments that other people already know aren't going to work and things like that. And in science, it does make sense to do that sort of thing. We've talked about making like, blog posts or something about sort of like failed stories for that purpose so that people are aware, like, hey, we tried this and it didn't really work, but if somebody else wants to try it, go for it. We haven't gotten around to that, but maybe if there's enough public interest, we can check that out as well.

And something you mentioned at the start of our chat is that you think about, at the pudding, you think about the types of questions and conversations you write about as the types of things you want to talk about over beer. Right? Yeah. I was going to say, I think starting with a question like that probably gives you a sense of direction and some sort of interest there.

Absolutely. It really gives you a sense of focus. Like I said, sometimes I have the bad habit of when I find a new data set, I'm like, I want to explore all the things, and I could easily spend like weeks and weeks just having fun and playing with the data set. But when you go in with at least some sort of vague purpose in mind, it helps to give you a sense of like, okay, here's where I should at least start. Because those big data sets can also be really intimidating if you don't know where to start. And I've done that too, where I've like downloaded huge data sets and been excited and then been like, I don't know what to do with all of this. And then it just sits on my computer and I don't do anything with it. So it, it definitely gives you like a direction to go and then you can branch off from there.

And I think I've definitely been there with large data sets and I think a lot of us probably have as well. So you're in good, good company. As we discussed earlier, this is a really new phenomenon, this interactive database stuff. And it's essentially a revolution in terms of how we can tell stories, how we can receive stories. The fact that the paradigm of storytelling in journalism in particular used to be from the printing press to the broadsheets, and that you'd have static information passed to you from a newspaper. Now that we have web based infrastructures, a lot of JavaScript and, and D3, which allows practitioners such as yourself and your colleagues to build this type of stuff, I think the future is an incredibly interesting space. And I'm just wondering in your mind, what type of future can we expect to see in terms of interactive data viz and the impact on journalism and storytelling in particular?

You know, that's such a great question and I think it can go in any number of ways. People are experimenting with all sorts of stuff in this space right now. They're trying out different technologies. So virtual reality and augmented reality are being brought into the data viz space and we're experimenting with the ways that we present data. I mentioned a couple minutes ago that we occasionally kill stories and that sounds like a bad thing, but we actually are set up so that we have the flexibility to do that. And that allows us to experiment with things which when you experiment, sometimes things are going to fail. And I think we're not the only people that are doing that. You know, lots of people are experimenting and trying to find new and exciting ways to present information. And you know, on the Internet, people's attention is the most valuable commodity. And everybody's always looking for your attention if you're a consumer of anything on the Internet. And so it's hard to break through the noise. And so I think that's where these interactive graphics and some of this new experimentation and things like that are really being leveraged. And so I think it's an exciting future. I have high hopes for it. I know that there's some discussion within the community of where everything is headed, but from where I'm sitting, I'm looking forward to it. I think we're in a good spot, as am I.

And I do think this idea of turning what essentially have been consumers as passive readers essentially into active storytellers to themselves is incredibly interesting. Getting people doing something and interacting as soon as possible. I mean, that's something we strive for at datacamp as well, really, is getting people coding as soon as possible and giving them kind of automated but interactive feedback.

Absolutely.

So we do have a philosophy here of getting people doing stuff with their hands, either on a keyboard or on a mouse or whatever it may be, as soon as possible.

Yeah, absolutely. And it really, like brings them into the story and it kind of makes it their own. And like you said, that's how Data Camp works as well. You know, you feel like you're actually like gaining the experience and doing the thing.

We'll jump right back into our chat with Amber Thomas after a short segment. We take a break from your regularly scheduled programming to continue our recurring segment, Data Framed for Social Good. As always, I'm catching up with Peter Bull, co founder and data scientist at Driven Data, which runs data science competitions and solves data problems with nonprofits, NGOs, governments and philanthropic funders.

Hugo, of all your wonderful segments, this is by far my absolute favorite. Please don't investigate me for bias.

Well, you may have some bias, Peter, but let's hope some accuracy as well. To recap, we've talked about the history of data for good, the current landscape, and ways for learners to get involved. What will we cover today?

I want to dig in to the broadest ways for data scientists and learners to get involved in having a social impact. We often get asked, what can I do if I have a few hours a week? And luckily there are some great options. First and foremost, I'd like to plug contributing to open source. The absolute easiest thing to do is to improve the documentation for a project and it really has a huge impact. Just think about all the times you read the installation instructions for a project and they just worked. Now think about the times that they didn't work. If your first contribution is updating that documentation, you're doing an amazing service for the community. Second, I want to talk about some communities that self organize into project teams. The most active group is called Data for Democracy. And to get involved, all you have to do is join their Slack channel. They have teams working on Things as wide ranging as a model for vehicle crashes in Boston to a data science code of ethics. Finally, I'd encourage everyone to take a look at driven data. We run machine learning competitions for nonprofits, and each one has a social impact. It's very easy to contribute from wherever you are and whenever you have time. These competitions rely on the creative ideas of competitors to find out the most effective solutions. We've had people predicting poverty levels, identifying animals in video footage, and automatically organizing school budgets. The winners for the competitions come from all over the world and have all different backgrounds.

Thanks again, Peter, for coming and sharing these ways to get into data for good. I'm excited for many of these and will probably even sign up for a driven data competition myself.

We'd love that. But don't ask me for tips. I can't play favorites. You'll have to win by using all the most interesting things you've learned in the course of your interviews.

Time to get straight back into our chat with Amber. I've got the strong sense that you and your colleagues are all generalists, essentially, in terms of you do a lot of different things. But does everyone have their own specialization or how does this work?

A little bit. So we are a pretty small team. We very recently got up to six people. So when you have a small team and you're trying to put out these stories and things like that, it. It really helps us to have people that can be generalists and do all the parts. But yeah, we do have things that we are better at. So, like when I first started at the pudding, I was definitely data analysis was my forte and I was still learning D3 and I was making a bunch of mistakes and stuff failed, like all the time. But I was able to, like, lean into my ability to do data analysis. And that's the case for some of my other colleagues. Some of them are great with design and we're constantly tapping them for design input, and some are great with the programming side. So we're constantly tapping them for help with fixing bugs and things like that. So we do all have, like, our strengths. And so we have people who are like the go to resource if you have an issue with, you know, X, Y or z. But we all have the ability to do all of the pieces if we need to.

I'm sure we have a bunch of listeners out there who would love to get involved in this type of work. So what skills are needed to be successful at data journalism, visual storytelling and so on?

You know, there's a lot of things so it sounds like I'm, you know, listing off a ton of technologies that you need to know, and I don't know that that's always the case. Our team uses a variety of tools for analysis. Like I said, I use R, one of my colleagues uses Python, one uses SQL, JavaScript. And so we use all sorts of stuff for analysis. So it really, the tool itself doesn't make as big of a difference.

And they may change in the future, right?

Totally, totally. So that's actually what I was going to say is one of the best things to have in this field is the ability to adapt and learn new things as you go, whether that's a new language or a new statistical test. Because we're not focused on the same topic every time, we're constantly having to learn new methods of analysis. So we're just trying to stay right on the edge of things as much as possible. So the ability to just learn new things and be ready to. To do that stuff as you go is really important. And I think on a similar note, having kind of a beginner's mindset is also really helpful. And basically what I mean by that is just our stories are aimed at the general public. And so when you're writing a story, you kind of become a mini expert on a topic and it's really easy to slip into, like giving a bunch of jargon or making things not quite accessible. And if you try to approach things as a beginner would, it really helps you to communicate things in an easy way and in a way that other people who are actually beginners can appreciate and can understand. So I think being able to think about things like a beginner would is really helpful as well. So if you are a beginner, you're in good company because you have an exact beginner's mindset.

I think that's really cool. And it seems to me that a lot of the topics you've worked on, you may have been a beginner at the start in some sense. I mean, I don't know how much you knew about the atmosphere of Mars beforehand, but I'm sure you learned a lot along the way, which allowed you, in that case, to have that beginner's mindset from the start, essentially.

Exactly. Yeah. I didn't really know anything about the atmosphere of Mars. I knew very little. I knew it was cold, but I didn't know how cold. So, yeah, so I think, like, you have to be willing to be a beginner all the time. So that's hard sometimes. But the more you practice, it I think the easier it gets to just. You're always going to kind of feel like you don't know anything, but that's okay because you can learn it and then you'll know the things. There's two other things I think are really important. One is the ability to give and receive feedback. So I mentioned that my team works a lot. You know, whether we're working collaboratively or if I'm working on a story on my own, I still get a lot of feedback from my team. And we have like designated sessions where we just give feedback to one another. And that also happens a lot online. People post, you know, things that they're working on on Twitter. And I've had pretty good experiences with people giving feedback on the Internet. But of course, you know, sometimes people on the Internet aren't really very nice about things. So finding a way to give feedback to people that's constructive and can help them become better and then also being able to receive that feedback without feeling like they are criticizing you as a person is really important. And that's something that I am still working on all the time. You know, I get feedback and I'm like, oh no, I worked so hard on that. And then I'm like, oh wait, no, they're not criticizing me, they're trying to make me better. So that kind of mind flip and that ability to work on that is really important, I think, for sure. And the last thing is kind of like tied into all of this is just like communication is so important. And I think that's the case for so many fields, especially in the data world, because you're often not working in a silo, you're working in collaboration with other teams and sometimes with clients and sometimes with the general public. And so being able to communicate what you have done and what you're planning on doing is really, really important.

I agree completely. And I think communication is something that we all know is incredibly important, but you don't necessarily see it in the job listings. You see, you know, like know how to use Hadoop or something like that.

Right.

Communication isn't something that's necessarily stressed in terms of, you know, working data scientists having to communicate with managers or non technical stakeholders and this type of stuff. So it is really key to keep on reiterating this again and again.

Yeah, I think it, it really should be on more like job listings and stuff. But I think it's also fair to put that on a resume if you're applying for jobs because I think everybody kind of realizes that it is important, but I don't know why it doesn't end up on job listings.

We haven't spoken too much about specific techniques and methodologies. We have a bit. But I'm just wondering what one of your favorite data sciencey techniques or methodologies is.

So I don't know that I have a favorite. And part of that just comes down to I'm really bad at picking favorites of anything. I'm very indecisive when it comes to stuff like that. But I think it's also because we're constantly using different methods of analysis. So I don't know that I have a favorite technique. But one thing that I do a lot that I absolutely love doing. And so I'm going to take a moment to preach it here because I think it makes sense within the R environment. I use R markdown a lot, which is very similar to like Jupyter notebooks or things if you're a Python user. And basically the way that I use them is I include my analysis, but in between like chunks of code, I also include notes about why I did something or where the information came from. Or sometimes it's like this was the stack overflow post that helped me figure this out. And the reason I love doing that is because it helped me a lot when I was learning how to do things so that I could go back and figure out what I was thinking and what my thought process was. But also sometimes the pudding has sort of a client branch called Polygraph and so we work with clients a lot and those sort of markdown documents are really great to send to clients of. Like, here's an analysis of your data and here's like how we worked through it and they can read your, you know, inner thoughts and, and see what you've found before you make it all like pretty and on the Internet. So I found that that's been really fantastic. And so I always make those documents and I have never once regretted it. So I think that's my favorite data sciencey thing that I do.

Fantastic. And I think once again, that speaks to a certain form of like comprehensive communication as well. Developing this document so that other people have access to it and you have access to it downstream as well.

Honestly, like, I put a lot of those when I was still learning. I had made a portfolio online and, you know, I had put a lot of those things into this portfolio and everybody was like, oh, that's so great that like you make this for other people. And I was like, yeah, totally. And I do make it for other people. But I also, like you said, totally make it for myself because sometimes I forget like why I did something or and I've gone back to stuff like a year out and been like, why did I do that? And I've been really glad that I wrote it down.

Yeah. I mean, I say this probably too often for my listeners on and on the podcast, but, you know, when I talk about commenting, my code, it's for other people. But the most important other person is me in three weeks.

Yep. Yep. And you always think you're going to remember what you were doing, but I don't know. In my experience, I never do.

No future me, hates, hates me now. I can tell you that. Great. So, Amber, I'm wondering, as a final, final question, do you have a call to action for our listeners out there?

Yeah. So I think for anybody who's looking to start, start in this field or if you're trying to figure out if data journalism is something that you might enjoy, my biggest advice is just to get started. So start with a smallish question that you're really excited about and start exploring it. And don't be discouraged if you have to go and collect some data on your own. Again, keep it to a reasonable size so that you don't get completely overwhelmed. But I found that starting with something you're excited about and something you're personally invested in helps you really follow a project all the way through. And you can learn so many fun things along the way. So feel free to like go on the Internet and look up how to do stuff and ask people there for, for advice and feedback because I think there's a lot of really strong data science and data viz community members out there who are really excited to help beginners. And if you have a story idea that you think would be great for the pudding, we are always looking for freelancers. So if you've got something that is a proof of concept with analysis and a little bit of a general story, send it our way. We'd love to hear it.

And I'll just reiterate that all of Amber's work, we're going to put in the show notes and check out everything on the pudding. It's all fantastic. Start with a small question and send through some ideas to them if you're interested. Amber, it's been an absolute pleasure having you on the show.

Yeah, thanks so much. This was so great. I appreciate being here.

Thanks for joining our conversation with Amber about data journalism, interactive visualization and storytelling. Through a series of case studies, we saw how data journalism can leverage interactive data visualization to allow the reader to make the story theirs, to speak directly to them, and to give them agency in the process. This is key when data sets are so rich in information and different readers will necessarily be asking and interested in different questions. We also saw that although a lot of skills go into making this work, the most important skills are the ability to ask questions, learn new skills, collaborate and communicate. And if you have an idea for the pudding, make sure to reach out. Also, make sure to check out our next episode, a conversation with Eve Hilpish, Founder and Managing Partner of the Python Quants, a group focusing on the use of open source technologies for financial data science, artificial intelligence, algorithmic trading and computational finance. We'll be talking about data science and how it is disrupting finance. Why are banks such as bank of America and JP Morgan adopting the open source data science ecosystem? What are the major subdisciplines of finance that data science is and can have a large impact in? How has the rise of data science changed the financial world and how the work is done and thought about? These are great questions. I should know. I ask them. Join us next week for the answers. Don't forget that. For those interested, we've also got a special offer this week for Data Framed listeners. The opportunity to try datacamp yourself. All you need to do is email salesatacamp.com that's salesatacamp.com with the subject line podcast and redeem your free two week trial. I'm your host Hugo Bown Anderson. You can follow me on Twitter at Hugo Bowne and Data Camp at datacamp. You can find all our episodes and show notes@datacamp.com Community podcast. Sam.