#58 Critical Thinking in Data Science (with Debbie Berebichez)

This week I'll be speaking with Dr. Debbie Barebaches about the importance of critical thinking in data science. Debbie is a physicist, TV host and data scientist and is currently the Chief Data Scientist at Metis in New York City. In a world and a professional space plagued by buzz terms like AI, big data, deep learning and neural networks, conversations around skill sets, and less than productive programming language wars, what has happened to critical thinking in data science and data thinking in general? Now, what type of critical thinking skills are even necessary as data science, AI and machine learning become even more present in all of our lives? And how spread do they need to be across organizations and society? Stick around to find out. Welcome to Data Framed, the weekly datacamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow DataCamp on Twitter @DataCamp and me data at Hugo Bown. You can find all our episodes and show notes@data camp.com community podcast. This is Data Frame. I also wanted to let you know that in the show notes you can find a form to make guest suggestions for the podcast. We welcome all suggestions and encourage you to have as panoramic a view as possible with respect to the diversity and inclusivity of our guests. Also, if you enjoy the podcast, it would be great if you could leave us a rating and a review on itunes, but only if you enjoy it or read out some of our favorite reviews on the podcast. So write one for your chance to be quoted. Now it's time for the show. Hi there Debbie, and welcome to Data Framed.

Hi Hugo, It's a pleasure of mine to be here.

It is such a pleasure to have you on the show and I'm really excited to be here today to talk in particular about critical thinking in data science and what that actually means and as we know, to get critical about critical thinking and to see what aspects of data science in the space, what ways we are being critical, where we can actually improve aspects of critical thinking, particularly with respect to data thinking in general. By before we get into that, I'd love to know a bit about you, so could you start off by telling us what you're known for in the data community?

Sure. Thank you. Well, I'm not sure I'm that well known in the data science community, but if I am, I would say it's because I'm a big promoter of both critical thinking and of getting minorities such as women, and especially Hispanic women to enter the fields of stem, including data science. And I've promoted and have started a bunch of initiatives geared towards getting more women to get into science, technology and engineering. And the second reason could be because I co host a TV show for the Discovery Channel called Outrageous Acts of Science. So I know that a lot of people know me from there.

Right. And you also are at Metis, aren't you? The Data Science bootcamp.

Absolutely. I was gonna say that next. So I'm the chief data scientist at Metis. Metis is a science training company that is part of Kaplan, the large education company. And we basically have two modes of teaching data science. One is through boot camps, which we host in person at four locations in New York, Chicago, San Francisco and Seattle. And the second mode of teaching is through corporate training and other products. So we teach live online Intro to Data Science as a pre boot camp course, but we also customize various courses for corporations that need either visualization courses or Python programming or big data techniques and whatnot. And we've had quite a bit of success with that.

That's great. And I really look forward later on to talking about kind of the relationship to your work at Metis boot camps in general, how they can prepare people for a job market where in the job market in some respects, coding skills are at the forefront and not critical thinking skills and how to deal with that trade off in the education space, which is something we think a lot about.

Absolutely.

On top of that, though, you mentioned you're a big promoter of women, in particular Hispanic women in the space. And correct me if I'm wrong, I may mess this up completely, but you were the first Mexican woman to get a PhD from Stanford in physics.

Wow. You didn't get it wrong. So I got that right. Yes.

Fantastic.

That's right. And I think it's an important statistic, not so much to brag about it, but to show that examples like mine of persevering and working really hard and making your dream come true exist out there. And they're so important to talk about because they really serve as inspiration for people who sometimes think that their particular minority group or so is not suited for a career in data science or stem.

And so is this how you got interested in data science and computation, initially through a physics PhD?

Yeah.

Yeah.

I have kind of, I guess, not so atypical background for data science. I did my PhD in physics at Stanford, like you said, and I did theoretical physics. I did a lot of computational work the last two years. And so I learned about models and programming and working with data. Then I moved to New York to do two postdocs at Columbia University and at the Courant Institute of part of nyu, after which I decided, like a lot of physicists, to work in Wall street for a few years as what is sometimes derogatorily called a quant. I was involved in creating risk models and I did a lot of data analysis. And that's when I realized that my skills in math and programming had other alternative ways of being applied, not just in physics. So then after Wall Street, I thought that that was not the field for me because I didn care about just making money, even though making money is nice. But I had bigger aspirations and I wanted to do data and ethics and help the world and, and change the world in many ways. And so I heard about this new field, sort of new field for me at least, called Data Science about 10 years ago, and I took a course. It was kind of like a boot camp. I had the skills, but I didn't know how to translate them into the different techniques and algorithm that are typical of data science. And so after taking that, I jumped ships and I started my career in data science.

Awesome. That's a really interesting trajectory. And I just want to step back a bit and, you know, if you don't want to talk about this, we don't have to. But I'm just wondering, coming from where you were in Mexico, did you have kind of a social, cultural, and even familial or parental support to go down this path?

No, I didn't. And that is precisely why I care so much about inspiring and helping other young women who, like myself, feel attracted to a career in science or engineering, but who, for some reason, whether it be financial or social, feel that they cannot achieve their dreams. From a very young age, growing up in Mexico City, I was discouraged from pursuing a career in physics and math because I was a girl. And I was told from by friends and parents and teachers in school that I better pick something more feminine and that to do physics I had to practically be a genius, which I knew I wasn't. And so they really discouraged me so much that I became insecure about my math skills and about my ability to conquer and study the field. And so years later, when it came to go to university, I picked philosophy as an undergrad because I thought that that was something similar to physics. It had a lot of questions and you could use your imagination to ask yourself, why are we here? And you know, all kinds of things that had to do with objects that surround us and their meaning and whatnot. And. But I realized, Hugo that the more I try to hide my love for physics and math, the more that this inner voice telling me to go for it and to study it was screaming at me. And until two years into the bachelor's program in Mexico, I decided behind everyone's back to apply to schools in the US As a transfer. And it was difficult because in Mexico we were paying an eighth of what universities cost in the US and especially as a foreign student, it's very hard to find scholarships and financial help. But I was extremely lucky that I got full scholarship offered to me by Brandeis University in Massachusetts. And so in the middle of my BA in philosophy, I transferred to Brandeis in the winter. I hadn't seen the snow before, and I picked up philosophy courses. But right in my first semester, I had the courage to take my first physics intro to Physics class. It was a very large classroom with 100 students, and the class was Astronomy 101. And in that class, I realized that my passion and my love for physics was not going to go away. And I befriended the teaching assistant in the classroom who was a graduate student by the name of Rupesh, who came from India, and he came from Darjeeling Town in the Himalayas. And Rupesh and I became friends, and we would meet all the time. And he was the first person who truly believed in me. And he told me that I wasn't the typical student that just wanted to get an A in the homework, that my questions were just so curious and I was so inquisitive, and that I really, really cared about knowing about the planets and quantum mechanics and statistical mechanics and all kinds of things, and so really encouraged me to try to do physics until one day we were walking in Harvard Square in Cambridge, and we sat under a tree and I looked at Rupesh with tears in my eyes and I said, rupesh, I just don't want to die without trying. I don't want to die without trying to do physics. He got up and we didn't have cell phones at the time, but he called his advisor who was the head of the physics department at Brandeis, Dr. Wardle, who was the professor in my astronomy class. And he said, I have a student here who has a scholarship for only two years because she's a transfer student. And I know the BA in physics takes normally four years to complete, but she's really, really passionate. What can we do about it? Dr. Wardle called me into his office and. And we had a conversation. And he basically told me me and Rupesh, who was there with me. He said, believe it or not, there's somebody else who's done this in the past at Brandeis. His name is Ed Witten. He is. Well, I know for those people who know physics and know who he is, he's basically the father of string theory. So he definitely qualifies as a genius. And so I thought he was pulling my leg, like, okay, Ed Witten, like, there's no way I could achieve this. But he said, Ed switched at Brandeis from history to physics, and he did it in only two years because I couldn't ask my family to pay for another extra two years to stay there. And so what Dr. Wardle offered is he gave me a book called Div. Grad and Curl, which is Vector Calculus in Three dimensions. Basically, he said to me, if by the end of the summer you're able to master this material. And Hugo, I didn't even remember algebra at this point.

And of course, there's a whole bunch of linear algebra which goes into this vector calculus. Right.

Of course, there's so much background you have to know to even get into studying this book. So he said, if in two months, because this was in the month of May, you're able to master this material, we'll give you a test and we'll let you skip through the first two years of the physics major, so you can basically finish the whole BA in only two years. So Rupesh looked at me and he said, we're gonna do this. And he decided, incredibly, to devote his entire summer from mid June to end of August, to teaching me and mentoring me and basically covering all the subjects that I needed to master in order to enter the third year of physics in September. And it was amazing because I was so incredibly hardworking and passionate that I didn't move from my desk every day. Rupesh taught me from 9 in the morning till 9pm, and we did. We didn't have much time. So it was just practical in knowing how to solve derivatives. On Saturday, Sunday, we'll do integrals, Monday, first three chapters of Classical Mechanics, and. You get the idea. So at the end of the summer, I presented the test and I passed. And I tried to not burn too many capacitors in my first electronics lab at the time. And I remember how incredibly grateful I was to Rupesh, this person that absolutely changed the course of my life. And I tell this story every time I have an opportunity because it's incredible to me what Rupesh told me. I basically always wanted to pay him for all that he dedicated to me and all the effort he put into tutoring me. And he said to me that when he was growing up in India, in Darjeeling, there was an old man who used to climb up to his little town in this mountainous terrain and used to teach him and his sisters the tabla, the musical instrument, math and English. And every time the family wanted to compensate this old man, he said, no, the only way you could ever pay me back was is if you do this with someone else in the world. That beautiful story is how my mission in life began. And Rupesh passed the torch of knowledge to me to inspire, help and encourage other minorities who, like myself, dream of becoming scientists or engineers, but who for some reason lack the confidence or the skills at the time. And that has really informed my career. It has been the passion that connects everything that I've done. And I'm incredibly grateful to that pay it forward story. So after graduating with highest honors from Brandeis is when I went to Stanford and I reconnected with Rupesh only about seven years after that because he had gone to the South Pole to be a sub millimeter astronomer. And we connected and he was incredibly proud that I managed to graduate and do my research with a Nobel Prize winner at Stanford. And it was a great story.

Firstly, Debbie, thank you so much for sharing that beautiful story. Secondly, I wish I had a box of tissues with me right now. And thirdly, I feel like I was sitting there under that tree with you and Rapesh, solving all the vector calculus challenges. And I want to give Ripesh a big hug and a bunch of cash right now as well. But of course I'll do exactly what I'm trying to do and what we need to be doing, which is paying it forward. And I think that's actually provides a great segue into talking about critical thinking and data science, how we think about critical thinking as educators, being critical of critical thinking. And maybe I want to frame this conversation by saying there's just a lot of talk around the skills, aspiring data scientists, data analysts, data fluent, data literate people need to know. And sometimes to me, anyway, the conversation around this seems to be a little bit superficial. And I was wondering, firstly, if that's the case for you and secondly, if it is what seems superficial about it.

Yes, I'm so glad you're asking this question, Hugo. I can't tell you how many times I have visited programs where I've been a mentor for high school students. And I'll give you one example, one of these afternoon programs Was receiving quite a bit of funding. And there were three groups of young girls from high school working in data science, and they had been taught SQL, so they were masters at it, much more than I was ever proficient at their age. So I was like, wow, these girls are really impressive. There were three groups. They were working at a museum, and so one of them was working with a data set that was about birds in the museum, and they were trying to find patterns by looking at their demographics of the birds and their flying patterns and all this kind of information. Another group was looking at astronomical objects, and a third group was working with turtles because the museum had a whole bunch of turtles that they in an exhibit. So I went to the third group that was working with turtles, and I looked at the data that they were working with, and one of the columns said, weight. So the weight of the turtles. And so I said, oh, wow. So just out of curiosity, how big are the turtles that you're working with? Have you ever seen them? And they said, oh, yeah, we have. They're like, about the size of the palm of my hand. I said, oh, cute. I'd love to see those turtles, you know? And I said, okay, so is the weight here that you have in the column? You don't have any units for it because you just have the number. And the numbers are around 150 and 200 and 300. So is this weight in pounds? Is it in kilograms? You know, what is this weight in? What are the units? All of a sudden, these six girls in the group got all quiet, and none of them ventured to answer until one of them raised her hand and said, oh, I think it's in pounds. And I said, oh, wow. Let's see. I'm about 5 foot 3, and I weigh probably about 120 pounds. So this is interesting, because a turtle that's the size of my hand, basically, you're telling me it weighs double the amount of pounds that I do. Does that make sense? And then they all laughed and said, oh, yeah, you're right, it doesn't make sense. And we had this very nice conversation, and we went back and forth. It turns out after an hour, we finally found a teacher who knew and for certain gave us the information that the weight was actually in grams.

Wow.

And so the girls were surprised. And that story really caught my attention because I had been visiting a lot of schools and programs that are trying to teach coding in a very kind of fast and superficial way, just to be able to say our students know how to code. And I realized that but in an effort to get more and more people to know the skills for data science and for data analysis, in a world that's going way too fast, where we need to prepare our students for jobs in AI and machine learning and whatnot, we are forgetting what all of this is for. Coding and analyzing data has a purpose. It's not an end in itself. The purpose is to be able to solve problems and to have insights about what the data is telling us. And if we're not taught to ask the right questions and to think critically about where the data comes from, why is it being used or collected in a certain way? What other data could help or hurt my data set? What biases are being introduced by these data set? If we're not teaching our kids to think what's behind these techniques, then we're basically failing because we're just making them like robots who can only perform a simple task if and only if the next data set they see is similar in scope and structure to the one that they're learning to work with. It was a very moving and in a way also painful experience to see because I realized how needed are those critical skills, and not only in the education at the high school level, but how many projects haven't we seen at companies, at very large companies, in advanced data science groups where there is a significant bias being introduced because no one bothered to include a certain minority but important group in the statistical sample, or bias was introduced because people didn't bother to check what some outliers in the data set were describing, et cetera. And so I'm very, very passionate about teaching the critical thinking skills that are behind our why for why we do data science.

We'll jump right back into our interview with Debbie after a short segment. Now it's time for a segment called Data Science Tools for Getting Stuff Done and Giving it to the World. So I'm here with Jared Lander, who's chief data scientist at Lander Analytics. He's also an adjunct professor at Columbia University's business school. He organizes the world's largest R meetup as well as New York and Washington, DC's R conferences. And he's also acclaimed author of the book R for Everyone. Great to have you on the show again, Jared.

Thank you. It's always great hanging out with you.

It really is. So we're here today once again to talk about data science tools for getting stuff done and giving it to the world. And we're actually talking about two complementary technologies today and tools called plumber and docker. So I thought maybe you could start by telling us about Docker, what it is and what problems it solves for working data scientists.

Docker is a tool that came out of more of the dev world, and you can think of it as a mini virtual machine. It's not exactly that, but it's a good analogy. It's a way of keeping different environments separate in a computing system. Typically, developers use it in a very different way than data scientists. For developers, they would build a system and then deploy it in Docker so it stays the same, whereas data scientists sometimes use it that way. They also use it to create a reproducible environment. They can have a Docker image for their analysis. They've done open it up a year later, and it's the same environment they were in. With the added benefit, they can then deploy in that environment without worrying about dependencies changing.

Great. So now we have an understanding of Docker. What's Plumber and what problems does it solve?

Plumber is a very R way of exposing your r functions as restful APIs. This allows any user who can do a curl request to take advantage of R scripts that you've written without having to know what R is, without having to have R installed or about even having to know what the RESTful API is calling. They communicate the RESTful API, R does the work, gives the answer back through the API, and everyone's happy.

Awesome. So why would you want to expose your R code as a RESTful API or a web API? Maybe you can give us an example example or two.

Let's say you built some sort of XGBOOST model, or a deep learning model, or a time series model, any type of advanced machine learning. And it works really great for you. And you want to give someone the ability to take their data, feed it into the model, and get a score or prediction back, or forecast. All different words for the same thing. Traditionally, they wanted to use the model you built in R. They'd have to load up your model in R, provide the data, get a score by exposing it as an API. They don't have to use R. And while that's great by itself in that it lets anybody use your tool, it's really great when you're sort of throwing it over the wall. You have developers building the front end. They rely on some prediction, some score, a forecast you made in the back end. They make their web app communicate through the API to your machine. That keeps the domain separate, keeps them from getting messy with each other.

Awesome. So is there anything cool about Plumber and Docker in combination that you'd like to mention?

Yeah, using these two together is really great. You can put them together with sometimes the help of Docker Compose and allows you to create a bunch of different Docker images for different parts of your API, which is very easy to maintain. Then each of those Docker containers exposes through Plumber, a separate API, and your user just has to hit one URL with different endpoints and they can get different parts of the work done automatically for them.

Yeah, that sounds incredibly useful. So this is all open source. I'm just wondering if there are any paid alternatives and what the differences are.

There are a number of different alternatives that you can get through Amazon or Azure, but the one I'm the biggest fan of is a company called Algorithmia, so much so that I even directed my company to become business partners with them. For Algorithmia, all you need is your R script and maybe a binary version of your model saved as an RDS or an R data file. They handle the rest. You provide the script through their web interface. They take care of all the Docker and the infrastructure, all the machine scaling, all done automatically. So it's a nice paid alternative to rolling your own of Docker and Plumber.

Awesome. Thanks, Jared, for that introduction to Docker and Plumber, and look forward to speaking with you again soon.

Likewise.

After that interlude, it's time to jump back back into our chat with Debbie. You've spoken to so many essential points there. One of the overarching one is critical thinking and what I like to think of data thinking or data understanding before even there's a movement to put data into models and throw models at data before even looking at, as you say, units or important features, or really getting to know your data, getting to understand it, and performing that type of exploratory data analysis. And a related point that underlied a lot of what you were discussing there is thinking about the data collection process. And if you're collecting data in a certain way, what are you leaving out? What are your instruments not picking up? Is your data censored for any of these reasons? Are you leaving out certain demographics because they don't use a particular part of your service?

Exactly. Exactly. And I think I see a lot of companies that don't really know what data science is about because it has become this buzzword and everyone's to be in it, but nobody really knows exactly what you can get out of it. And what's happening is a lot of companies investing significant dollar amounts in big Data and solving big problems. Because they have collected so much data, they just build a huge infrastructure and try to find insights, but without really knowing if. First of all, those insights are important for the company. Second of all, if they find them, would they be able to use them for something and enact policies or something that's actually going to be helpful for the goals of the company. And I always remind them this kind of simple example. I said, you know, one of my heroes in physics is Ti Brah, who was a very famous Danish astronomer. Basically, he was locked up in a tower in an island in Denmark, which I actually had the opportunity to visit last summer.

Oh, really?

Yes. And wow. He lived in the 1500s, an amazing man, but he. He also had a. Apparently he was a nobleman. He had an awful personality and he lost his nose in a duel and.

They say he replaced it with a golden bridge. I think. Yeah, okay, great.

And I think that has been discredited a bit. That's what they told me in the museum. But anyways, yeah, this very interesting character. But the amazing thing about him is that he looked at the sky without any telescope. He basically had created this sophisticated instrument, but in the 1500s, it took him years, and he created a catalog of only about a thousand stars. That's it. So that's a very, very small data sets by today's standards. But from only those thousand data points, I think it was like 1800 or so to be more accurate. But he helped the theories that were later created by Kepler and Copernicus and where the laws of planetary motion were derived. Basically Kepler used that, and then Newton, Isaac Newton used it as the basis of. For the law of gravity. So from those thousand data points came universal theories that we're still using today that are incredibly powerful and deep. And that is a good example to say that sometimes we can put a lot of investment into huge data sets. But when we're talking about data literacy, large data sets also have a lot of noise. And you have to start by teaching that the most important thing is the insight that you're going to derive from that data set and not its size.

I'd like to speak to this idea of the focus on big data and the fact that a lot of us are collecting as much data as possible, hoping that, you know, thinking that all the information we need will be contained in there, even before asking critical questions, which is very dangerous. But before that, I just want to say tangentially, Tycho Brahe and Kepler's story is so wild, I haven't looked into it in a while. But if I recall correctly, Kepler, you know, wanted to unlock the secrets of planetary motion and figure out what was happening. And he realized that Tycho had the data. And so this is a story of someone realizing someone else has this data. And he went to work with him in Tycho Brahe's, I think, final years. And Tycho didn't even give him all the data at that point. He was actually very secretive about the data he, he had. And even when Brahe died, Kepler had to, to struggle with Brahe's family in order to get the data. So there are all types of like data secrecy and data privacy issues at that point as well.

And also data ownership because what.

Exactly. That's what I meant. Yeah.

Most people know who Kepler was, but if you ask people about Tico Brahe, very few, you know, non science people know. And that's because a lot of the credit went to Kepler. And some people argue that, you know, the one that did all the meticulous observations and had theories about it was Tycho. And so he deserved more credit. And so, you know, it was kind of a crazy time and lots of fights about data were happening.

And of course this is, you know, we're talking about a decoupling or a separation of, let's say humans into the people who are fantastic at collecting data and the people who are fantastic at analyzing it as well. This is a division in a lot of ways.

Yeah, absolutely.

And so, but this focus on big data, the fact that even a lot of companies valuations are based around the fact that they have so much data and it must be useful in the future. Right. And this is incredibly dangerous for practitioners, but also for society.

Absolutely. I mean, we did have a tipping point in that we had the hope in the 70s of AI and changing the landscape of our society and it didn't quite deliver in its promise because we didn't have the capacity to analyze very, very large data sets like we do now. And there was a tipping point where now we are able to analyze these much, much larger data sets. I mean, I think we every day in the world we produce like 10 to the 18 bytes of data, like 3 exabytes of data, something like that we generate. So obviously these are enormous of scales, but what's important is not that we now have this capacity to analyze it, but are we really getting a significant marginal insight or are the insights that we're getting commensurate with the ones that we were getting when we didn't have such large data sets? And I think that question is still out there, like we haven't been able to answer because as you know, the real important applications of AI are, are still being created and worked on. A lot of the AI things that we see out there are still simplistic in that they don't use all of the incredible and deep capacities that AI has to solve problems. So dimensionality of the data matters. It matters a lot. And probably for certain problems is going to be hugely important. But my point is more about when you're educating people or when you're a company investing in certain technology, you have to be able to walk before you run. So start analyzing the smaller data sets, come up with strategies that are based more on critical thinking and the questions that you're trying to solve rather than the size of your data set and the size of the infrastructure that you've built.

Great. So I've got a thought experiment for you which may happen all the time, I have no idea. But a student, aspiring data scientist, data analyst, comes to you and says, I need to learn some data thinking skills, some critical thinking skills to work with data. What are the top three critical thinking skills that you think I should learn? Debbie?

Thanks for that question, Hugo. I think the first one is you have to be a skeptic about data. You have to always, just like when you read a scientific paper, you have to know who paid for this research? Was it the drug company that is sponsoring a paper that says their drug is the only and best drug in the world? Like, clearly I'm not going to trust that paper. So a health, this skepticism about the team that collected the data, what biases could have been introduced, where was this data taken, how was it collected, what things were left out, what variables would be important in the future, etc. All those questions I think are super important. So if you don't ask them before even doing exploratory data analysis, it means you're thinking about the data and your relationship with the data is going to be limited. It the second one and I, this one, I, I came up with it from another famous physicist, Richard Feynman, who said, the ability to not fool oneself is one of the hardest and most important skills one can acquire in life. Because it's, it's very easy. Sometimes we think, oh, it's, I wouldn't be fooled by anyone, not any marketing campaign and not any government is going to fool me. But we fool ourselves much more often than the people interpreting the data out there. So the ability to not fall in love with what we think our data should be telling us. That is what I call fooling yourself. That is super important. The third skill is connecting the code and the algorithms to the real world. Like my example with high school girls that were working with the data, to be working with a database for three months and forgetting that behind the data are actual turtles. In this example, that's a big mistake. The same way, you know, when Facebook is incredible at doing face recognition and analyzing relationships between groups and people. But if they're forgetting that behind those connections are real people with real lives and real consequences, then we're failing. You know, we need to really connect our analysis to the world out there.

I agree, and I just want to go through those again, because I'm sure our listeners are scribbling away trying to remember all of this. So the first one was a healthy skepticism about data. The second, the ability to not fool yourself. And the third, connecting the code and the real world and all the stakeholders that actually exist on the ground.

Correct. Thank you.

So I just want to build slightly on the ability not to fool yourself. I mean, all of these are incredibly important. But there's a paper called I hope I get this right. Many analysts. One data set that we've discussed once or twice on the podcast before, and it gives a whole bunch of statisticians and domain experts a data set, separates them into teams and gives them the same data set and asks. It's a data set of, I think, either yellow or red cards given to football players in football or soccer matches. And the question is, are these decisions to give cards? Is there some sort of ethnic bias or racial bias in these decisions? The fact is, what happened was 70% of the team said one thing, 30% said the other thing, either yes or no. And then when they got to see everyone else's results, nearly all the teams were even more sure of their own techniques and their own results. There are a lot of reasons for this, but one of the points is that people go in with a certain bias already. And if you have a bias going into a data set, you make all of these micro decisions as an analyst, which helps you get to the place that you already thought you were going.

Right?

It. Yeah. You reminded me, funnily, of a paper that I discussed. I don't even think you could consider it a scientific, sophisticated paper, but it was a paper done for the Astrology, not Astronomy, but Astrology association in India years ago, and I talked about it at a conference because they first decided the hypothesis is that through some charts of some kid Astrological charts that tell you certain characteristics about some kids. If these people that were there, the gurus and the chart readers and predictors were able to guess, I think they gave themselves a pretty low score. They said, if we're able to guess 60% of the outcomes. And I think the question was whether these students were intellectually gifted or just going to be average students in school based just on their astrological chart. And if we're able to get 60% of them right, then that means we are gurus and astrology is true. And we are able to predict this with very high confidence. That was their confidence level. And the funny thing is, even though they did slightly worse than a coin toss, that is they did like four, they got 49% of them right. Mean, you know, and anybody in their right mind would be able to say, well, clearly that they did even worse than chance, you know, top coin would have done better. But they themselves patted themselves in the back saying, you see, we got 49% right. We can do this. So it's a very funny paper and I encourage people to read it because it's so easy to fool ourselves.

Absolutely. And the best thing about doing worse than a coin toss is you could actually just switch all your decisions and do better. We'll jump right back into our chat with Debbie after a short segment. In this segment, we'll continue discussing stories behind the normal distribution with Justin Boyce. Hi, Justin.

Hi, Hugo.

So, when we left off last time, you told me about using De Moivre's discovery that led us to be able to approximate binomial distributions with normal ones. I was thinking, since the Poisson distribution is a special case of the binomial, can it too be approximated as normal?

Yes, it can. A Poisson distribution with rate lambda is approximately normally distributed with mean and variance lambda. Provided lambda is large, so many distributions.

Can be approximated by the nor distribution then.

That's right. Now, I told a story about a gamma distribution in the last season of the podcast. An example of a gamma distributed variable is the amount of time I'd have to watch baseball before seeing three no hitters. The story there is that the waiting time for the arrival of k successive Poisson processes is gamma distributed in the limit of large k. The gamma distribution is approximately normal. So, yes, the story of the normal distribution can match the story of all of these different distributions that can be approximated by it in certain limits.

So last time we got to the relationship between the binomial and normal through De Moivre's work. What came next.

Lots of mathematicians in the 18th century were working on problems that were dancing around the normal distribution. One of those main problems was the issue of how to report summaries of repeated measurements.

What we call aggregations, right?

Yes. Today it is almost second nature to use the average of the observations. But that was not always the case. Even before de Moivre's time, astronomers were starting to make very precise measurements of objects in the sky. They would measure the same position using different methods with different points of reference and get slightly different values. They did not know how to aggregate the results. There is an amusing example where Kepler reports what he calls the mean of measurement. But it is neither an average nor a median. To figure out what to report, we ultimately need to quantify errors. Fortunately, great thinkers in both mathematics and astronomy started thinking about this problem.

Interesting. So astronomy was important in driving statistical thinking forward?

Oh yes. Importantly, two of the greatest minds in history, Laplace and Gauss, were thinking about this. They were both interested in the movements of Jupiter and Saturn in particular. And in 1809 Gauss published his treatise, the Theory of Heavenly Bodies moving about the sun in conic sections. Tacked on to the end of this work was Gauss exposition of how errors and repeated measurements are distributed.

So I guess this is where he derived the normal distribution, since we also call it the Gaussian distribution.

Yes. Now he derived it in sort of a roundabout way though. He started with some desiderata and derived the result from that. First, the most probable error in measurement is zero. Second, the errors are symmetric about zero. And finally, he assumed that the most probable value of a single measurement repeated over and over is the arithmetic mean of the measurements. From these desiderata he showed that the errors are normally distributed.

Okay, okay, so Gauss was a genius, no doubt. But doesn't his argument put the cart before the proverbial horse?

Yes, Gauss argument was circular. Though I won't go into the details here as to exactly how it was circular. Importantly, Laplace saw the result and built upon a result he had derived prior to Gauss book on planetary motion. Laplace found that any sum or mean of measurements will be approximately normally distributed if there are many measurements.

Aha. The central limit theorem.

Yes, the caveats and conditions of the central limit theorem came later. I like to use the theorem itself as the story behind the normal distribution. So here it is. Any quantity that emerges as the sum of a large number of sub processes tends to be normally distributed, provided none of the sub processes is very broadly distributed.

So Justin, how does this work for astronomical measurements?

Well, I like to think of it like this. Each measurement is the result of many sub processes. In the astronomical case, you have all of the optical components, distortions due to the atmosphere, due to the experimenter, et cetera. The sum of all of these can give normally distributed errors.

Well, this sounds like it comes up all the time.

Indeed it does. And if you're willing, I'll come on for one more segment about the normal distribution and see where it comes up and where it is overused. Sound good?

Let's do it. But before you go, this history of the development of the normal distribution is really interesting. Justin, do you have any tips for our listeners if they want to, to learn more about the history?

Sure. I really like the books by Steven Stiegler. I learned most of what I shared in this podcast from his book called the History of Statistics.

Ah, Stiegler. He also wrote the Seven Pillars of Statistical Wisdom.

Yes, and that's also a great book. Now I also put a link in the show Notes to a paper by Saul Stahl about the history of the normal distribution if anybody wants a quicker introduction.

Thanks for that, Justin. See you soon.

You bet.

Time to get straight back into our chat with Debbie. So we've been talking about critical thinking at an individual and societal level. I'm wondering how you think about the needs for all these skills, critical thinking skills, how they should be spread through organizations. And what I mean by that is what type of critical thinking and data thinking skills will be needed and are needed for people who don't even work directly with data themselves, but in jobs impacted by data?

Yes, that's an excellent question. Because I think the more that our field of data science grows, the more that we get different dependencies in companies, different groups needing insights, or even having contact with the data. And not everybody's going to be a data scientist. We're going to have people just interpret visualizations that come from the data, others using APIs and having to interpret what the algorithms come out with and whatnot. So I think it's essential that we spread the critical thinking message across organizations. And it has to start early in school because the ability to ask the right questions in an industry setting is incredibly important. And I don't think we're putting enough emphasis in it. So I think everybody in an organization has to be trained about things such as data ethics. How is the data being collected? Are we using it for the right purpose? Data ownership, data privacy, data security, all kinds of issues that impact the manipulation of data. And so that's part of the critical thinking process.

Hopefully this aspect of understanding on the part of people in society and other working professionals who aren't data scientists will result in less burden on the data scientists. What I really mean by that is, well, there are a few ways to frame it. The first way, I think it was probably Nate Silver who said this. Any quotation? I don't know who it was. I'll just say it's Nate Silver generally. But it was probably Nate Silver who said something like when a data scientist gets something right, they're thought of as a God and when they get something wrong, they're thought of as that they made the worst mistake ever, as opposed to a job in which sometimes you get it right and sometimes you get it wrong. Another way to frame it is it is kind of viewed by people without data skills like, I have no idea how to deal with this. So this is what you're going to do and you have kind of you're a prophet or you're the holder of divine knowledge or the high priest of data science I like to call them. And whether this will actually help as people develop more data skills who aren't data scientists will actually help bridge this gap in a lot of ways. So how do you think about these types of issues and challenges when building data science curricula at Metis and elsewhere?

Yeah, it's very important for me to learn. I'm not an expert in the field of learning science, but it's very important to me to learn how to best best built curriculum that optimizes these critical thinking principles and questions that I'm talking about. And so it really depends on the curriculum. So for example, we built with a team with Kathy o', Neill, who I know you've interviewed before, who I love, and a group of other seven executive women with the funding from Moody's analytics and the help of Girls, Inc. We developed the first data science science curriculum for high school girls of underserved backgrounds. And we deployed it in New York in several high schools. And so I think it was just this amazing experience because we try to emphasize focusing on the topic and what the consequences were of every single step in the process, from data collecting to choosing the algorithm to knowing how to measure the accuracy, the recall, the precision, like everything that we were doing, where it comes from, how to choose the metric that was right for the problem at hand, etc. And so the intention was very conscious to be about how to get the most insight about the limitations and the successes of the challenge or the problem at hand. Whereas when I built curriculum for the Metis boot camp, currently in my position, I want the students to have a pretty broad set of tools with which they can cry really hard problems. And so I may not focus on getting every single clustering algorithm there is in the curriculum, but I will focus on how to analyze the results of the clustering algorithms that we will see and how to know if we're using the right algorithms for the problem at hand and how to be able to ask that question of our colleagues, of our communities, etc. Because we all have limitations to our knowledge.

Yeah. There are two things there I want to focus on. The first is, as you said at Metis, thinking about the actual problems and thinking about the question at hand before even getting coding, I think is incredibly important. And also educating people through questions that really pertain to them and are interesting to them. So students will ask me if I want to embark upon my first data science project, what would you suggest I do? Well, I say, well, what are you interested in? And if they have a fitness tracker, for example, I say, maybe you could analyze your own fitness data. If you're a foodie, scrape Yelp reviews of restaurants and work with that type of stuff. If you love movies, if you're a cinephile, the OMDB has a fantastic API.

And that's exactly what we do at Metis. We have our students in the bootcamp use their own data set and they create their own projects. So it's really cool. I encourage people to go to made@metis.com and it's a site where we have some of our greatest projects. And it's incredible because you see people that had very basic math and programming skills coming in, and in three months they're able to analyze contamination sources in the ocean or some healthcare related thing, or an app that helps you choose the best restaurant for crepes that evening and stuff. Like, it's really, really cool what you can do.

Yeah. And I'll build on that by saying I've been to several of Metis graduation presentations. What do you call them?

Career Day.

Yeah, they're incredible. And seeing all the learner students there present the work they've done is amazing. And I know that. For example, you know, I've had Emily Robinson on the podcast. I work with her now at Data Cam and she completed me. I think she went to Etsy straight from Metis. I could be wrong there.

Yes, we love Emily.

Yeah, incredible. So we're going to wrap up in a few minutes. We've talked about the state of play of critical thinking today, but I'd like to. It's a prediction problem. So what does the future of data science look like to you, Debbie?

To me, it's going to merge with the industry of IoT or the Internet of Things. That is, as we see the ubiquitous US sensors. That is sensors are simply everywhere from medical devices to buildings that are smart buildings, testing, you know, our comfort level to apps that measure our behavior.

I mean, you're right, we wear them and we carry them in our pockets.

Right, exactly. And just like, you know, the personal computer came to revolutionize the information technology field the same way IoT is going to revolutionize and we're going to see a new paradigm where we're going to collect substantial more amounts of data about ourselves, our behaviors, our connections. And so issues that have to do with data privacy, data ownership, security, analysis insights are going to become ever more important. So what I predict is that with more automation, we're going to have more needs to have people that are not necessarily the data scientists working with the data, but are working in the field to analyze the ethical consequences of it, to act as peer reviewing committees to see if there should be policies or regulations that should be enforced around certain applications, et cetera. So that's what I see for a future more and more need for sort of adjacent professions that help with the data analysis process.

Yeah, I think you're right in terms of defining it anyway, or describing it as emerging between Data Science and IoT and Automation. I can't quite remember. Did you give a talk on the Internet of Things at the NYR Jared's conference a few years ago?

Yes, I did. On the other. Yeah, yeah, okay, great.

Well, I love that talk and Jared puts all those talks up online, so I'll find a link for that and put that in the show notes as well, if anyone's interested. So I just wanted to get a bit technical. I'm wondering what one of your favorite data sciencey techniques or methodologies is just something you love to do.

I actually really, really love singular value, the composition svd. I've always loved linear algebra and just the thought of being able to reduce the dimensionality of a problem is so sexy to me. In physics we deal with it all the time and my first encounter with it was when I worked briefly with David Botstein who's this is many, many years ago at Stanford. He's one of the creators of Genentech, the bio biotechnogony and we were analyzing the data coming from DNA microarrays which Basically compare a sample of healthy DNA with a sample that came from a patient in order to conclude whether the patient had and had cancer and in the case of positive answer what type of breast cancer it was. So it was really, really interesting because obviously there are so many genes in our genome that the dimension of the problem was humongous. And so to apply SVD and be able to reduce it to the dimensions that were most important enabled them to come up with pretty customized drugs that I have heard because I have since stopped working in that topic, but I've heard are working quite well for different types of breast cancer. So the applications of SVD are incredible. And so I don't know, I just really like, like that conceptually and anything that has to do with that, even NLP and I don't know, just seeing what you can get by sacrificing a bit of information is just really interesting to me.

Well, I'm sold. I mean, you've motivated through linear algebra, which I also love, and then you gave some incredibly important examples of its use. And for those of you out there who know of pca, I definitely suggest you to check out SVD as well. Yeah, I've got one final question for do you have a final call to action for our listeners out there?

Yes, I do. I'll repeat, Hugo, what I said in my Grace Hopper celebration keynote speech a little over a year ago. Think deeply, be bold and help others.

I think that's fantastic, Debbie. And what we'll do is link to your Grace Hopper talk as well in the show notes because I think the way you explained in that talk all of these things, why it's important to think deeply, be bold and help others, which you've gone through this conversation as well. I think that talk can provide more context there also.

Wonderful. This has been such a an awesome conversation, Hugo. Thank you.

Thank you so much, Debbie. It's been an absolute pleasure having you on the show. Thanks for joining the conversation with Debbie about the absolute need for critical thinking skills in data science. As we saw, Debbie's top three critical thinking skills were as follows. The first one was to have a healthy skepticism about data. The second, the ability to not fool yourself. And the third, connecting the code and the real world and all the stakeholders that actually exist on the ground ground. Debbie illustrated this third one through the story of the young high school students who were super proficient in sequel but had forgotten that the data they were looking at was connected to the real world of turtles and their weights. As Debbie said, this is a lesson for all of us. And she said that, for example, and I quote forgetting that behind the data are actual turtles. In this example, that's a big mistake. The same way when Facebook is incredible at doing facial recognition and analyzing relationships between groups and people. But if they're forgetting that behind those connections are real people with real lives and real consequences, then we're failing. We need to really connect our analysis to the world out there. End quote. We also saw the power of data thinking in the limit of small data via the example of Tycho Brahe's astronomical data that he collected without a telescope, mind you, which Kepler used to build his laws of planetary motion and Newton used to build his theory of gravity. In an era where sheer amount of data is equated with power, it's important to remember that big data doesn't mean good data, and it definitely doesn't mean simply signal Next week I'll be speaking with Sean Law about data science research and development at TD Ameritrade. Sean's work on the exploration team uses cutting edge theories and tools to build proofs of concept. At td, they think about a wide array of questions from conversational agents that can help customers quickly get to information that they need. And going beyond chatbots, using modern time series analysis and more advanced techniques like recurrent neural networks to predict the next time a customer might call and what they might be calling for, as well as helping investors leverage alternative data sets and make more informed decisions. Now, what does this proof of concept work on the edge of data science look like at TD Ameritrade and how does it differ from building prototypes and products? How does exploration also differ from production? Join us next week to find out. I'm your host Hugo Bown Anderson. You can follow datacamp on Twitter @datacamp and me @hugobound. You can find all the our episodes and show notes@datacamp.com community podcast.