#18 Deep Learning at NVIDIA

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Michelle Gill, a deep learning expert at Nvidia, an artificial intelligence company that builds GPUs, the processes that everybody uses for deep learning. We'll talk specifically about the modern superpower of deep learning and where it has the largest impact, past, present and future. Filtered through the lens of Michelle's work at Nvidia. Where is deep learning most effective? Where is it not? Where should we channel our skepticism of the hype surrounding it? Stick around to find out. I'm Hugo Bown Anderson, a Data Scientist at DataCamp, and this is Dataframed. Welcome to dataframed, a weekly datacamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter hugobown and datacamp@datacamp. You can find all our episodes and show notes@datacamp.com Community Podcast hi Michelle, and welcome to Data Framed.

Thanks. It's great to be here.

So great to have you on the show and I'm really excited to have you on to talk about your work at Nvidia and the modern deep learning landscape in particular. But first I'd like to find out a bit about you. How did you get into data science originally?

So my path is a bit circuitous. I started my career as a scientist and I still am in many ways a scientist. I went to graduate school in biochemistry and I did a lot of programming then to analyze my data. A lot of times one reaches a limit of what you can do with things like spreadsheets. I continued the practice of writing code to analyze my data during my postdoc and, and during my postdoc, I think my love of coding really solidified. I switched from some proprietary tools to using Python and a little bit of R. And sort of historically, this was a really exciting time in the Python community. At least Anyway, for me, Numpy Scipy were kind of coming of age and things like Ipython were being developed, so it was a fun time to switch to Python. So that sort of became this other thing that I loved doing was tinkering with open source and incorporating it in my research whenever possible. And so there was this new thing on, this new sort of data science thing that I was vaguely aware of and I was kind of curious about it. So I sort of, I took the scientific approach, I guess is what someone said to me once. And that sort of. I agree with that. I sort of studied what data science was. I started following data scientists on Twitter and there was a couple PI data conferences in New York City that I went to just to learn more about the field and how it related and differed from what I was doing as a sort of a more traditional scientific researcher. And I continued on with that sort of study of data science. I incorporated some machine learning work and some software development work when I moved from my postdoc to the National Cancer Institute. And so I was able to really sort of dig deep and learn a bit more about machine learning. And I think, I guess I was slow to come to the decision, but after those couple of years I felt like I was ready to pursue this data science thing full time.

I can relate to a lot of that. I mean, my background was in academic research. It was actually during my postdoc as well that I switched to Python and R and it was going to meetups in New York City and conferences and PI data conferences and seeing the community there that really encouraged me to pursue data science as well.

Yeah, there's such a wonderful, wonderful community and you know, there's so many opportunities to get involved just a little bit. If you want to get started and help out with a software project or help out with a conference, get to know people, there's really cool opportunities. I think in data science I sort of formalize things. I have a very good friend of mine, Debris Barabichez, and she is the chief data scientist at Metis. So I joined the bootcamp as sort of this hybrid teaching assistant student because I had had a lot of machine learning experience, some machine learning experience already. And after that they hired me as a full time instructor. So that's sort of my origin story.

That's great. Medicine and Debbie are both fantastic. Debbie is a fantastic educator and what vitality she has, what a sense of life. I think.

Yeah, Debbie is one of my favorite people. She's just such an amazing. She's an amazing scientist, and she's an amazing human being.

So now you work for Nvidia. So. So I'd like you to tell us a bit about what Nvidia does.

Yes, yes. So Nvidia has sort of its own origin story, I guess, if you will. Nvidia began as a gaming company that made GPUs and was founded in 1993. These GPUs were used primarily for things like gaming.

And so just quickly. GPUs are graphics processing units.

Yes, thank you for. Yes, absolutely, great interjection. GPUs are graphics processing units. They're cards that you can stick in your computer and help improve things like video playback. But GPUs are useful for rendering video, like in games, because they can perform parallel calculations, massive amounts of parallel calculations. And Nvidia decided to open up this opportunity or this ability beyond just gaming and video resolution. And so they released something called cuda, which is a toolkit that makes this compute power accessible. So that meant that GPUs could be harnessed to do other types of calculations. And this CUDA was released on, I think 2007. And I remember, and I remember when it was incorporated into some software that I was using. I used Matlab a lot at the time. I remember because I started putting my simulations on the gpu. And it was simple things, but it made, it made such a difference. So the rest is kind of history, so to speak. CUDA fueled a lot of advancements in high performance computing. So my background is associated with molecular dynamics simulations, but also deep learning. So today Nvidia is a general purpose AI company. We have GPU and GPU related resources in the cloud for desktop, for visualization, for virtualization, all all kinds of things and services associated with that as well.

What an amazing story. I mean, just like your and mine, the trajectory seems to have shifted quite dramatically from being a gaming company to this realization that the chip sale building had many different purposes.

Yeah, yeah. And you know, in the early days of GPU of Cuda, it was a lot of scientists sort of tinkering around, but they were there at the right time when deep learning was also growing as a field and someone put two and two together that we should be doing some of these deep learning calculations on GPUs. And it was incredibly powerful.

I really want to know what you do at Nvidia, but before that I'd like to step back and could you tell me what deep learning is or how you would describe it?

Sure. I'm sure that you will get as Many definitions, as you do ask, as people that you ask for this, but I think of deep learning as sort of a subset of machine learning. And what's different from deep learning versus machine learning is that the models in deep learning are sort of inspired by the way the brain works. And I don't, I don't generally consider them as an imitation of the way the brain works, but I do think they're inspired by that. And so these models tend to be a little bit more complex, but like traditional machine learning models, they're good at learning patterns and recognizing trends. So that's sort of how I, how I just like to describe it.

Yeah, and I like that you added the, the nuance that although they're inspired by the way the brain works, that analogy can be stretched quite far in, in probably unproductive ways, particularly in the, in the news media, I think.

Yes, I would agree with that.

And that's something maybe we'll get to a bit later is societal perceptions of deep learning. And something we're going to talk about a bit later as well is where deep learning can have the biggest impact. But before we get there, maybe you can give us the rundown of what, what you do at Nvidia.

Sure, absolutely. So I said that Nvidia has all kinds of services around deep learning and artificial intelligence. So I'm a member of one of those teams. I'm in the professional services group at Nvidia. And to really sort of put it into a nutshell, we help companies get started with deep learning. You know, maybe these companies have some data science experience, but they don't have deep learning experience. And so what we do, there's a range of things that we do. We hold workshops with these clients where we talk about what's the current state of deep learning with regard to a current industry. So I'm working with a client right now that's in sort of industrial chemistry and material science. We talk about for that specific industry. Where is deep learning at? And what are business problems that can be approached with deep learning? So we talk to them about their problems and we sort of brainstorm with them where they could start. You know, sometimes this is about prioritizing projects and finding a good way for them to get started. The other end of the spectrum is, you know, if a company is interested in working with the professional services group, we can help them and mentor them through this process. So we can design, we can help them design and implement like a proof of concept pipeline and all the experimentation that goes around finding models and Refining model parameters, et cetera, setting up data, data preparation, the workflow, et cetera, we can help them really validate that approach. And then at the end we can say, okay, you have sort of a proof of concept first project. Now how does that map to your technology stack? So that's sort of the gamut of what we do in the professional services group.

Fantastic. And it sounds as though a large part of your job is customer, client, company facing. And I'm wondering, when you run these workshops or you're designing and implementing these proof of concept experiments, what are the type of people in the room and how technical are they from the companies that you're working with?

So when we're designing sort of experiments, they do, a lot of times tend to be more technical people. So maybe data science leads when we're really nailing down the details, but when we're brainstorming, it can be, it can be sort of C level individuals at a company. So if we're brainstorming ideas and talking big picture, like art of what's possible, those would be more executives. Those are obviously higher level conversations. But once we really get down to the nitty gritty, then we would probably be working with a data science team and sometimes with it as well, because there's a lot of stakeholders involved in sort of getting this process up and running.

Yeah. And I'm sure those executive level, higher level conversations are both incredibly fulfilling and also probably quite frustrating in terms of having to perform translation between, you know, the language that we all speak at a technical level and the language that you need to describe these type of stuff to these incredibly important stakeholders.

Yeah, I agree. I think technical mastery is an incredibly important skill as a data scientist, you know, but I think it's not the only skill out there. I think the other skill that is really important is, is learning your audience and learning how to communicate with your audience so that they understand the effect and the impact that data science can have across their business. But we also don't want them to leave thinking that data science is a Magic 8 ball that just gives you an answer. So it's always a fine line and sort of trying to show them what it can and can't do. And what do you need to get started?

In which industries and or disciplines do you see top deep learning having the biggest impact currently?

So I think this is maybe one of my favorite parts of my job is, you know, I come from science background, I come from biochemistry and I love thinking about where that field is going with regards to chemistry. But it's also really fun to think about other fields. And so there's opportunities all over the map. But sort of two that I think are really interesting right now. Number one is manufacturing. So. So that industry tends to be a little bit automated already. And so either it's the case that a lot of the necessary data for deep learning is already being collected. Things like images of products as it moves along a conveyor belt, something like that, if it's not already done, it's relatively straightforward to sometimes add that data collection. And there's all kinds of applications of deep learning and things like quality control, operations management, et cetera, that I think in the manufacturing area it can be incredibly impactful, especially for companies that are just getting started. The other one, and this is certainly biased by things that are near and dear to my heart, is healthcare. We have a lot of, there's a lot of buzz and excitement around imaging for patient diagnosis, analysis of EMR records, etc. And those are exciting applications and I think that they're starting to get some headway. But I'm of course sort of very interested in pharma deep learning. So understanding how we can apply deep learning on the molecular level, there's starting to be some really exciting progress. So I'm sort of active in reading those papers and trying to be part of that community. So those are big ones where I think it's really exciting.

They're both, both huge in the healthcare example and pharma. What, what type of gains are to be made? What type of questions can, can we answer with, with deep learning?

I think one of the, the early approaches that I see being successful in healthcare and pharma is it isn't going to be, you know, these applications are more sophisticated than sort of categorizing cats versus dogs. Right. I think what they will do early on is help scientists make better decisions. So if you are designing a drug, for example, and you know that some chemical structure of a drug has properties that you like, but maybe it has some properties that you don't, how do you find something similar that has the good properties but maybe not the bad properties? So we have, there's networks out there that are being developed and models that can help, that can be trained so that you have molecules assorting themselves by their features and so you can sample their neighbors, sort of. It's kind of crazy. Yeah, but so that's a really cool application and that's pretty, pretty recent. But I think that's an interesting way of applying deep learning. Again, it's not a Magic 8 ball. Instead it's, you know, if you have a needle in a haystack, it's taking away some of the hay and it helps you make better decisions about how you spend your time. As someone who's in R and D.

I think that's a wonderful analogy of the needle in the haystack. So essentially also, we're getting kind of a lot of gains in efficiency, right?

Right. It doesn't just give you the answer, it doesn't just spit out the answer, but I think it can help you make better choices about the type of experiments that you do. So you become more efficient.

Exactly. We'll jump right back into our interview with Michelle Gill after a short segment. Let's now dive into a segment called Statistical Pitfalls. I'm here with Michael Bettencourt, applied statistician and one of the core developers of the open source statistical modeling platform, Stan. Great to have you on the show, Mike.

Thanks for having me, Hugo. You know, the weirdest thing happened to me the other day.

What's that?

Well, I was at a party and who else did I see but an ex girlfriend that I hadn't seen in years?

Wow, that's quite the improbable coincidence, right?

It was so improbable that I nearly convinced myself that fate had destined us to meet again and that I should go over and declare my love for her. Alas, in reality, there was nothing particularly special about our meeting. And had I assigned any special meaning to those circumstances, I would only have been setting myself up for disappointment. I would have fallen, Hugo, for the fallacy of the post Hawk hypothesis.

This sounds like another statistical pitfall, Mike.

Indeed it is, Hugo. Now, a post hoc hypothesis is an explanation for an observation motivated by the observation itself. By tailoring a narrative to that particular observation at the expense of what else we could just as easily have observed, we assign special meaning to that observation. This then leads us to overstate the significance of that hypothesis, only to find that it doesn't generalize to other observations. This overconfidence leads to poor decisions and, as we all saw, coming regret. But postdoc hypotheses aren't just about social embarrassment. They're also an important factor in the replication crisis that has struck various scientific and industrial fields. Even ignoring deliberate attempts to achieve statistical significance, the subtle influence of observed data on a published hypothesis, for example, through exploratory data analysis or publication bias towards novel results, leads to the same kind of overfitting.

So is it possible to avoid these influences?

Well, in order to avoid a postdoc hypothesis, we have to be careful to model not the particular details of their observed data, but rather the measurement process itself. Let's consider, for example, coincidences. Now, the circumstances of any particular coincidence are typically ignorable. What we really want to model is our remarking on the coincidence in the first place. Is it improbable that I ran into that particular ex girlfriend at that particular party on that particular night? Definitely. Is it improbable that I ran into at least one ex and at least one party over the last few years and thought I should go redeclare my love? Well, not so much. When we model the observation of the coincidence itself, we are not led to place any special meaning on that one event, and hence we avoid much of the regretful overfitting.

And the same applies to data analysis?

Absolutely. Given how easy it is for the observed data to subtly influence our hypotheses in practice, we have to be extremely vigilant to identify and model the measurement process itself as opposed to any particular observation. In other words, we have to consider the post hoc hypotheses that we would have derived not just from the observed data, but also from similar data that we just as easily could have observed.

Mike, thank you so much for that introduction to the statistical pitfall of the post hoc hypothesis.

Anything to help our listeners avoid an awkward party.

Hugo, awesome. After that interlude, it's time to jump back into our chat with Michelle. So we've been talking about deep learning and where you're seeing it being impactful. We haven't really spoken to what deep learning is good at in the sense of, I mean, it's known to be good at images and text, for example. Right?

Yeah, no, I would say that those are certainly more well established applications. You know, imaging, image categorization, image segmentation. We are starting to know a fair bit about that in a general sense, in general applications. And we're also starting, we're becoming pretty good dealing with text and, you know, analysis of sentiment, translation. Those things are becoming or are already relatively well established. But thinking about how do we represent chemicals and three dimensional structures of enzymes and drugs? That is a whole new concept and that's why these fields are challenging. And that's one of the things that we help clients understand, like what's the state of the art? How do people create features for a deep learning model based on drugs, based on three dimensional structures? What does that even mean and what are the challenges associated with that? You know, I sort of talk about sometimes Talking about these models as, like Legos, but I don't know. Did you ever have Capsella when you were little? These little. There's these little toys that you stick them together and they have gears and things inside them.

No, I didn't, but maybe we didn't have them in Australia. I'm not sure. Capsella.

So Capsella. Yeah. You can build sort of robotic devices, but you stick. The point is that you stick pieces together, kind of like a Lego. So how do we slot in the type of featurization methods that we need for chemistry? And then, you know, we build on the pieces of a neural network. You know, I think about it like Legos, like, what are the parts that we swap in and out to get where we want? And there's. There's challenges with it, too, scientific data. There's just not enough of it. Scientists don't publish negative results, but these are the challenges. But these are also the really cool things about deep learning.

That is incredibly interesting because I haven't really thought about challenges or questions in which deep learning isn't the obvious place to go. How can we engineer features which may be appropriate for deep learning?

Right. And it might be the case in the future. We have these sort of primitives. We have a convolutional primitive and we have a recurrent primitive. What kind of primitives? What kind of building blocks will we have in the future for things like enzymes or chemicals, which are more like a molecular graph? You know, we're starting to have some of those developed. But what's the answer going to be? We don't really know.

Yeah. And the other thing you've just spoken to, there are really neural network or deep learning architectures and different types of architectures. So for those of you who don't know, you can build up these deep learning algorithms using what are called layers, inspired by layers of neurons. But you can build them up in a variety of different ways. Right. And you can get recurrent neural networks, as you say, convolutional neural networks, all of these type of stuff. And I think for beginners coming into deep learning, it seems like some sort of arcane wizardry, how people actually choose which networks to use for which applications. Right?

Yeah. And that's a great point, and I think it's good to bring it back. They all have features, so convolutions are good at picking up pixels, things that are nearby in space. So you can imagine why they might be good at picking out features of an image. You slide these little feature detectors around in the two dimensional plane of an image. And so once you sort of start to understand how these basics work, you can start to think about how you might apply them to more complex problems. But it takes a while. Like as you start to learn the field, these recurrent networks that we talked about, they pick up things, they look backwards or sometimes forwards in time. So, so when you have data that's sequential, they can help pick up those nuances. So understanding those sort of basics can help you start to think about other problems or how you might combine these blocks, these Legos together to understand or think about what you would use for a new problem.

And I think, you know, this type of sequential data time series is one common example and text is another. Right, because text is sequential. And these LSTMs, these long short term memory networks, which are a form of recurrent network, have been very good at doing a whole bunch of stuff with text in recent times.

Yep, absolutely. And again, they can look, look at, look to the left and the right in the, in the line of text and see who's nearby. And that's why they are, they're used to that.

Let me ask you this. As I said, it seems like some sort of wizardry, this choosing of architectures or being a deep learning expert. What I mean by that is not wizardry, of course, but there's a, as opposed to being a science, there's almost a heavily artistic quality associated with this. And do you see this type of choosing architectures becoming more and more scientific and more and more robust?

I think certainly things will become robust. We will start to learn more about how these architectures, these layers you set, as you alluded to of building blocks, how they behave. I think it'll become more robust and I think it will become a little more standardized with time. So with classifying images, we have a set of architectures that work pretty well. And I'm not saying there's not room for improvement, but we sort of know you can pick from a handful and that's a good place to start. Does that make sense?

That makes perfect sense. And I suppose also in the future of the machine learning landscape, we're actually going to see a lot more automated machine learning in the coming five, 10, 15 years. And perhaps this will play a role in helping to choose architectures also, for.

Sure, things are evolving really fast right now. You know, software libraries that are used for deep learning, software libraries that contain various architectures, it's evolving really, really quickly that will start to stabilize and I don't think it'll Stabilize all at once. I think some fields will stabilize more early on, and then, you know, later on, other fields will stabilize as well. But these things come of age, and I think that they will become more accessible, and I think we'll understand them and know more about them as we progress as a field.

Great. So we've talked a lot about the capabilities of deep learning, but as we all know, there's a lot of buzz around deep learning at the moment, and a lot of it is warranted. But I think we need to be skeptical with anything that elicits this type of buzz in society. And we do see headlines such as, AI creates copy of itself in a way that threatens humanity. I mean, I made that one up. But it's, you know, this type of stuff. So I'm just wondering where your skepticism comes into play or if you could speak to what isn't deep learning capable of or what deep learning isn't capable of?

Yeah, so it is a really powerful technique, but it isn't the human brain. It's good at making decisions when involving data that it's seen before. So I'm taking a simple example. But if you show a deep learning network, a lot of cats versus dogs images, it can probably learn to differentiate between cats versus dogs. But if you swap the context or the nature of the data, that can be challenging. So I heard a story about someone who was training a network to differentiate between wolves and huskies. And it turned out, unbeknownst to this person, that all of the husky pictures had snow in the background. And so if the wolf pictures would have like a forest or grass or something, and so if you swap the backgrounds out, you could fool the neural network because it was learning. It just learns things that are consistent with husky. And when you have snow in the background, it learns that snow is consistent with husky.

That's incredible.

You can fool them. I mean, sometimes there's missing data, so underrepresented population samples, and you may not know that, and that may prove to be important, or it may not. You don't, you don't always know a priori sometimes. So scientific data, a lot of times we don't have negative results. And that that means we're missing a whole class of data. So that makes it challenging. Sometimes the nature of the data just fundamentally changes with time. If you're doing some deep learning on consumers as generations change, you wouldn't expect that model to change, evolve, without being adapted on its own. It's also not good at reasoning about the unknown. So if you have like A if you're doing some deep learning on video and like say you're showing a baseball game and the picture is maybe off the. They're not shown on the screen, you only see the batter. We as humans, when we see the pitch coming in, we can infer, you know, instantly where the pitcher is because we understand that a pitcher is the person who throws the ball. Deep learning networks don't necessarily know that. So there's quirks with them. There's a whole field that's devoted to understanding and exploiting, but also understanding the way that neural networks can be exploited. It's sort of called this adversarial deep learning. So understanding that is really important too. If we need to rely on our networks, how easily can they be fooled and how can we fix that?

Those are some incredible examples. And I think you first spoke to first the idea that you need a lot of data. Like essentially you need thousands of samples. Right. For most, at least thousands of samples. For most questions.

For most general techniques, yeah, you need a lot of data and you need data that represents the range of possibilities that you want your network to understand. So there's cutting edge techniques that are where you need less data. But again, those are really still evolving. It's important. You need a lot of data and sometimes there's fields where we just don't have as much.

And you also. This Wolf Husky example is incredible because what it really speaks to is we need to know why our algorithms are picking what they pick, because sure, it can differentiate these, but if we don't know what's doing it because of the background and we think of it because it's the subject or the figure in the image, that's a huge problem.

Yeah. And it's. Some people say deep learning is a black box. I'm not sure that I think it's entirely a black box. So picking apart this Husky example, we can study the filters that are used in a deep learning model and we can gain some insight. It may not be 100% crystal clear, but we do have some ways to pick and prod at why this network is doing this. Once we discover an issue, as the networks get more exotic and are less well understood, that does get more challenging. But it's important and it's important to keep in mind as the field evolves.

And, and as you said, it's an active field of research.

Right? Right. For sure.

I'd like to move slightly to a different topic, which is the idea of storytelling in data science and deep learning and machine learning in An IBT interview, you said that data science scientists would be best serviced working on their storytelling skills. You also said the real trick to surfing this new wave of deep learning technologies is translating data into tangible insights. My question is what is the importance of storytelling in data science, particularly when moving from data to insights?

Yeah, that's a great question. And I want to clarify. Sometimes I sort of see the phrase storytelling in the context of data science. I sometimes see it taken to imply that maybe less technical rigor is used in the analytical process and in the modeling process. And that's not what I'm getting at at all. Data science and particularly deep learning are extremely technical in nature and they must be done in a thorough and rigorous way. And that's important. But a model is just once it's trained, it's useful, but it's just a model. How do you, how do you make it understandable to people who are stakeholders and decision makers and domain experts who maybe don't understand deep learning, but they want to use this model to help them in their daily lives with their business, et cetera. And I think being able to sort of translate what this model can do, but also what it can't, what are limitations, what are caveats? Helping them understand that and use that model, making it accessible to them, I think is what I'm getting at. And so how do you help them understand or envision how it's going to make their life better? Sometimes when people don't know much about deep learning companies that we work with, they don't even necessarily know how it's going to make their life better or make their business run better. So really putting that into context is what I'm getting at. And that's why I think it's, it is as important as, you know, being a technical master of your, of your various techniques.

I couldn't agree more. And once again, this speaks back to our part of our conversation earlier when we were discussing who are the people in the room when you're doing your job right, who are the stakeholders? And they vary in level of technical expertise and domain knowledge widely.

Yeah. And it's also important to know how to communicate with them. You know, when you're scoping out a project, Data science isn't done in a bubble. We need to understand what are the requirements, you know, how do I work with, if I'm working with an advertising team, what are the metrics that they're using, what metrics drive their decisions and how can I make that metric be, how can I set up A model that captures and produces that metric, that's, that's its own sort of set of challenges. And likewise, I would say you want to make sure you're not overestimating or overfitting your model to things that we just don't have that kind of precision for in the business world. Data science is not Kaggle. I'll probably get some emails about that, but it's not about getting 0.01% improvements in accuracy. Sometimes when you have huge variability in your operations management, for example, just understanding where are the levers that are important to really push on is part of our jobs as data scientists.

Yeah. And understanding the constraints of the question, the financial constraints, the amount of time you have to implement a project, the technical constraints.

So where is this model going to be deployed? Is it going to be deployed in a computing center where I have tons of resources and maybe I don't need to spend a lot of time trimming down the architecture? Or is this model going to be deployed on a drone that runs on a battery with limited compute power? And so this model needs to be able to infer on something equivalent to a Raspberry PI or, you know, an Nvidia Jetson, something small. If that's the case, then I need to rethink what I'm doing because my model just simply won't work on a inside something as small as a drone.

We know that certain parts of data science and deep learning have been and will be automated in the future. What aspects of data science and deep learning, well, deep learning in particular do you think won't be automated?

So I think with deep learning, a lot of the near term applications, I think there will be a human in the loop, but the deep learning aspect will be helping that human make better decisions, prioritize where he or she spends their time. So taking for example, medical diagnostics, if we have deep learning involved, maybe the diagnosis is, or maybe the deep learning model is doing some diagnosis on and making calls for situations where it's very clear cut. And maybe the human intervention, or maybe the human is told, you should spend more time on this one to make a final decision. Because I'm not, because the model isn't sure. So I think that's how we can help people do their jobs better and improve the way businesses run. So I think that's like the near term aspect. And I think deep learning models just don't quite have the sophistication to make all the nuanced decisions that we do as humans. Yet they can do simple things pretty well.

So what that means for working data scientists in deep learning is almost acting as translators and interfaces between deep learning architectures and the real world and other stakeholders.

Right? Right. The stakeholders need to know about the caveats and where a human needs to come in. What can this model do that the data science team built or that the data science team and the professional services group at Nvidia designed? What can it do, what can't it do and what should they be aware of? These are the things we have to try to translate.

Right. So to your mind, what are the biggest challenges facing deep learning currently as a field?

My inner curmudgeon sometimes says hype. I don't like to see a field sold as something that it is not yet. It is not or is not yet. But then that leads to disappointment. So I think it holds a lot of promise. But I think that we have to keep our expectations realistic and understand that these things are progressions. The other challenge I think are what challenges? Availability of talent. So you've heard these stories or maybe you haven't, but industry hiring away tons and tons of researchers from academia who have this background. I totally understand that. But we also need, we need to be able to train the next generation of deep learning practitioners. It becomes a little bit like it further constrains the supply. I would say another big challenge is moving away from people and hype for a minute is I think there's something around making sure that in situations where a model needs to be interpretable that it is in fact interpretable and dealing with situations where we have underrepresented groups or categories in our data and how do we make that better so that these models have more broad applicability?

We'll jump right back into our interview with Michelle after a short segment. Let's now jump into a segment called Blog Post of the Week with Spencer Boucher. Hey there, Spence.

Hey, what's up Hugo? Today let's highlight a Katie Nuggets post by Matthew Mayo about automated machine learning or just Automl for short. I think that this post is a great introduction to the idea and, and it's going to be a great place to start for anybody who's curious about the topic, especially as it continues to get just more and more attention in both industry and in academia.

I'm sure a lot of our listeners are wondering what on earth is automl?

Yes.

So that's exactly the first thing that Matthew tackles in this post. But actually first he hits the critical point of what AutoML isn't when we talk about AutoML we aren't talking about automating the entire data science process. We're just talking about the modeling step of a data science pipeline, which is really just one piece. As any data scientist will tell you, AutoML, at least as it exists today, means hyperparameter optimization and model assessment. Data scientists will still need to be intimately familiar with the types of models available and when each one is appropriate. Just as always, AutoML is all about intelligently searching over the space of possible models, hyperparameters, and and features to produce the best results. When you're first learning about machine learning, you usually do these things in a brute force way with a grid search of many, many possibilities. It turns out, however, that there's a lot of really cool math that you can layer on top of your actual model to make sure that it ends up the best that it can be.

Why is it important?

Matthew points out that it's really just the next step in a chain of automation evolution that's been going on for decades now. So I really like one quote that Matthew includes from Sebastian Raschka, who we actually just featured on the podcast recently when he said this programming relieves us by managing rote tasks. Machine learning allows computers to learn how to best perform those rote tasks, and automated machine learning allows computers to learn how to optimize the outcome of learning how to perform those rote actions.

So if you wanted to start using AutoML, where would you start?

Well, the KDNuggets post compares and contrasts two popular Python packages that do so. They both happen to plug into the SciKit Learn framework, so if you're already comfortable in that toolset, that's one fantastic place to start. First is the autosklearn package, which performs its automated search using Bayesian optimization. Second is a package called TPOT that takes a different approach, leveraging another idea called genetic programming. Both Bayesian optimization and genetic programming are fascinating areas of intense research in their own right. They just happen to pair nicely with your favorite machine learning model, say, neural networks or boosted decision trees like caviar and a fine wine. It's not mentioned in the KD Nuggets post, but if you're on the R side, H2O is doing some interesting things on the AutoML front in their R package, so you might want to check that out.

Sounds really cool, Spencer. It seems like it might be an easy way for some people to take some of the monotony out of machine learning modeling while improving model performance at the same time.

Yes, exactly. So go read the post from the show notes and then go forth and automate yourself.

Data Scientists for those interested in the future of automated machine learning, you'll love next week's episode of Data Framed, in which I'll be speaking about AutoML with Randy Olson, the creator of Teapot and specialist in artificial intelligence, machine learning and data visualization. Thanks once again, Spencer.

Anytime, Hugo.

You're right. Time to get straight back into our chat with Michelle Gill. So something we haven't. Haven't talked about. We haven't really gotten so technical. I'm wondering what languages you use and what packages and libraries.

Good question. So we do a lot of our work in Python. There's great library support I use. So beyond sort of the standard data science kit of Pandas and scikit Learn, the deep learning libraries that I use are things like TensorFlow and Keras and Pytorch. Those are sort of my favorites. There's a chemistry library that's very new for, for deep learning and chemistry called deepchem that I'm sort of tinkering with for a project. But yeah, those are the big ones. Nvidia has a version of Docker that exposes the gpu. So we containerize our development environments, which adds a lot of reproducibility and is really nice.

Yeah, that makes a lot of sense. And for someone getting started in deep learning, what would you suggest? What packages or languages would you suggest they they try?

If you happen to know a language and there are deep learning libraries, there's nothing wrong with getting started there and seeing if you like it before you invest a lot of time learning another language. If you get serious about it though, I think the next step is probably to start working in Python. I think Keras is a really nice way to get started. It's very. It has high level sort of implementations of a lot of model primitives and things that you would need to use as a deep learner. I also like Pytorch. If you use Numpy a lot. Pytorch is in my opinion very Pythonic. So I quite like Pytorch. Those are my two favorites if you're getting started and if you know Python.

Yeah. And in fact the creator of Keras, Francois Chollet, his Keras book, his Deep Learning with Keris book was out late last year and I can definitely recommend that.

I've read his book and it's great. Yeah, and it's a good mix of practical, like there's the coding aspects of deep learning, but then there's understanding a little Bit of the theory and I don't think one needs to be deriving all of the equations for, you know, back prop for a neural network. But I think giving beginners a little bit of intuition as to what we're doing can help you understand what levers you should pull when you're trying to tune a model. And I think his book does a nice job of that.

I agree. And also for people who aren't proficient in Python but use R, there is now an R interface to Keras that Francois and JJ Allaire of RStudio fame have been working on and they've actually just published a book together which is a translate a porting of Francois book to the R ecosystem, which is worth checking out.

That's awesome. That is actually exactly why I said if there's something in a language that you know, there's nothing wrong with starting there. And I didn't remember how far along the Keras R interfaces. But yeah, if you know R really well and you just want to get started tinkering, there's nothing wrong with that. Get started with that. See if you like it, if it meets your needs, great. If you need to move on, then deal with that. But I'm a big fan of like getting some results and getting your hands wet as quickly as you can so you get experience.

Exactly. So we've talked a lot about deep learning. I'm wondering what one of your favorite data sciencey techniques or methodologies is, deep learning or otherwise.

Yeah, so I have one and it's actually much more sort of old school machine learning. When I was doing some software development at the nih, I worked on a compressed sensing project which essentially helped us collect our experiments much more quickly. We collect the data more quickly because we could actually skip some of the data points. And so that compressed sensing project used L1 normed regularization. So L1 norm takes the absolute value of the difference between the errors and it applies a penalty so that you can. It's used to prevent coefficients from becoming too large. But L1 regularization specifically selects for sparsity. So sometimes it's used for feature selection. And to me, I love that because it sort of stands for model simplicity. And that's a nice counterpoint in my mind to all this crazy model multi layer, 151 layer discussion. I believe that models should be only complex enough to accomplish the task at hand, but no more. So and so. Sure. With deep learning we sometimes need these really complicated models, but I don't think that we need to always sort of like bring a battleship to the lake, so to speak. I think models should be simple enough to do what they need. So that's why I like L1 Norm.

I love the L1 Norm. And you may, I think you may see that in the wild called lasso regression as well, right?

Yes, yes, absolutely. That's a great point. It's also called lasso.

Yeah. And yeah. That act of performing feature selection using such regularization can be really illuminating, I think.

Yeah, yeah. And I think it's good to have that grounding in basic machine learning. Even if you want to move on to deep learning, a lot of these techniques show up. So L1 regularization is used in deep learning models to prevent overfitting. Exactly the same reason it's used in general machine learning. So you sort of. You understand these concepts and then you can realize, oh, they apply in deep learning as well too. So it's good to have a foundation.

So my last question is, do you have a final call to action for our listeners out there?

Sure, sure. I, I have sort of my, my pet call to action. There's. Deep learning is such a new field that a lot of the practitioners have come straight from, you know, a degree, whatever that may be, to deep learning. And that's fine, that's important. Generalists can become a lot of things. But I'd love to see more people in the field with external domain expertise because I think those two things are really powerful when married together. So, you know, I'm sort of one of those people. I came from a biochemistry background, so I have this biochemistry knowledge that I try to blend with my deep learning work whenever possible. And it's. It's fun to see these tangential links. So I did a lot of signal processing as a scientist and convolutions. I knew convolutions back when I was processing signals, you know, so. So I see them in convnets, I'm like, oh, yeah, I know what they're doing. But you also have these. You understand sort of things about the way they work because you've done this other thing in your life or you have insights or you have creativity. There's still, there's value from other domains, and I would love to see that grow in deep learning.

Yeah. And I can relate to your story about convolutions incredibly well. In my postdoc, I did a whole bunch of image segmentation for, well, cytoskeletal polymers in particular, but we had to do a whole bunch of convolution of image data there as well. We weren't doing deep learning per se, but then when I saw convolutional neural networks and saw how the boxes move around, I was like, oh, yeah, I get that.

Yes.

That's the.

Like, those are the light bulbs that it's fun working with people who have domain knowledge because they're like, oh, yeah, I know it from blah, blah, blah. And yeah, that's great. Like, these are the cool connections that I think they make deep learning and they make the field of data science in general, it's so broad. I think they make it fun. And these people with different backgrounds, I think we add diversity to the field and I think that's great. I want someone to kick around ideas with who thinks very differently than I do. Helps me pull myself out of ruts.

Exactly. And that's what makes it so much fun.

Yeah. I would love to see more people with all kinds of domain expertise. I follow some people on Twitter who have music backgrounds and are doing deep learning in the arts. That's awesome. You know, that's more power to you, man. Like, this is a set of tools and you should wield them in ways that are fun and interesting and help you accomplish what you want.

I love it. Michelle, this has been so much fun and it's been such a pleasure having you on the show.

Thank you. It was all my pleasure. I really appreciate it.

Thanks for joining. Our conversation with Michelle about the modern superpower of deep learning. We saw two of the most intriguing verticals that deep learning has the potential to impact are manufacturing and healthcare, and that although deep learning is powerful, it isn't magic. Deep learning is good at making decisions involving the type of data it's seen before. Different data or contexts can be challenging, though, and it isn't good at reasoning about the unknown. We also saw, as we're seeing more and more of that, we need to be thinking more about interpretability of our algorithms and that some central challenges facing the field of deep learning are hype, the availability of talent, and the lack of domain expertise in deep learning experts. Make sure to check out our next episode. A conversation with Randy Olson, lead data scientist at Life Epigenetics. Randy specializes in artificial intelligence, machine learning and data visualization, is a community leader for Data is Beautiful and open. Science advocate Randy also created Teapot, a data science assistant, and a Python automated machine learning tool that optimizes machine learning pipelines using genetic programming. We'll be talking about automated machine learning, the verticals it could impact, and how it will change a large portion of what data scientists do every day. I'm your host, Hugo Bown Anderson. You can follow me on Twitter at hugobaun and datacampattacamp. You can find all our episodes and show notes@datacamp.com Community podcast. Sam.