#24 Data Science in the Cloud

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

In this episode of Data Framed, a Data Camp podcast, I'll be speaking about the superpower of data science in the cloud with Paige Bailey, Senior cloud Developer Advocate at Microsoft and more generally, lifetime advocate for machine learning, data visualization, AI, Python, and Rstats. Paige and I caught up on May 12, 2018, directly after the Microsoft build and Google I O conferences, and she was reporting directly from the frontier of cloud developments. What is the future of data science in the cloud? Stick around to find out. I'm Hugo Baun Anderson, a Data Scientist at DataCamp, and this is Data Framed. Welcome to Data Framed, a weekly datacamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter hugobown and datacamp@datacamp. You can find all our episodes and show notes@datacamp.com Community Podcast hi Paige, and welcome to Data Framed.

Hi Hugo. I'm happy to be here.

I'm really happy to have you here. And we're here today to talk about data science in the cloud and what a week for it, right? Maybe you can tell us why this is such a great week to have this conversation.

Okay. I am so pumped to talk about AI and machine learning and data science in the cloud this week because Build was Monday, Tuesday, Wednesday. Build is Microsoft's primary developer conference of the year and it's huge. It usually has around 16,000 people in person and usually about half a million watching the keynotes online, so it's crazy big. Then Tuesday, Wednesday, Thursday. So happening simultaneously was Google's developer conference called IO, and it also has a lot of people attending in person and a lot of people watching online, and both had tons and tons of announcements about AI, about data science, about machine learning. It felt like the entire conferences were focused on AI and machine learning.

Incredible. And I can't wait to get into talking about all the very recent developments but to create a bit of suspense, we're not going to do that just yet. And I want to find out a bit about you first. So how did you get into data science originally?

Cool. So I started off with data science before I think it was called data science. As an undergrad I studied geophysics and I was fortunate enough to get to do two NSF fellowships. NSF is National Science foundation. And they were both kind of focused on the planetary sciences. The first was at the Laboratory for Atmospheric and Space Physics in Boulder, and the second was at Southwest Research Institute in San Antonio, Texas. And both of them were all about analyzing large amounts of data, spotting outliers, doing statistical analysis, and then either discovering relationships kind of just by virtue of the data alone and also kind of graphing results and communicating them in a way that other folks could understand.

That's wild. So what then happened in your professional life that led you to be in your current position at Microsoft?

The plan was my, my undergrad research university, Rice was, was very focused on kind of getting everybody it could into grad school. Like that was kind of the expectation. So the plan was always to go to grad school. But after those first two internships, I did a third internship with a company called Chevron, which is very focused on oil and gas. They're one of the, the major oil companies in the world and the pay was quite good. Not going to lie, especially as a college kid who was doing an internship. And the project that they gave me was also very interesting. So they asked me to create a culture database for some of the oil and gas projects that they were working on in culture is gis, Shapefiles and other associated people ish data that you need to have in an oil and gas project in order to make sure that you're drilling in the right places, that you actually own the leases to determine when leases expire, that sort of thing. So that was the first project was building this culture database. And then when I finished that after like three weeks and they still had, you know, multiple weeks of me to, to be there, I started working on more on the three dimensional data and on, on doing a lot of other research focused work in the deep water space in the Gulf of Mexico. And that was just fascinating to me, like the size of the data sets and how expensive the data sets were and kind of the realization that nowhere else has this kind of data. That if you wanted to do any sort of three dimensional subsurface work, the oil industry was kind of the only place you could do it. That Sold me real quick. So they gave me a job offer after that summer. I was delighted to say yes. And that was, that was kind of the long and short of it. And it was also my mom got very sick my senior year of college and as the person who needed to take care of her, being able to support myself and support my mom and also do something really, really cool at work, that it just sounded like the best place to be.

Yeah. And how long were you there for?

Yes, I was at chevron for four and a half years. So I was there from 2013 up until 2018.

And then you made a move to Microsoft?

Yes, I transitioned from Chevron working as a data scientist to Microsoft. And the progression through Chevron started off with doing Earth science application support and also small scale plugin creation and database work and GIS work to eventually being a data scientist because most of the plugins that I was making were data analysis plugins and most of the real time data processing and real time drilling visualizations that I was making were data sciencey visualizations. So it was just kind of a. Oh, hey, that's what she's doing anyway. Let's just call her that now.

Exactly. So now you're a senior cloud developer advocate at Microsoft, and I want to explore what. You know, there's a lot of meat in there. There's the cloud, there's developer, and there's also advocacy work. And I want to explore what that means, but first through what kind of popular impressions there are of what you do. So what do your colleagues at Microsoft think that you do?

Cool. So senior cloud developer advocate. You're right, that is a completely. Whenever I first saw the job title, I was like, what the heck is this? You know, it sounds cool, but what actually would I be doing? Yeah, my colleagues at Microsoft, it depends probably on what division that they're in, because I guarantee you a lot of folks haven't heard of it either. But the idea is that we have this team of people who are ingrained within various communities. It might be JavaScript, it might be Go, it might be the security world, it might be Python or R. And the folks who are active in those communities, who contribute to open source libraries, who speak at a lot of conferences, who have been doing that work for their day job as an engineer. The idea is to get those people into Microsoft and to still keep engaging with the communities, still keep doing engineering. So making pull requests to the projects that we're currently working on, to the products that we have on Azure, but also to make sure that whatever's not working for the communities that they're a part of and enthusiastic and passionate about, that gets fixed. So if I'm a Python and R person, so if I'm using say, a deep learning virtual machine on Azure and I notice that something's not straightforward for a Python developer, I'll take that feedback and I'll give it to the product team and then we'll build a roadmap to incorporate that change into the product. Or if somebody at a conference tells me, you know, Cosmos DB sounds really, really cool, but I would love to see a tutorial that's focused on it as a graph database, then I would take that feedback back as well and probably end up helping to write that tutorial or that quick start. So does that make sense?

That makes absolute sense. So you've mentioned Microsoft Azure several times and I want to kind of move into this space of thinking about cloud computing platforms with particular reference to data science. And I know Azure can be used for a lot of other things as well, but I'm really wondering what data science means to you, what the cloud means to you and what, where they intersect, because there are multiple possible definitions of both of these things.

So cloud computing is probably, it's a huge revolution in the computing space and it's also probably going to be one of the most transformative technologies that any of us experience in our lifetime. And it's mostly because of suddenly you really do have the ability to democratize any sort of computational power that you have. And what does like that's, that's very kind of inspiring and vague and hand wavy the explanation. But the cloud, just think of it as a collection of servers that is sitting somewhere that you can leverage as needed, spin up and spin down as needed, and a lot of additional kind of software focused and security focused tools that sit on it. So say I am a grad student who needs to analyze, you know, a terabyte's worth of biology data. And historically if you were in grad school and you needed to use, you know, some high powered computer, it might be that you had to stay up until 2am to use it. Because that's when you know that's the slot that you got as a brand new first year grad student. You know, if you needed to use a computer that had a certain threshold for a GPU or graphics card or something, then you, you just had to, to wait your turn. With the cloud, suddenly you can provision anything that you need so you can provision additional storage space, you can provision one of the most powerful computers that is out there. You could automatically deploy a web application like a single page web app, as something called a serverless function, which you only pay for the amount of time that people are actually looking at the webpage. Or you might trigger a serverless function for your website where you're only paid for the compute time when somebody pings a REST API. It's amazing the stuff that you can suddenly do. But the thing that excites me the most about the cloud is that it's the very, very first time that we're actually able to do deep learning in a very serious way.

And that's one of the great things, I think, that you've motivated cloud based computing in a lot of ways. And one of the things underlying what you're saying, I think, is that when our algorithms need to scale or our data sets need to scale, or we've got kind of one off projects, we don't need to be continually buying more hardware or changing hardware setups or anything along those lines. And I think deep learning is a great example.

Absolutely. Those are excellent, excellent points. And coming from the oil industry, which is kind of outdated, especially in the IT space, but being even requesting a database. Requesting a database. When I worked at Chevron, it would often take six months to go through all the bureaucratic paperwork. So if you wanted a new machine, you had to schedule it like six months in advance as to what the specs should be, how much, you know, how much hard disk space you would need, all of all of these things. And then, you know, you actually get the machine six months later when it turns out your requirements could have changed. Like, you know, suddenly that graphics card is no longer the best graphics card on the street. If you had an application called Patrel, the requirements for that were constantly changing. So now, you know, potentially you couldn't run that application and you would have to order a new graphics card. It was just awful. It was an awful, awful user experience.

Oh yeah. And I love the fact that, you know, when doing stuff on the cloud and collaborating with other people, you don't necessarily need to be worrying about requirements files or virtual environments and making sure that what happens on my laptop or my computer can run on theirs as well, because we all can verify that we're using exactly the same environments up there.

Absolutely. It's being able to concentrate on the thing that you're supposed to be concentrating on, which is actually doing the data science work and kind of abstracting away all of the others, you know, non data science focused Requirements like you don't need to worry about how much disk space you have on your machine. You don't need to worry about package management necessarily. If you're using, if you're using a deep learning vm, you don't have to worry about drivers which are, were always the biggest, biggest headache drivers for your graphics card. They were, they were always so atrocious, but now it's, you know, you spin up a vm, you use it however you need, and then you spin it down when you're done and it's all for like the price of a cup of coffee.

Awesome. That's really cool. So we've really got a nice working definition of the cloud. What's your working definition of data science? And I hesitate to ask this because putting it into words can sometimes it'd be a bit trite, but I think, you know, everyone has their approaches it from a different perspective. So I'd love yours there.

Absolutely. So data science, machine learning and deep learning, I have different definitions for all of the three and I think that dependent on who you talk to, they would probably have drastically different, different opinions than I do. So data science I usually think of as heavy on statistics, heavy on visualizing, cleaning and understanding data sets. So not necessarily predictive modeling because a lot of people are still focused on kind of descriptive statist. So taking historic data, visualizing it and understanding patterns and relationships. And I do think that that still, that still counts as data science.

I love this, especially because you are a machine learning and AI advocate. To hear you say that data science is not necessarily always involving predictive analytics.

Absolutely. And another thing is that, you know, you can, you can derive business value in so many ways and a lot of companies, they don't really have any insights into what data they have at all. You know, it's not in a centralized location, it's usually a really very poor quality. It's often housed in like, you know, 20 billion spreadsheets. And if it's valuable to them to answer a question that isn't necessarily a predictive modeling question, and that does require a lot of rigorous analysis to do, then I would definitely count that as data science. And I don't, I usually think that data science requires some sort of Python and R. I don't think that Hadley Wickham has this great quote that you can't really do data science in a gui and I am a firm believer of that. So I know that a lot of data analysis can be done in something like Excel and I think that that's wonderful. But for data science, I usually think it requires knowing a little bit of Python, a little bit of R, and a little bit of SQL.

And does that speak to the fact that something scientific needs to be reproducible?

For example, reproducibility and then also usually working with a variety of data sources. So the data engineering aspects, usually that requires some sort of programming, unless you want to have a very, very painful life of merging data sets and Excel spreadsheets. But science is all about reproducibility. If you can't be empirical about what analysis you're doing, then I don't think that you can call it a science.

We'll jump right back into our interview with Paige Bailey after a short segment. Our newest segment on the Dataframe podcast is Data Framed for Social Good. I'll be checking in with Peter Bull, co founder and data scientist at Driven Data, a social enterprise that works with nonprofits to use data effectively and runs machine learning competitions that have a social impact.

Hey Hugo, I'm so excited for this segment. There's a ton of valuable work that data scientists can do, and I'm looking forward to sharing some of the ways to get involved.

I feel like in the last few years I've seen a trend of projects where people want to use data in the service of humanity. Is this something that you've noticed in your work?

Well, first, it's worth noting that Data for Good has a long lineage. I'd like to share with you the story of Jon Snow.

Are you still there, Peter?

Yeah, well, I just wanted to make sure that there was some blank air so you can add the Game of Thrones theme song or maybe some dragons that are screeching in post production. You're going to do that, right?

Sorry, no.

Ah, man. Well, seems like a missed opportunity. Anyway, this John Snow is not the king in the north. Instead, he's just a doctor in 1850s London. While he was alive, people thought that cholera was caused by bad air. During an outbreak in London, he collected and mapped data on the location of all of the cases. By tracking these cases over time, he was able to pinpoint the source of the epidemic as a neighborhood water pump. He's now considered one of the founders of epidemiology. But this is one of the early stories of Data for Good. And right around the same time, Florence Nightingale was using data to save the lives of British soldiers in the Crimean War. She was a nurse in the army and kept meticulous records of causes of death. She created what today we know as a polar area diagram, A sophisticated data visualization that helped communicate the changes in the causes of death of soldiers over time. The power of her data and these diagrams helped convince army leadership that their biggest enemy was disease, not the opposing army.

What a great example of using data to tell a story and communicate to people not trained in statistics. Thanks for sharing.

Always a delight, Hugo. Next time we chat, I'll share more about how things look 165 years later and how data scientists are doing even more today to make the world a better place.

After that interlude, it's time to jump back into our chat with Paige. So before we get to the current and future developments with respect to data science in the cloud, could you give me a brief history of data science in the cloud?

Sure.

So, a brief history of data science in the cloud. I'm probably not the best person to give this, but I think that everyone would probably agree that it started with Google. So being able to take massive data sets, understand them, and then also apply kind of distributed processing techniques to that data. Over time we've transitioned from using things like Hadoop to using more and more Spark, if you have familiarity with that, and gone from using kind of CPU machines to GPUs to now we have, now we have things like FPGAs and also custom silicon for algorithms like TPUs. So there's a branch of processors called it's custom ASICs and TPUs would be one example of that. Tensor processing units. But the transition has gone from oh wow, we sure do have a lot of data, wouldn't it be great to understand it? To just kind of building more and more tools to enable that understanding. And I think that one of the biggest step changes in that has been the kind of open sourcing of tools that are exceptionally powerful for deep learning and large scale data analysis. So again, like Apache Spark for spinning up clusters of machines and doing either just data processing in general or with things like mllib doing distributed machine learning or with Spark, deep learning doing distributed TensorFlow or anything else. I think that open source tooling for for data science is one of the best advantages that we have as a discipline.

Okay, great. So I'm itching to find out about the new developments that you've discovered and been involved in this week. But you're mentioning open source. I'm really interested in the future of a trade off between open source software and productization of data science products and for example, machine learning products. So I'm wondering if you can speak to how you think that will evolve in the future.

Yeah. So a trend that we're seeing often is taking open source tools again, things like Apache Spark, things like maybe scikit Learn or Kubernetes, which isn't necessarily related to data science, but it kind of is. Because if you build a machine learning model, you probably want to scale it at some point and the way that you're going to scale it is probably going to be with containers and then once you've got containers with Kubernetes. But the thing that we see over and over again is that we have these great open source tools and then everybody realizes, man, it sure is hard to get all these open source tools to play nicely together. So you end up with companies like Cloudera that help businesses make sense of Hadoop ecosystems and all of the associated packages. With those you get companies like databricks that at first were all about Spark and now it's a whole suite of other tooling that just makes it so that if you're a data scientist and you want to do either machine learning or data data analysis, all you have to concentrate on is on that and not necessarily about making sure that your Spark cluster is working the way it's supposed to be and rescaling it yourself. It just has auto scaling and wonderful notebooks. But I think that the trend is going to be that we're still going to have open source tools. I think that we're going to see more and more companies spring up to help you make sense of those open source tools and make them production ready. And I also think that unique to deep learning and to machine learning, the value is in the data. So having, you know, having algorithms is great. Open sourcing algorithms is great. You suddenly get these wonderful, you know, huge communities of folks to work on projects, but having it doesn't do any good to have algorithms if you don't have data to apply it to and if you don't have specific business problems. So I think I would love to see more open source tooling around data engineering. I think that the tidyverse tools are phenomenal and they're beautiful, they're intuitive to developers, they have consistent naming conventions, they solve very specific tasks, but they're also easily composable and extensible. I love tidyverse and I would especially love to see similar tooling in the Python community. And I mean, I know we have Pandas, but I personally, I still love doing data engineering in R. Just I think it's much more intuitive. But I would love to see similar things for go and for a Number of other languages for sure.

So I can't wait anymore. Paige, I want you to tell me about the newest developments in cloud computing that you've discovered this week and in particular ones that you think will be impactful in the data science ecosystem.

Cool beans. Okay, awesome. So do you want to hear first about the Google stuff or about the Microsoft stuff?

It's up to you.

Okay, I'll go with Google first. So Google announced a number of things. First off, they rebranded their research division from Google Research to Google AI. And that's not to say that they're not going to do computer science specific research anymore, but it's being very, very transparent that they're focused extremely hard on developing AI tooling, developing new AI products. So that's very telling.

Huge.

Yeah. They also had that amazing demo where a computer calls and schedules an appointment and uses ums and uhs and sounds very much like a human. So that. That was Google Duplex. They also have a new feature for Google Maps where you have extended insights into locations around you. So Microsoft has a similar product called Location Insights, but it's basically the integration that's been done with Maps is you can pull down Street View onto your phone. So as you're walking along, it's showing you directions, almost like an augmented reality situation because you know your camera is pointed at the, at the street and it looks exactly like the street on your camera. You can point your camera at various buildings around you and it does automatic detection of what you're looking at there. There was another. Another great announcement around photos, so automatically being able to recolor images. There's another announcement about helping compose emails and a ton of announcements, not necessarily at I O specifically, but at the TensorFlow Developer Summit recently they announced TensorFlow for Swift, the programming language TensorFlow JS for JavaScript, and they were even working on node bindings so that you can leverage the GPU in your laptop and do deep learning in the browser. There is lots of great research around. They have. So one of Google's flagship projects is doing diabetic retinopathy imaging or diabetic retinopathy diagnosis rather. So this is a disease that if you catch it in time, if you catch it early, you can prevent it and it's very easy to prevent, but if you don't catch it in time, then the person actually goes blind. So the. And the only thing that you need in order to kind of diagnose the illness is a picture of the eye. So Google was able to build an algorithm that detected better than better than doctors, even whether or not a person had diabetic retinopathy and that was included in a new FDA product that was approved. So this is, I think, one of the first algorithms that has received FDA approval in a medical device for use. And that will, I think that's going to kind of transform the number of algorithms that we see in healthcare devices, especially for things like medical imaging. So all of that just massively cool stuff.

And the first set of developments you mentioned are really focused on user experience. The second set I find really interesting because they're kind of speaking to how developers can use what Google's, Google's working on. And I'm just wondering, you know, this last example, the diabetic retinopathy, is this a product that working data scientists can get involved in, or is it closed source and proprietary or.

Everything's open source. That's, that's what I love about TensorFlow so much and about, well, you know, and SciKit learn as well. But TensorFlow, all of the projects that I just mentioned are open source. Like even, I think even the data set, the diabetic retinopathy data set, and there's also a lung X ray dataset. But if you want to get involved with either product or either project, rather like all of it's freely available online, that's fantastic.

Just out of interest, why do you think Google open source is all of these things? Because, you know, in the end they are a business, right?

So again, it kind of, at least to me, it points back to the data, right? So if you having all of these powerful algorithms, it's wonderful. It doesn't mean anything if you don't have data. And also the first cohort of tooling that I mentioned, so all of these additional capabilities for photos, all of this additional augmented reality stuff for the maps components, all of this additional composability for emails, if you're using those tools, then you're kind of feeding the data machine, right? Like the data that you input into it. Even though these are wonderful platforms and they're incredibly useful for all of their, all of the folks who use them, you're still feeding in data, which helps make the algorithms better, which ultimately improves Google's tools, some of which they do sell. And the other thing is too, if you're doing data analysis, deep learning, machine learning on a cloud environment, the two most expensive things that you can do is compute. So what resources are you using in order to actually carry about the work that you're doing? And then also storage for the data Right. So if you open source the algorithms and you show people like, wow, isn't this awesome? Look at all this cool stuff you can do. Isn't this really, really interesting? Then if that person wants to turn around and do it, they have to pay for data that they have stored and they also have to pay for the compute that they use in order to analyze that data. So it's kind of like giving you free leather seats for a car if you buy a car. Right?

That's a great analogy.

Yeah. So I think, personally, I think it's incredibly wise to open source this, this software and to build a great ecosystem with quick starts and tutorials, and to also build straightforward API interfaces and kind of delightful tooling so that folks will actually want to use the products and want to use them really heavily.

We'll jump right back into our conversation with Paige Bailey after a short segment. It's now time for a segment called Data Science Best Practices. I'm back here with Eric Ma, a data scientist and research investigator at Novartis who helped create our Python network analysis curriculum here at datacamp. Fun fact. This summer, Eric and I are going to be at SCIPY in Austin, Texas together teaching Bayesian statistics. How are you doing today, Eric?

I'm feeling great, Hugo, except it's been cold here in New England for way.

Too long and here I am rocking the sunny weather in Sydney.

I'm envious. Man, it must be great down there.

For sure. So, Eric, I recently saw a tweet you posted about software engineering practices, but more importantly, those for data scientists. What's up with that?

So this stems from a lot of frustration I've been experiencing with my code. I've noticed a pattern emerging from my work. If I take the time to apply good software engineering practices to my data analysis code, my future self is much less frustrated than if I didn't.

Interesting. What do you mean by software engineering practices?

So by software engineering practices, I'm thinking along several lines. These are, number one, refactoring, number two, testing, and number three, version control. Now, data scientists don't have to be seasoned experts in these three things, but I think it's worth mastering some basic ideas.

Great. So it sounds like we're about to import software engineering best practices as data science best practices. Am I right?

Exactly. Exactly.

Great. Tell me about refactoring.

Here's the basic idea of refactoring. That is to break out repeatedly used code in a separate callable function to determine when I need to refactor a chunk of code or not the litmus test for me is really quite simple. The moment I select a chunk of code and hit copy, that's a candidate for refactoring. Now, another criteria I've also used is to see if my code's business logic is expressed too verbosely or not. Here's an example. If I have three logical items that I'm doing, then why should I be reading 17 lines of code to see that I'm accomplishing three things? Shouldn't it be more like three lines, give or take? So that's the kind of thinking that goes into refactoring.

What about when actually doing the refactoring?

Well, then I'll ask myself a few more questions as well. So, for example, when designing the function, what's the most logical set of arguments that I need to pass into it? When designing the body of the function, are my variables named appropriately for the function? And generically enough, Another thing is, does it do one logical thing and only one logical thing? Do I have good documentation? Am I writing my doc strings right? Do I have an appropriate test, a small use case that makes that I can use for testing the correctness of this function? Now, these questions help me keep my code well written.

And do you store these functions in other files or execute them in the opening cells of the jupyter notebook you'll be working in, or what?

So my bias is towards storing them in a separate file. However, I believe the final choice should be dependent on two factors. The first one is who's using the notebook, which is the audience, and two, whether the function is used across notebooks or not. Now, on the latter point, it's pretty crystal clear to me that if I'm going to use the same function across multiple notebooks, for example one named GetData, right, which loads the data into memory, then that function is going to be used across multiple notebooks, and therefore I should refactor that one outside into a separate PY file. Now, as for audience, if it's me doing development work in which I'm prototyping the function and prototyping the tests for them, then I am my own audience and I'll leave those functions in the notebook until they're ready. On the other hand, if this notebook is going to be used as part of a document of some sort of that gets presented to others, then it ultimately must be refactored into a separate PY file. So my preferences here are quite clear. I believe that notebooks are best used as an interactive prototyping tool or as a presentation tool, and therefore they should be free from clutter.

Thanks, Eric, for that thoughtful introduction to refactoring. We'll need to get you back soon to discuss testing and version control.

Looking forward to it, Hugo.

Time to get straight back into our chat with Paige. Having API products is something that I'm really very interested in because I think for all the good that, for example, online machine learning competitions provide us, for example, they do convince a lot of beginners that the output of a machine learning model is a CSV file.

No.

Right, right. But that's, that's what you do, right? And this, this is not the case now, and this isn't the case going into the future.

Clearly not at all. Like, but I feel, I feel your pain. Like there, there have been. So often I've worked with data scientists and it's either. Okay, well, the result of this analysis is either going to, to be the static document that, you know, I crank out for my boss, and it looks very similar to a traditional business report, or it's a CSV file with, you know, a series of IDs, and then whatever, whatever you've classified them as. And that's not, you know, any, any data scientist who does that now, don't do it anymore. Like, that's, that's not okay. And it's not respectful for any of the software engineers that you work with, because the ideal situation is that you have something like a container where, you know, you have your algorithm. It doesn't matter if it's a protobuf file. So like a PB or a dot PY file if you're doing something with Scikit Learn or a dot R file if you're doing something with Carrot or any of the other R machine learning packages. But then you also have something like a schema JSON to define inputs and outputs. So like, I expect to get, you know, a jpeg Less than 4 megabytes in size, and I will output a classification and a confidence level or something of that nature. And then also, you know, some sort of script to initialize and run your model. But the idea is you package it up into a container and that way any software developer in your organization can ping it the same way as they would arrest API. Right? Like that's, that's making something that can fit into an application. And if you don't have, like, it doesn't matter if you have the best algorithm in the world, like the most accurate, insightful classifier ever. If you can't fit it into a business process, then it doesn't matter. You might as well not have created it at all. It's the same as. It's the same as doing any sort of research work. As a scientist, if you can't communicate what you've just done, then it, the research might as well never have happened. Communication and being able to integrate your algorithm into existing business applications, existing processes, that's what we should all be trying to do.

I couldn't agree more. And this really speaks to productionizing whatever you're working on, right?

Yes. And also to your point, before being able to scale it so you might start off with 1,000 customers, but if you're doing your job right, then eventually you'll have a million. What does that mean in terms of changing your computational workloads? Or do you need to use Spark now? And if you are using Spark, do you need to spin up clusters of machines that are CPU enabled or do you need GPU enabled machines? Then also, how do you deal with the data that you have in storage? Do you want to incorporate streaming data and all of these other things? The entire concept behind having kind of an end to end machine learning life cycle and how you would go about retraining models over time and doing kind of checks to verify that the data coming in is what you expect to see and what was consistent with the first iteration of the model or all of that is incredibly needed. And I don't think enough institutions are doing it yet. There's this great paper called TFX TensorFlow Extended that was released about halfway through 2017 and it is amazing. I reread this paper at least once every three weeks, but it is just, it is phenomenal and it goes through all of the steps that you should be doing as a machine learning engineer to make sure that your model is doing what you hope it's supposed to be doing and is production ready.

Well, this actually speaks to my next question. Question that was really stimulated by everything you were saying. And we'll include a link to the TFX paper in the show notes for the listeners out there. But my question is if you have data scientists, machine learning engineer, building these models, you have data engineers working on the, you know, data storage databases, data lakes, whatever it may be. You have software engineers at the other side using the output of these productionized machine learning models, whose job is it to make sure that the models stay doing what you think they're doing?

So I personally think that it has to be a constant collaboration between the machine learning engineer, the software Engineer, and the DevOps practitioner because you, you have to have a DevOps mindset. You have to kind of understand what do I need to be logging in terms of my model's output? How often should I double check to make sure that the output makes sense and is still the, the accuracy that I want and that I need. And then also what phase gates should I build into my automated model retraining steps if that system breaks down? So what do I mean by that? I mean that if so, say you build a model, you're really happy with it, it gives you 85% accuracy or 90% accuracy or whatever. You deploy it out to the world and it continues to work great for the first two months. And then that third month, you know, suddenly your accuracy drops below whatever threshold the business had a requirement for it to be. So suddenly it's like 80% accuracy. Then there should be like a flag in your DevOps process that notices that change and then automatically triggers. Okay, well, data scientists need to look at the model, retune hyperparameters, incorporate additional data, or maybe something happened to change, the consumer base changed or something of that nature. But whatever the reason, data scientists need to reevaluate the model and re architect it.

And this challenge is known as model drift, right?

Yes, model drift. But the DevOps for data science space is, I think it's going to be huge. And again, there aren't a whole bunch of people who are currently focused on that area. Like right now it seems like everything on the Internet is all about like, oh, hey, let's build an algorithm. You know, like let's, let's figure out how to do that. But nobody ever really focuses much on the kind of the data engineering aspect. So getting data into a state that it needs to be in order to get into the model or, or what do you do with a model once it's been created. And that's what, that's kind of what I love about data camp too is that you guys have so many great courses on the data engineering side as well as the algorithm building. Like I said, it's going to be much, much more important as machine learning models are deployed out into the wild to make sure that they're kept up to date and that they're incorporated into applications in a responsible manner. And that's going to take DevOps practices.

Listeners out there, please listen to this if you want to be ahead of the curve and think about very necessary aspects of the work we all do, listen to page and think about DevOps for, for data science. Now this idea of productionizing machine learning models and having APIs, I think is actually will dovetail very nicely into a few of the announcements that Microsoft build. Because I am aware that, you know, one thing that's, that's happening, that's really exciting is the Cognitive Services API, right?

Yes. So Cognitive Services are very cool and a lot of folks have similar services, but what they are is basically taking a deep learning model that's been trained against millions of images, millions of tagged images, or lots and lots of videos, lots and lots of plain speech. So taking lots of data, taking very high end machines, training up deep learning models over the course of weeks and then deploying them as rest APIs that anyone can call. So what does that mean? That means that the companies that have most of the data, right, so the Googles, the Amazons, the Facebooks, the Microsofts, the whatever, Microsoft, Google and Amazon have decided to expose those models as REST APIs that mean you don't have to retrain anything really. You don't have to, you know, have millions of images or millions of tagged pieces of data on your own site. In order to build a model, all you have to do is Write less than 10 lines of code in whatever language you want, ping the REST API and suddenly you get back. For the speech to text API, you get back spoken words in English or in any other language that you choose. For the Vision API, you get back a plain text description of the image as well as a whole bunch of different tags of things that the API thinks it sees in that image with confidence levels for each. For the face API, you get back emotions, you get back estimated age, estimated gender, you get back specific locations. So like I write left or I right top, I right bottom, whatever. Lots and lots of different locations that you can use for various things in your, and you don't need to know machine learning or deep learning at all in order to leverage those APIs.

That, that is awesome and I think incredibly powerful and very exciting. But there is a dark side to publicly exposing trained models with APIs, right?

There is totally a dark side. If you expose these APIs to folks who don't necessarily understand machine learning or deep learning as a process, it might be very, very confusing to them. Why? Sometimes it works and sometimes it doesn't, right? And also there's the inherent bias of any data set that's used to train the model itself. Let me start with that first one though. So a friend was recently looking at one of our Cognitive Services APIs. This is one of the custom ones. So you're able to upload specific photos and have it trained on those photos. For example, if you have a corpus of employees, you could train the model on your employees and suddenly it would be able to say like, oh, I see Paige bailey, confidence level 75%. So she did not know that the not aspects then kind of the not classifications are just as important as the R classifications. So you could upload five pictures of me and try to train against that. But if you don't have any examples of things that aren't me, it would just kind of naturally assume that any person who looked vaguely like me was me. And then there were also a lot of results given with precision and recall, and she didn't quite know what those meant. There was sort of a misunderstanding about the kinds of pictures that you should upload. So for representative training samples, ideally you would want to have lots of different pictures of my face and in various lightings from various angles, and that would give you kind of the best classification result. But that's never explicitly stated in any API. And then also it's never going to be 100% accurate to classify. Right? It's always going to be some confidence level and it's never going to be 100%. So helping people understand that uncertainty, I think is going to be an incredible responsibility for everybody who has these developer facing APIs that are very powerful and are incredibly useful and do a great job at democratizing AI. But if you, if you don't let people know some of the dangers of using them, then then it could get real bad real quick.

And that problem's associated with people who may not have enough context and enough expertise in machine learning, deep learning AI. But there's another problem that a lot of research is going into at the moment, which is, you know, if you have bad actors who do have a lot of experience, they can actually, for example, extract sensitive information about the data that was used to build the model if they're clever enough about it, right?

Absolutely. That's something called adversarial machine learning. And being able to kind of reverse engineer the model, reverse engineer the data is really, really fascinating. There's a research team out of Google called Clever Hans that does amazing work in this space, but it gets very scary. Like, you know, you can have something called a single pixel attack and that is real term where if you have a classifier that you know is supposed to be examining an image and then making some sort of classification on it. So like this is dog, this is cat, that's llama. If you introduce A single red pixel or a single pixel of any color in a very specific location. You can suddenly go from accurately classifying with a high confidence level to classifying incorrectly, but with an even higher confidence level. So that's a single pixel attack. You can also just introduce random noise, like 0.07% random noise and get back a completely inaccurate classification because the images, they look pretty much the same to a human. For the random noise, one that looks almost exactly the same. If it was a picture of a panda, it would still look like a panda, but to the classifier, you know, it's just so overfit that it doesn't know any better. It's very scary.

It is. And we'll include a few links in in the show notes with respect to these poisoning attacks and also extraction attacks. There are a lot of smart people working on these, these types of challenges. They have actually been able to show that these types of attacks are very hard to defend against as well. But we'll include all of that in in the show notes. We discussed the Cognitive Services API. What else happened at Microsoft Build that you find find really exciting.

Cool. If you're a C developer, there's a new thing called ML.net. this is some of the more vanilla Scikit learning machine learning algorithms you can suddenly use as a. NET developer. That was one announcement. There is also FPGAs. FPGAs are incredibly fast inferencers, which means that the predictive capacity is much, much faster than a TPU. We showed an example using TensorFlow and classifying chips for a company called Jabil. There was also an announcement about something called Cognitive Search, which I thought was incredibly cool. That is basically you just throw all of the data that you have in various stores in Azure you might have some CSV files or PDF documents, and Blob storage, you might have some databases. And Cognitive Search just kind of goes through extracts N grams, so it extracts keywords and it automatically builds a knowledge graph for you. So what does that mean? It means that, so say you have PDFs, it goes through and extracts all of the text from those documents, but it also looks at embedded images within the PDFs and using the Cognitive Services that I was mentioning before, it gives like a plain text description and also a whole bunch of tags for things that it thinks it sees in the image and then automatically links them up together. So the example that we showed in the keynote was the NBA just kind of put all of its players photos in it. It put all of these PDF documents called Game Notes for player performance. And suddenly you were able to find linkages between, like LeBron James likes wearing Nike basketball shoes. Wow. And that was because it noticed LeBron in an image. Like it was able to detect him specifically and also Nike shoes specifically. And that was just rock hard.

Awesome.

Yeah. So we're thinking about correlations of patterns in that sense, which is really cool to have that type of recognition.

Yeah. And it helps you as a data scientist also ask a lot more interesting questions. So if you, if you see these patterns and relationships, you can start thinking like, oh, well, you know, now I suddenly have data that I didn't have before. I've got, you know, text that was extracted from PDFs, but then also counts of how many instances of seeing Lebron and Nike shoes in the same image. And yeah, it's just, it gets, it gets very interesting very quickly. And I think we also announced the model logging capabilities of Azure Machine Learning. So being able to track model performance over time and then also being able to deploy it as a containerized instance, you can package everything up in containers in a completely open source way. But Azure Model management also allows you to ping it as a serverless REST API call. So that would reduce the amount of money that you would need to pay in order to leverage that model at will. So it's kind of like creating your own very specialized cognitive service, just as a data scientist working for a company. So I love the idea of having a model marketplace where data scientists from wherever they happen to be, build a model on certain data sets, they deploy it as kind of this REST API, this containerized instance, and then people can ping it and pay accordingly. I think that would be such a cool thing.

Yeah, I couldn't agree more. And I think this also speaks to kind of the burgeoning or the emergence of more and more transfer learning occurring in the machine learning space.

Yes. And if you haven't taken a look at it yet, take a look at TensorFlow Hub, which was also announced during the TensorFlow Developer Summit. It's an open source portal for sharing data sets, but mostly trained models and also model components. So people can kind of Lego brick and architect their own model by using bits and pieces of other models.

That's awesome. So I've got time for one more question and what I'd like to know is for beginners, people who, you know, have done data science locally or on Data Camp or wherever it may be, what can they do to get started with cloud based data science?

So what you can do to get started with cloud Based data science is. We have man like that. That is a great question. So I, I would, I would highly recommend looking at the documentation on Azure and on Google for GCP and on aws. But dependent on whatever cloud provider you want to use, they'll probably have documentation and quick starts and tutorials. I don't know of a course that focuses specifically on distributed machine learning, but I would love to have one.

You should come and teach one at Data Camp sometime.

I would love to, if y' all would like. And the packages that I would probably recommend most are mllib with Spark for. For kind of distributed machine learning. And that would be either with CPUs and I think, I think it's just CPUs, but I'm not sure, so don't quote me on that. And then also Spark Deep Learning for the deep learning approaches and that does support GPUs and then also dist keras. But those three packages are incredibly useful for distributed machine learning at scale and then machine learning on the cloud in general. You could probably use the same tools that you love and adore already. So scikit, Learn, Carrot and tensorflow. It's just be that you would be accessing different kinds of data, you would use it in the same way, but the connections would probably be a little bit less intuitive and then being able to productionize it. So once it's created, how would you be able to deploy it?

Fantastic. Paige, this has been such an absolute pleasure. Thank you for coming on the show.

It's been awesome. Thank you so much for having me. And I wish you were at PyCon this year.

Yeah, me too. Yeah. I'm in Australia, of course, as you know. But Pycon is one of my. Actually on that. PyCon is one of my favorite conferences because in my daily work I kind of get stuck in using Python really like the scientific Python stack. And I forget how incredibly broad the community and uses of Python are. So that's one of the things I always encourage people to do when they go to Python. A lot of the value I think is just finding out what everything else is doing with this kind of incredible language. Right?

Yeah, it's everything from web development to like sysadmin tasks to, you know, it's everything.

Well, thanks once again, Paige. It was an absolute pleasure.

Thank you.

Thanks for joining our conversation with Paige about data science in the cloud. Paige told us about a plethora of developments from both the Microsoft Build and Google I O conferences, from Microsoft Cognitive Services API to Google's Diabetic retinopathy detection algorithm and Google researchers rebranding as Google AI. We also saw the importance of working data scientists to have a DevOps mindset when thinking about putting their models into production and how cloud computing may actually be one of the most transformative technologies that any of us experience in our lifetime. Make sure to check out our next episode, a conversation with Jared Lander about how to build data science communities. Jared is the chief data scientist of Landa Analytics, a data science consultancy based in New York City. He's also the organizer of the New York Open Statistical Programming Meetup and the New York R conference, as well as being an adjunct professor of statistics at Columbia University. He also likes pizza, but as Jared says, who doesn't? I'm your host, Hugo Bown Anderson. You can follow me on Twitter hugobown and datacamp@datacamp. You can find all our episodes and show notes@datacamp.com community podcast.