Data Scientist vs. Machine Learning Engineer

Sam.

Hi everyone, welcome. My name is Sandhya Simhan. I'm the Director of marketing at DeepLearning AI. Welcome to our expert panel, Data Scientists vs. Machine Learning Engineer. This event is presented to you by DeepLearning AI and 4th Brain. 4th Brain is a company that creates accessible and flexible pathways to AI careers by training candidates with valuable practical and interpersonal skills. You can learn more about their programs on their website. Fourthbrain AI Data Scientist and Machine Learning Engineer are both highly sought after roles in the tech industry. When it comes to the difference between these roles and the responsibilities, there has been much confusion. So in fact we've received many questions from regarding this topic from our community today. We hope to provide you with some inspiring insights through the discussion with our amazing panel of speakers. You can find the list of topics we'll be covering if you scroll down to check out the event description. After the panel discussion we will be taking questions from the community via slido. If you have signed up for the Slido ticket, check your email for the access link to post and upvote questions there. And now, without further ado, let's welcome our panelists today. They are Aishwarya Srinivasan, AI and ML Innovation Leader at IBM Kate Strattoni, Founder of datacated and Steve Nouri, Head of Data Science of the Australian Computer Society. Welcome everyone. Let's start off with a brief intro. Aishwarya, do you want to go first?

Hey, thank you so much Sandhya. I'm really really glad to be here. I can't tell you how many positive comments and DMs I have got often after we released the poster for this event. I'm super excited to be here and amazing set of panelists with me, Kate and Ste been like in touch with them in the recent past and they're doing amazing work. So yeah currently as you mentioned I'm currently AI and ML Innovation Leader at IBM. So previously I'm coming from a data science background. I have a computer science and data science masters and I have been working hands on on building machine learning models so I can speak better for what data science requires and how can you build your career up in the field. So from being a data scientist I've been moving up the ladder in more of a leadership where I am still working hands on because that's something I always wanted to do. So I made sure that that's part of my role where I'm still working on hands on machine learning models. Apart from which I'm working very closely with product management teams and clients. So where we try to understand that what are the best kind of products and services we can build in IBM cloud and how can it benefit the best for for the clients. This also involves me working closely with IBM Research so that we can bring up the latest state of the art models from there and integr integrate them into our products. So that's what I do briefly.

Awesome. Kate.

Hello. Hello everybody. Thank you so much for having me here. Really, really excited to be part of this panel with Aishwarya and Steve and Sanya. Great to meet you all. I know we've been kind of watching each other on LinkedIn and maybe some other social media. So really fun to spend the next hour together here. My name is Kate Strashney. I'm the founder of datacated, which is really dedicated to data. So I love all things data. I started my background in really finance and investments and sort of went into risk management, then took a little detour and spent some time in data analytics at a consulting company and then realized that my passion is really for talking to people and building communities and this is what I really do with dedicated now. Now I'm not an expert data scientist nor machine learning engineer, but I do have conversations with several companies that either specialize in this space or people that have taken this path and journey to becoming a data science and machine learning engineer. Really hoping to share what I'm seeing across the community and take questions from the audience when the time is right. It's a little bit weird for me to be part of a panel versus moderating. I typically on the other side of the show, you know, asking the questions but it's definitely fun to be on the other side here. So thanks for having me.

You're very welcome. We're excited to have you here. And last but not least, Steve.

Thank you very much Sandia. It's great to be here among this awesome panelists. I know all of them from LinkedIn and we interact and I guess we are all passionate about data. My background is software engineering. I was an IT graduate a long time ago when data science was not, I guess a word Back then I was working with data a lot, querying SQL and other stuff. But then I found that my passion is in actually finding interesting insight in the data from report building and back then we didn't call it, but it is more of an analysis analyst job. So I then transitioned into an analyst role about 12 years ago and learned machine learning from Andrew ng's video on YouTube which is awesome. I loved it back then. It was a little bit different, more lengthier and more interesting I guess 12 years ago. But then yeah, I learned a lot from deep learning, deep learning AI courses. I transitioned into head of Data Science at Australian Computer Society where I help different functions to deliver data science and analytics projects. And also I am an AI expert at the ISO International Standard Organization helping to contributing to standards in AI trustworthiness and data sharing. Apart from that, I sometimes write for Forbes and help people learn about AI and data science on LinkedIn.

Man of multi hats. So thank you all for joining us today. So for the next 40 minutes or so we're going to be discussing topics like especially the main differences between these two roles, how you might shift from one to the other and so on. So to start off for the audience who isn't familiar with what a data scientist or machine learning engineer does, can each of you use one sentence to describe what these roles do in your own words? Maybe Aishwarya, you could start us off.

So one of the simple explanations that I can think of is as a data scientist you need, you not only have to have good grasp on coding and math and statistics, but you also need to have a very good hold on subject matter expertise in whichever industry you're working. Whereas in machine learning engineering your focus is more towards coding and as well as math and statistics, but you need not really know so much of subject matter expertise.

I'm assuming I'm next Sandhya, since that's the order we took. So I'll just jump in here. So essentially I'd say a data scientist is someone that can apply the scientific method to data, right? They conduct exploratory data analysis, they get insights, they, as Aishwarya mentioned, they interface with the business. Whereas a machine learning engineer is basically someone that takes the models that are built by the data scientist and makes them work in a production environment a lot of times at scale. So those are the main differences there.

And then Steve, Awesome.

Yeah, you guys are making my life so easy. On the last and I pretty much agree with all of it and as Kate and Ashwin mentioned, so I guess data scientist is more of a scientist and the scientists do more experiments is more of a creative, I guess role and the other one is more of an engineer which engineers are trying to maintain and operate things. And that would be a quick one.

Fair enough. Why do you guys think that the concept or description of these two roles is still a little vague in the community? You know, it's been almost 10 years that we, these, these terms are, have been, you know, used in the industry. Why do we think it's still a little bit vague? Steve?

Yeah, so the reality is many jobs right now, they evolve and change. It's not only about the data science or AI roles. Based on the new innovations, the customer needs and all other parameters around the job itself, they would evolve. And that's the same with these roles. When they started it was more of a generic terminologies and then a lot of complexity was added into these roles. Many other, I guess other roles came out of the data science term, like the data professional terminology and it evolved to cater for the new world and it will change again as well. Also the complexity of each one of them just makes it wake because then each company based on the size, based on the, I guess the needs and the industry that they're performing, they might need a little bit of different capabilities or different requirements for from each one of these roles. It's not like ISO will come up with some sort of standard for the name or for the terminology and everybody going to follow it. So I totally agree. Still nobody knows exactly what each one means.

Fair enough.

It's definitely one of those things that's evolving as the industry itself evolves. Fair enough. Can you give us, can one of you give us some examples of what each role does in a typical workday, maybe? Aishwarya sure.

So just adding on to what Steve mentioned, that there's this little blurriness and 10 years is probably not a lot of time when you think about like a new field emerging and these different companies are having their own sort of definition for machine learning engineer and data scientist. Some of the companies are requiring you to have research experience for being a data scientist and some of them are not. Some of the companies, for example, my previous role in data science was a mix of both data scientists and machine learning engineering where I was also like working on productionizing the models. So if I were to compare it in an ideal situation, what data scientists would be working on is building the models, getting their hands dirty with data, understanding how are they going to formulate the business statement, how are they going to help in a particular use case and how would, how would they like, you know, dig deeper into data, make sense out of it, see if there's any data quality issues, see if there's any bias existing in the data, basically work. Put most of their time on the data, understanding it and choosing the right algorithms, optimizing the algorithms and building those and testing them as a proof of concept that that's what typically a data scientist would be working on. So that's inclined towards research, inclined towards going deep into mathematics and statistics. Whereas machine learning engineer, in a typical sense would mean that you're picking up what the data scientists have built and you're trying to scale it, you're trying to productionalize it, you're trying to get it into an organizational workflow. How does the models that are being built by the data scientists reach to your tips in your phones or your computers where you're actually using them? So they are the ones who are getting it into the, into the, you know, workflow or the ontology of an organization where companies or other users can actually make use of these models.

So do you see data scientists as sort of mining the gold and machine learning engineers are the one who make it usable to others?

Yeah, usable, absolutely. Yeah.

Fair enough. Kate, could you chat a little bit about what you think the typical career progression for each of these roles is, having seen so many of these folks in your community?

Yes, absolutely. Happy to talk about that. I think in most roles, including in the data space, there are typically two ways that you can take your career. There's a management path, and then there could be an individual contributor path. And I see the same for the data scientists, for example, where you typically start as an entry level data scientist or maybe even an intern, and then eventually you grow through the ranks and you become a senior data scientist. At this point, when you're in that position, you can really start to think about do you want to become a manager of data science teams or do you want to remain like this highly specialized data scientist where you can focus on your individual contributions. Now, as you go up from the path from that entry level to that senior level data scientist, you tend to develop more skills, you get more responsibility where you used to be maybe managing a specific process in that whole data science process. Maybe now you're looking at things more end to end. And more importantly, you're also learning kind of how to work with that business and solve business problems. Now, for machine learning, I think you typically start out with learning a programming language and thinking Python would be popular with our audience here. Learning some fundamentals of computer engineering, like data structures, algorithms. And then it's really helpful if you're starting out with an undergraduate degree in something like computer science. Computer programming. I know back in the day we didn't have things like data science, but now it's really, really available for us. And then some roles that basically machine learning engineers start with include, you can start as a maybe software developer or computer engineer and similar roles and then you can progress into a machine learning engineer. Now individuals in this space that also possess strong leadership skills, they can also look into management positions such as those that I mentioned in the data science space. And in addition to getting a job for both in the data scientist and ML space, I think because their demand is so high, you don't have to commit to just working for an organization. I think there are plenty of opportunities there to do consulting, freelance or you can actually teach the next level, you know, the next generation of data scientists and machine learning engineers. So plenty of hope out there for those who are looking to learn this stuff.

Awesome. I mean many of our audience you spoke a little bit about sort of, you know, the initial phase, I would say like many of our audience members are, you know, thinking about starting a career in either data science, machine learning, trying to figure out which one is more suited for, you know, their interest and their passion. So to help them figure out which track is a better fit, there are some critical skills that we think are required. But you know, we'd love to understand how those skills are, you know, required for each role. So I'm going to name a couple skills and if you guys want to pipe in and chat a little bit about, you know, how, where you think you know, is more relevant. So how much math and software engineering do you think is actually required.

So.

I can answer this one. So you know, the reality is a lot of people ask me about which one of these roles are more interesting or more important, which one is suitable for me. So first of all, the answer is it's based on each individual people's preference, skills, plans for the future. There is no answer about which one is a better job, no such a thing available. Okay, so if you want to choose either of them, spend some time like listening to these kind of events going to help you to understand what entails and if you are up for the challenge. Secondly, the industry that you might want to join is also important. There are different requirements in different industries and definitely I guess if you're more techy person you would like to go to these techie companies that will do some more hardcore stuff. And if you have some background knowledge about specific industry like finance or biomedicine, then you might want to opt for one of those industries which they need some business understanding, like heavy business understanding. Apart from that the technical skills or the skills that we can just quickly mention about each one of these roles, I would say as we already defined a little bit each one of them. We can see that data scientists probably need more of hands on with data. Maybe understanding SQL is one of the major skills. I know that my friends always share About SQL on LinkedIn and everybody loves it. So that's SQL is one of the major ones. It's not that, believe me, if you are starting it, spend really good time on I guess this particular one. But then mass and stats is something that probably sits across both of them and maybe heavier in the data science side where the science comes to the I guess to play the role. And the visualization part is also important, more important for the data scientists. I guess many of these skills are important for everyone. For example, visualization is something that no matter what you do, you want to present some sort of data in an understandable way. You need to have some skills. But then maybe a deeper understanding is needed for a data scientist. But on the other hand, machine learning engineers, as Shwary mentioned, is just more of a software engineering kind of skill that you need to make sure that you understand object oriented programming a lot more deeper than the data scientists and maybe hardcore coding skills. And this would be a major difference for me. GitHub is also something that you need to work with it a lot. And also using IDEs that are more relevant for software engineers versus that probably data scientists would use jupyter notebooks more.

Great ideas. Yeah, I love how you're distinguishing between even just the tools that the individual roles use. Kate, could you chat a little bit about what you think comes from, you know, like sort of minimal educational qualification? You know, if one role is more oriented around research and another is more oriented around say business insights like how do the educational qualifications play into that?

Yeah, I think, you know, the answer used to be you have to have a programming degree, a computer science degree or physics, statistics, something of that sort of. But I think we're slowly moving away from that where you'll see some of these big tech companies saying you don't need to have an undergraduate degree in anything relevant or even a degree to begin with because they value experience so much more over your actual credentials. Can you get the job done? That's literally what most companies care about. I think you'll still see companies using those credentials to maybe weed out applicants because if they have a huge influx of people applying to a job, they'll need a reason to kind of narrow that list down. And I think that's where that might become useful. But I think the more you can highlight your Project experience and showcase exactly what you can accomplish and the skill sets that you have, I think that becomes a lot more powerful than maybe a degree. I'm definitely not swaying people against getting a degree. I think it does become a lot easier if you have that foundational background where you don't have to relearn all of this stuff on your own. But we are living in an age where you can take boot camps, you can read textbooks, you can go online and Google information, which is what most people end up doing is googling information when they're stuck. So I do think there's hope for those that might not have a technical background.

The one thing that every engineer needs to know is stack overflow, right?

Yes, exactly.

So I think one of the distinguishing aspects of engineering is not just sort of, let's say creating the, or manipulating the program that you have, but sort of, you know, putting it into production. So the sort of the MLOPs as we call it, the operations skill set of scaling, of putting it into large, you know, real world situations. So Aishwarya, could you chat a little bit about how much you think a data scientist versus an MLE needs to know about that MLOps skill set?

Absolutely. Again, like pointing out to the fact that every individual company has very different kind of roles and responsibilities for a data scientist versus an mla. For example, in my team when I was working as a data scientist, we didn't have separate people doing data engineering or data visualization or ML ops. So a data scientist was another word for like a full stack engineer is, is a unicorn in data science where the same person is able to do the entire pipeline of data science and also has the subject matter expert expertise in different fields. So as Kate was mentioning, MLOps is something which need not be necessary for any, for any particular role unless and until it is specifically mentioned in their job description. But what's, what's sort of important in certain data science roles is that subject matter expertise. For example, like in the recent times where we have seen multiple companies facing issues with respect to ethical AI, we have more awareness being created that on the team where we are building these AI systems, where we are building these machine learning systems, we need people from diverse backgrounds, we need philosopher, we need, we need people who are expert in psychology so that they can give us a better perspective on what could be an ill effect of this kind of a system. And that's where I, that, that's where I wanted to say that you know, mlops is something which is very specific and it could or could not be absorbed in a role like data science in certain companies. It is absorbed in certain companies they have specialized people for data engineering, they have specialized people for doing ML engineering, they have specialized people for even doing database. So yeah, it's again like I would encourage people that whenever you're applying for a data science position, do not just look at the title of the role and dig a little deeper into okay, what's the job description? Go on, look for people who are working in same team or similar team in the Same company on LinkedIn and reach out to them them, talk to them about what is their day to day look like. Do they engage in mlops? If you're interested then you can go ahead. If they're not, it's your decision then.

No, that's a really fair point. We know that there's a lot of people who come from non traditional backgrounds, like some non cs, non engineering people like myself with psychology who are like hey, how can I be perhaps not a machine learning engineer but still in this space in this field helping develop these sorts of AI ML based products. So how would you advise someone like that who's coming from that sort of non technical background to sort of get their feet wet and kind of enter the industry?

Absolutely. So just as an example, like people who are developing web applications, it's not just about the back end that you're working. It's not just how you create the website, it's also about how do users interact with it, how attractive is it, how easy it is for you to use. So all that thing come in a front end or like a UX designer. Similarly we have various different components when it comes to like building data science or building machine learning models. So when, when you were saying that you know people who are coming from different background compared to cs. Even in my team there are people who are coming from operations research, people who are coming from health care especially who create a whole lot of difference for us. For example me, I'm coming from a computer science background and when I was working on a healthcare project we had this other person coming from a healthcare background, from a neuroscience background and could give us so much more perspective on what the data means. One of the, one of the things which Andrew has been recently like pushing a lot on is having data science data centric approach and I cannot emphasize more on that, on how important that is because we are trying to stitch something up without knowing what's the content in it. And that is one of the major reasons of some of the failures that we have seen in AI system systems that we don't understand the data well and that's where it's super important that every company does engage people who are coming from a certain background in, in a different industry. If the company is you know, working on building automobiles, they have people who are coming from mechanical background and not just data scientists building, building these self driving technology because yeah, like as a data scientist I could build something without knowing that what could be, what could be thing that might go wrong in the mechanics of it. I would not know like how long does, does the car take to stop when I'm breaking it at a certain, certain pressure versus the other. So things like that require people coming from a subject matter expertise and people who are experts or like who have backgrounds in these various specialized fields are most welcome in the data science field. It not they might not be building models in Python R or like other coding languages but they are absolutely very essential to be on the table.

No, that makes a ton of sense. We do a number of these types of events and so recently, earlier this year we did an event with our sister company Workera which talked about taking your industry expertise and adding AI in order to move your career into the next phase. And obviously Fourth Brain was really founded around this idea that people come from such various different backgrounds and that enabling more of them to more readily access the type of knowledge getting them to the point where they can take a data scientist role and let's say bring their expertise in retail management to an actual retail company that is now trying to adopt AI practices like that is how we're really going to increase adoption of AI in a safe and ethical way to your point. So yeah, really love that are definitely big concept that we think about at AI fund, at Deep Learning and in our ecosystem overall. So I mean obviously there's a ton for getting people who don't do coding into kind of introducing, introducing them to AI. But there's a large number of people who already do have some of that technical background. You know, maybe they're a computer software engineer or something like that that are wanting to transition into data science specifically related roles. And so in your experience Steve, I'd love to hear what do you find is the most critical missing skill or missing knowledge of the candidates who you know, considered such a transition. Is it, you know, is it statistical expertise? Is it more on actually the software engineering side? You know, what do you find is the gap?

Yeah, that's a great question. Because AI and data science are becoming more interesting and trendy these days a lot of people are thinking about upscaling, transitioning or adding them into their own skills right now. So what is happening is some of them are jumping a little bit too fast into this and I can see based on the background they're missing some of the elements because they want to make sure that they quickly get up and running and they find a job in a relevant to data science. So they want to fast track things. Right. If they have done four or seven years of education in something else, they don't want to miss another five and seven years to learn data science from scratch. Going to university. What happens sometimes they pick the shiny things like machine learning, you know, connecting a machine learning algorithm to some data and get some results. And that is something that actually is misleading. I remember I had a class at university, I used to be a lecturer and I had a lot of students from different backgrounds just jumping into advanced machine learning course like the subject to learn machine learning on the final subject of the master of data science course because they thought that's pretty much what is needed. And I remember I was asked once that like, like, is that it? Like we just connect some data set to an algorithm and get it trained and that's, that's called data science. And why even that that is a thing. And the reality is it's not like I just told them like this, the class is showing you some of the stuff that you might see but the reality is we are not showing them the, the real world problems. The data sets are not actually that clean, that easy to find, to put together and they miss a lot of this when they're learning. So what I would say based on their background, maybe they miss some of the data related skills like the cleaning, the munging and the ETO part. If they're coming from very, I guess, strong programming background, they might miss the stats and math to understand what is happening in the algorithm where they might think oh that's a black box, we just need to connect it to the data and wait to see what comes out of it. And then we might want to connect it to kind of auto ML to optimize the algorithm to become more accurate. And that's something that I would see a lot within the new cohort generation jumping into data science.

That's really interesting. Yeah. So let's say for someone who's currently a data scientist then, and you know, maybe they may do a little bit of machine learning modeling but then they want to explore that potential career path in proper machine learning engineering. What would be the roadmap or maybe the resources that you would recommend them to make that type of a transition. So it's like if I already know how you know the data manipulation part, how do I learn the rest?

That's also depends heavily on people. So as I said, like I share a lot of learning resources on LinkedIn and I always say that there is no one size fits all. Like some people love reading books. I don't. I personally would rather watching Andrew's videos on YouTube and Coursera. That's, that's my preference. But, but that's the, that's the, that's the reality. Like people have different preferences. Some of them would like to listen to podcast, some, some of them might want to read different modes of learning. And, and that's, that's the reality. But what I would, I would say that you need to make sure that you get the latest and easy to understand information and resources from wherever it's possible. At this point it is mostly available on courses that are available on different platforms and also the videos because it's a little bit more like a faster way to generate these resources. If you think about books, it's. It takes time and sometimes you might miss the relevancy and some of them might become a little bit outdated, I guess. Blogs are also very interesting. I personally like blogs. They are much more quicker to read and pick up. And finally, I would say if, if you just search under Google, you might find a lot of results and you might get puzzled. It's better to find some people, like people that already know about it and get some guidance or follow, you know, people that share these kind of interesting insights that are handpicked and curated on different platforms like social media.

I think that you add to that.

Yeah, I just wanted to add. I definitely agree, Steve. Everyone does have their own methods of learning and I think for a lot of people it might be a combination of every. All of the above of what you just listed. Right. I think the flexibility of reading a book. I actually have a book here on machine learning engineering. I happen to have Andrew Berkov's new book that I was skimming through. There was an interesting forward by Cassie Kajorkov there. But anyways, there are so many ways to learn and I think right now people who are trying to break into this space are not at a lack of resource. Right. We're not looking for, oh man, I wish I had a book or I wish I had a course. It's more of the, oh my God, I have a Thousand or more courses that are reputable that I can choose from. So I think at this point it's really about going after the method that works for you, but also looking for maybe a program that is structured around getting you to your personal goal as quickly as possible. So if your goal is to be hired in the next four to six months, maybe find a program that says we'll have this structured program, here's what we're going to teach you and we're going to help you find a job or help you maybe spice up your LinkedIn profile to really prepare you for jobs and interviews. So I think you really have to ask yourself, what is it that you want? Do you want to become the best data scientist, the best machine learning engineer? Do you want to get to the job as quickly as possible? Do you want to just start slowly transitioning while you still have maybe another job in the business or in another space? So it all starts with finding out what you want. And then as Steve mentioned, there are plenty of, plenty of resources. I know Matt Gore from 4th Brain would appreciate the plug here, but 4thbrain AI does have a cool looking 16 week program so I encourage you to check that out. I was looking at it myself actually, just for fun, to see what's covered there and it looks like they do give you some hands on projects and some of that support. So find programs like that. I think that's really helpful.

Yeah, for sure. Fourth Brain does a great job of sort of, they have one program that focuses on sort of the, let's say the fundamentals of machine learning and kind of what that means. But then they're also launching a new program this summer that is oriented around mlops. So for the folks that do already have the data knowledge really are trying to learn the scalability and putting into production and sort of those, those engineering skills, if you will. And so that's a new course that they're launching, I believe next month or maybe later this month you can learn out. You can check out more@FOURTH BRAIN AI if you're interested in that. So moving on to our next question. So with the lines blurring right between Data scientists and MLEs, sort of, you know, as Aishwarya said, sometimes a data scientist, this is the equivalent of a full stack engineer. So because of these blurring roles or blurring lines, do you think that someone could potentially be both of these or do you think that they still have relatively distinct positions that you don't see combining later on?

Kate?

Yeah, I think they're still Distinct enough. I know you mentioned that they might, might have been around for about 10 years now. I think that's not a very long time. Ten years, I guess it could be for in some areas I think they're still distinct enough where I do see two separate roles. And interestingly I have a fun example of another area that we can think through where we can see these differences. So if we think about a nuclear scientist and a nuclear engineer, they have very distinct roles. For example, a nuclear scientist has to know the science behind the atom, right? They have to write the recipe for how do we actually extract energy from the atom. They have to know the specific and various interactions. But then you have the nuclear engineer who doesn't have to know all of that, right. They have to take that recipe, carry it to the world and you know, they have limited knowledge of the actual atomic physics, but they have other knowledge such as, you know, the materials that are used, economics and everything else that it takes to build that nuclear power plant. So I do see them as similar to that space. Not that I'm a nuclear expert by any means, but I think it helps when you have an analogy and you think through that. So yeah, I'm going to go with two separate roles, but who knows what happens in the future, right?

For sure. And I think as the field evolves, you'll also see the nature of roles change. I mean, 10 years ago, social media marketing wasn't a thing, you know, and for that matter, neither was content marketing. And now those are both, you know, entire departments at some large companies now. So definitely something to think about. As, you know, the specialization will that will happen as the industry matures. But this also begs the question, you know, will these, the tools and the skills that we think of as data scientist skills, do we think that they will become so fundamental to the way that we, you know, produce technology and produce software and programs and platforms that, you know, they'll get dissolved into, you know what software engineers do it. At one point you're a software engineer or you were software, you were a software engineer who did cloud things. And now, you know, that's. Those are no longer a distinction. Those are the same thing. You know, if you're a software engineer, you can't do cloud. Everyone kind of looks at you like, what time travel device did you hop out of? So, you know, do we think that data scientist roles will get dissolved into software engineer roles in the future?

I sure, yes, absolutely, yes. So one of the, one of the things that I see is, you know, probably back when in 2013, when I was doing my undergrad, Software engineering itself was like a specialization. People used to, you know, not everybody knew how to develop software, not everybody knew how to code in Python, R, et cetera. Whereas now I am appalled to see like books and like courses being published, you know, software engineering or like coding for kids and all. And it does like, you know, surprise me a lot that people these days want their kids at the age of five to learn how to code. So that's, that's a great example where we are trying to see that now. We are, when we are seeing this technology grow and we have all the, all the kids and like people in their junior school using phones and laptops and tablets and whatnot, other gadgets, they might as well know how that works, right? So that's where we have transformed on seeing how like software engineering itself or like coding itself have become, has become mainstream. And people from very different backgrounds now do they, they do know how to code. And the only place where I see data science becoming merged into software engineering is where we have more and more packages like scikit learn, like, like stats, models, etc. Like where we have all these models sitting inside these Python libraries or like other libraries, which is now available to you at your fingertips. It takes you two lines of code to run a model on a data set. So that's how the ease of use has become. And that's where I feel like data science is going to be integrated in software engineering. Because none of the industries are spared from data science. Like almost every industry has been using data science similar to what we see with cloud. Every industry is transforming their businesses into cloud. Similarly, every industry is transforming their businesses into data science. So what might still be intact is the people who are building these models because there still has to be research going on, on building better models, building optimized models, as in when we see challenges with respect to scalability, we might see such challenges coming as well in future. Like where we move from data to big data to cloud, that's how we'll probably move towards some other challenge and we'll have a new technology to tackle. That's where like data scientists who are building the optimized models who are building the, the core of IT are still going to be there. But use of data science is going to be mainstream. Like everybody is going to start using data science in like, pretty much in their like day to day jobs.

That's, that's a, that's a really interesting feature. Like, like the difference between the world before everyone got a smartphone in their hands versus after has dramatically changed just the way we go about our day to day lives. I'd love to see what happens when AI is sort of enabled in the larger public now with pivoting a little bit more with model building now becoming more automated.

Right.

You spoke to more advanced packages, more complicated systems that are available for use is, you know, do, do you guys feel that data science and machine learning are moving to become more focused on say business intelligence rather than technical involvement? So you know, are we, do you feel that we're going to move towards, you know, specialized machine learning practitioner? So you know, maybe a data engineer and who focuses on data processing and then somebody else who focus, you know, a data scientist that focuses on the specific types of research and then software developer or maybe you know, ML Ops person who, you know, focuses in that space. Or do you feel that we're going to move towards, you know, end to end generalization in the sense that, you know, there's going to be someone who's kind of expected to take it from hey, I have a small idea all the way through to hey, this is how, you know, the model now runs and provides insights to my, you know, business for Steve, I know you've, you've thought about this a lot. You know, would you like to share your perspective?

For sure, Sandy. So to be very honest, I would say both of them and that's, that might look a little bit crazy because as a general rule of thumb, when kind of special technology advances, you might have more specialized people to deal with that because of the breadth and depth of the knowledge. Right. So if you think about it, probably doctors used to be just all, all doctors and then maybe a couple of hundred years ago they become specialized and then now more specialized in the subset or very specific part that is just very complex. So that would happen similarly to data science. We have a lot of questions, we have a lot of problems that we still don't know how to solve it. I can just give you some examples based on the work that we are doing at ISO. Explainability, ethics, they're so complex, believe me, we're still, we have no idea how we're going to solve these challenges. And from one perspective it's like you connect these, you connect these things, these algorithms to specific libraries and they will give you some ideas about how these models are being built and how they come up with prediction. But that's not the case and it's not the reality. It's a lot more deeper than this. And we need a lot of people to actually just think about those problems. But then with the use of lots of AutoML and platforms, we might end up with some sort of citizen data scientists or software engineers that will be able to leverage those automated tools to deliver end to end results for smaller organizations. Like if you are a startup you want to quickly deliver something, a POC or MVP that is also possible using some tools. It might be similar to making a website these days. Like you can use WIX to quickly make some sort of website, but then you might want to write it from the scratch for some very complex projects.

No, that's really interesting. I mean I've always thought of industries as they mature, they become orchestras. You always have specialists, right? You have your violinist and your violist and your cellist and you know, et cetera. But at the end of the day you also need some generalists. You need a conductor who kind of knows a little bit about everyone's thing to sort of go, okay, this is how we're going to make music together. So definitely see the need for votes. I love that perspective. So a very last question before we head off to the slido. So we're going to do this rapid fire. What industries do you see the most need for and growth for in the use of machine learning engineers and data scientists? Aishwarya, you want to give us your top one or two?

For me I think the two most important ones are healthcare because that's one of the crucial places where we are playing with lives and that's where we need most research. That's where we need most most. Like better technology being served in that space. The other one is again like coming out of one of my experiences with United nations where we were working on like helping the climatic change and helping on how we can use the data and set up timelines for ourselves and pull up more initiatives which can help, you know, help us with the sustainable development goals. So these are probably two of the areas which is like I'm really fond.

Of healthcare and climate change. Excellent Steve.

I would say the ones that have more data would be probably more in the most need in future. Healthcare, education and finance.

And then Kate.

Yes.

So I don't think there will be any industries that are left untouched by the impact of data. But I will, I definitely keep my eye on financial services because that's where my background is and I kind of have most of my domain expertise there. But I definitely agree with both Steve and Aishwarya that healthcare is one of the most important areas because we can literally save lives. So I'm definitely keeping my eye on that.

Sounds good. And if you guys are interested in hearing more about the intersection of AI plus healthcare, we had a recent event where Andrew, our founder, spoke with Fei Fei Li from the Stanford Hai team and had a fantastic conversation about, you know, the sort of how AI can actually help in healthcare. So if that's something that you're interested in, please be sure to check that out as well. So now we're going to head over to our slido questions from our audience. So the first one is from Edgar. Do you think that having a PhD or academic experience is an advantage or maybe even a disadvantage in the sense that you might be overqualified for an industry job?

Sorry, you can go ahead.

That's all right. Sorry, yeah, I can have a quick one for this one because yeah, as a lecturer I used to get a lot of these questions like are we in the right place to learn data science? Is universities still relevant? And so the answer is yes and no. Again, sorry for being like this. I don't want to make it complex, but the reality is you will get a lot of softer skills at university that you will not get it online just looking at your camera. Right. You need to work in a cross functional team. That's kind of a learning plus, I guess, job experience at the same time. So if you're doing Ph.D. and you want to be a data scientist, you definitely are learning technical and softer skills at the same time that will help you. It's not 100% necessary to start with that, but I don't dislike having kind of an academic experience that will bring a lot of other things more than just understanding of the technical.

If I what Steve just mentioned. So this is a dilemma which I had when I finished undergrad and I wanted to pursue my higher education. I was confused between going for a PhD versus a master's and it was a safe decision for me to choose a Master's because that was something I could extend and convert that into a Ph.D. so that, that's what I went with. And while I was in my school, I realized that I am sort of a person who likes breakfast, who likes varied applications of data science, who likes to explore a lot and read something about reinforcement learning, read something about federated learning, try multiple different use cases in different perspectives. And that's what motivates me. And again, like as Steve said, the answer is yes and no because it depends on what you want. Like there are people who really want their focus on Say natural language processing or computer vision or robotics. And I have friends who have been, you know, dedicatedly working on robotics to build arms for disabled people. And that's, that's where you have to find your motivation on what drives you is, is it something which depth drives you or does breadth drive you? And I think like my attention spam is probably of a fish and that's, that's how I get excited with everything around me. So when I'm, when I am on LinkedIn, I'm just super excited to read about like so many different things that people keep posting about. And I love learning from there. And somewhere I feel the experience that I have gained in the last two years as a working professional is much more worth for me what, compared to what I could have gotten if I had gone for a PhD. So that is just my personal experience on how my learning curve works. So yeah, that's my answer to that.

Fantastic points. Next up, we have a question about a something that's a little off our main topic today, but could one of you explain what you think is the difference between a data engineer and a data analyst?

I mean, in the simplest terms I'd say a data engineer is someone who lays down the pipelines of data, someone who can understand where the data is flowing from and making sure that everything is structured perfectly. Well, it's never perfect, right, but as close to perfect as possible. And I'd say a data analyst is someone on the very other side of the spectrum who actually takes that data and analyzes it, maybe pull some insights, build some data visualizations. I'd say that's probably the simplest answer on that question.

Fair enough, thank you for that. So one of the biggest stumbling blocks in order to start a career in this space is, you know, coming in without prior experience. So, you know, let's say someone's, you know, learned the material, studied it, etc. But hasn't quite had the opportunity to, you know, apply it in a professional setting. How would you recommend that they sort of take that step, kind of getting that hands on practical experience.

So as an example, like when I was in school, I have, I had done a couple of internships or you know, capstone projects with companies. And one thing that I realized that there is only so much of real world experience that you can get with these internships or with respect to like projects that you're going to do on Kaggle, etc. Because there's a whole lot of different ballgame when you get into an industrial setting and where you're actually tackling with situations where you have either too much data, you don't have relevant data, you have bad quality data, and some of the situations where you don't even have data, like you don't even know where to pull that data from. So those kind of situations, I think you have to get into an industrial setting for you to get that exposure. But companies do not really expect you to have that exposure before getting into an industry. Like, if you're a fresher out of grad school, I think having enough experience with use cases which are closer to real life use cases is good enough. Where for example, you can say that, hey, I'm working on a social media data and I'm trying to extract certain sentiments to predict something, or I'm working on a prescriptive analytics project, etc. If your thinking process is something which is business oriented, which is problem solving oriented, I should say not business oriented, problem solving oriented, that is enough. Like where you're trying to understand that, okay, what am I trying to solve? So then you reverse engineer on how you, how you're going to solve it. The first question as a data scientist should be like, what am I trying to solve? What am I trying to achieve out of it? Before digging into, like, how do I go about this data?

Yep, you got to know where you're headed before you chart a map. Makes complete sense. So next question that we have from the slido audience that we're going to skip is about the difference between MLE and mlops. We actually have an event with Andrew as well, a number of our instructors of the mlops program coming later this month. So please audience, keep an eye out for that. If you want to stay up to date on the latest in our events, make sure you sign up to the batch as well as other communication from us on our website, DeepLearning AI. But we will, you know, we'll happily address that question in that, in that session. So now moving on to another next question is which of these two career paths do you think allows for more creative freedom?

I can kick things off here. I think I'm more of a creative person than a technical person. And I think the data science role probably allows the individual to be a bit more creative and explore different areas. Right. Because we get to work with different data sets, you get to visualize your data. Not sure if you can get even more, you know, I don't think what, I don't know what gets more creative than visualizing data. That that's literally my passion is Telling stories of data and visualizing data. And I think the main reason is because you get to be creative, you get to pick how do you visualize this, what charts do you use, which colors do you use and which methods are the most impactful and effective at telling those stories. So I'll definitely go with data scientists, but I'd love to hear from others which one you think is more creative.

100% agree with you, Kate. I do believe that data scientist is going to be more creative and playing with the data is just something that you need to have that curious mindset and creativity to deliver some results.

Absolutely, I would agree to that. One of the examples that we saw for machine learning engineer, where you're productionalizing it and that sort of becomes monotonous after a point because you're not getting too much into how the data is working or how is it going to affect impact you in a modeling perspective, but you are mostly concerned about the packaging of it. So I would also like go with data science where you are more creative on how you're creating the models, how you are developing, how you can extract more and more information out of the existing data that you have compared to doing the packaging out of it.

Makes sense. Yeah, I, I wouldn't say that I can speak to whether one is more creative than another, but I do think that one of the hallmarks of sort of career progression is your ability to be creative within the guidelines or limits of your particular role. You know, you could be an exceptionally quantitative type role in marketing, but there's still ways to be creative. And similarly, you know, I would encourage folks to think about themselves like if, you know, if you want to make something, if you want to be creative about how you do it, like even an engineer does have an opportunity, they can't quite, you know, say mechanical engineer, can't quite defy the laws of physics, but you know, they can definitely think about new ways to approach problems. And so I love that attitude. So what we will do is wrap this up today. So thank you all for your insightful discussion and thank you speakers for joining us today. This brings us to the end of today's event. We hope you enjoyed it. We're going to set up a follow up email with a survey for all of our attendees today. We'd love to hear about how to make our future events better. We're also giving out a 50% discount code for any of our online courses to up to 200 people who submit qualifying responses. So keep an eye out and make sure to send in your feedback. And thanks again to our co organizer 4th Brain for their support for today's event. We'll be collaborating with them for more future events, so if you don't want to miss them, make sure you sign up via the link in the description box below. And don't worry if you can't make a live event. We always record and make. You can always find these recordings on YouTube. So we hope you walk away today feeling inspired about your new next career step and we look forward to seeing you next time. And as always, keep learning.

Thank you so much Andrea.

Thank you.