Roles to play in the AI dev workflow

I'm going to say something slightly controversial, I think, and that is that I think of AI development as a component of software development, which a lot of data scientists will say, no, it's not. No it's not. But when I'm looking at it in production and I'm looking at us actually managing that, I see it in that larger context because all of those other activities are happening around it. Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at changelog because of rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. Digital Ocean's developer Cloud makes it simple to launch in the cloud and scale up as you grow. They have an intuitive control panel, predictable pricing, team accounts, worldwide availability with a 99.99 uptime SLA and 24.7365world class support to back that up. Digital Ocean makes it easy to dep, scale, store, secure and monitor your cloud environments. Head to dot co changelog to get started with a 100 credit again. Dot co changelog.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community and follow us on Twitter. We're PracticalAIFN.

Welcome to another fully connected episode of the Practical AI Podcast. This is where Chris and I keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss some of the latest AI news and we'll dig into a few learning resources to help you level up your machine learning game. I'm Daniel Whitenack, I'm a data Scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a principal AI strategist at Lockheed Martin. It's been quite a season in our lives, Chris.

Oh boy, 2020 has definitely had an impact on my life.

Yeah, definitely. I think we would not be right to just ignore everything that's happening in our world as we enter into these conversations. Of course we've got the unrest that's really happening in our country, but around the world as a result of injustices and police brutality and sort of systematic racism that's happened in our country but also around the world. And then that kind of piled on top of COVID virus related things. And then that piled on top of the economic impact and fallout of that and unemployment. And of course these things are not separate from AI things. And I think probably over the course of these coming years, I think it'll be years of fallout from everything that's happening. Totally. You know, it'll impact our conversations, so.

It will.

Yeah.

It's all real life. And a couple of thoughts there, you know, you talked about the injustice of what's happening in terms of, you know, and Black Lives Matter being able to come back out and be meaningful in this discussion, which I think is fantastic. It's a time of change right now. It's a time of massive shift and I know it impacts everybody in the audience. I know for me you mentioned Covid and you and I had talked a little bit about before the show and so I'll share very briefly with the audience what's happened recently to me. I'm actually choking up a little bit. So. My mother in law recently died of COVID so it's impacted my family and just wanted to share that with the audience. I've been kind of missing in action for a little while. I know you did an episode with Darwin AI recently and I thank you for doing that. And I know with the unrest, we paused to show briefly, but I've been kind of out of action. Just wanted to let folks know. I know so many people that say they hear about COVID in the news, but it hasn't touched their lives in a direct way. And speaking as someone who has had it touched directly, it is a serious disease. And so I just hope everybody will follow the safety guidelines and be aware. When you lose someone that you love, it changes how you see it. When you have other family members that have it, then you know you're working on them. And when you have a whole family in isolation, it makes a difference. So stay safe. People appreciate it and just wanted to let you know it's real and it's touched my life and thank you for letting me say that. It was important to me.

Yeah, thank you for sharing, Chris. I know it takes a lot to share that as well. And I know my thoughts and prayers have been with you and, and your family. And yeah, I think it's just another data point to motivate people to, like you say, take things seriously. But also, I mean, I think the AI community, a lot of the people that listen to this podcast, there are many meaningful ways that people can contribute, whether it's on the COVID and virus related front, whether it's on the racial injustice side of things or the, you know, economic side of things. Of course, there's community things that we can all do, being good neighbors, being, you know, caring for people, but then also being tech people, being AI people. I mean, there are some real intersections with AI technology. I mean, of course, on the policing front and that side of things, we've seen, you know, increased usage of things like facial recognition and other things that are concerning for certain groups, algorithmic decisions that are impacting certain groups. On the virus side of things, there's a whole bunch of AI people that are trying to come up with beneficial applications to help that scenario. Not necessarily all, you know, predicting Covid outcomes, but, you know, helping people get the right information. We had the episode with the COVID QA group that was working on that. We also talked to, had an episode about the Cord 19 data set as related to Covid. There's ways that AI people can contribute both in terms of data annotation, in terms of coding, in terms of jumping into open source projects. So I think I'd really encourage people, if you're interested in those things or wanting to know how to contribute to those things, or wanting to know how to make your voice heard in terms of good AI ethics related things. And reach out to us on our Slack channel, our Slack team. You can find us@changelock.com community or on our LinkedIn page or on Twitter. We really are wanting to have some discussions around these topics and point people to good resources. So I'm really hoping that people reach out and find some of those ways to contribute.

You know, I'm so glad that not only did you bring up the practitioner side of being an AI professional or an enthusiast, but also the AI ethics side. And as, as we've talked about before, I'm very involved in AI ethics and so, you know, as we talk about injustice, both the technical skills that you have there and the incredibly deep, rich thinking that we hear from people in this community. You have a voice and you can shape the future. This is really something that we have a role to play in. So definitely I am asking our listenership to engage, engage with these issues in real life and bring your expertise and your skills to bear on this.

Yeah. And later on, normally in these fully connected episodes, we take some time at the end to share some learning resources. I've pulled in a few that I've run into over the years as related to bias and fairness in AI. And so we'll talk about those later on in the episode. And maybe some places where you can find out about some of those things. But before we get there, we do want to kind of acknowledge that there are a lot of encouraging and exciting things coming out in the AI community in terms of advancing various efforts and various toolkits. And that's one of the things that we wanted to do in this episode, was highlight a couple of those. The first of those that I saw, which really excited me was the announcement from Streamlit that they finished a series A of funding for actually $21 million, which is kind of crazy. If you remember, we had Streamlit on the podcast that was episode 66 and we talked all about the Streamlit project and everything. So we definitely recommend you go back and listen to that. But in general, I think Streamlit is an incredible project. I don't know if you've been following it at all, Chris.

Certainly after we had the conversation with the team, I found it incredibly inspirational. And Streamlit is an open source framework to turn Python scripts into interactive apps. And I know prior to us engaging them, I wasn't really aware of that, but it's a super cool approach and it's showing the creativity. So.

Yeah, yeah. So I know for me, I have no, well, I don't, I don't know if I want to say absolutely zero, but I don't have much exposure and experience in terms of like front end engineering or building actual graphical interfaces or web apps or anything like that. At the same time, often when, when you're trying to integrate a machine learning application into a business process, there's a very human side of, of that that becomes very difficult if you aren't able to let people interact with what you're building in a visual way. So I'm thinking right now, I attended a couple workshops recently on active learning and sort of human in the loop methods. You could have this scenario where maybe you're working on the workshops we were talking about translation applications, machine translation applications. Sometimes when you deploy that, you might want to have a model in the loop that tries to identify bad translations or something that your machine translation application is producing and then have a, a user actually review and look at those and correct them. And so you've got this kind of graphical piece, but also the user piece, the non technical user piece, potentially that's interacting with that. And so, yeah, I see that scenario, I think, popping up all the time. And Streamlit, I think fits right in there, which is why it seems like to me that it's getting a lot of attention is that there's an often seen pain point that isn't really well dealt with.

I agree. And not only that, but I think that our community often undervalues what that user experience should be. We tend to think we go straight to the model, we go straight to the training and talk about the latest algorithms. And when you're doing this stuff for real in production and you have a community that is expecting performance from you, not fulfilling the appropriate type of user interface and user experience really degrades from the work that you are doing in AI. If you don't either yourself or people you work with have those skill sets, you can lose the value in something that would otherwise be great very, very quickly. And as I've worked in a professional context, that point has been driven home to me over and over again. So I tend to approach AI from a user perspective, even if I'm the developer doing the work.

And Streamlit is talking about the ways they'll use this money, they want to extend the application. And I should mention too, this is open source application that you can use. So you kind of just PIP install, I think you PIP install, if I remember right back to Streamlit, and then run it locally. And they have a whole bunch of different customizations that you can add, like little sliders and text input and file upload and plotting and all sorts of ways you can configure it. And so I think that when they're talking about extending, of course, extending that and the customizability of it and customized layouts, they also talk about building a programmable state. One of the things I was curious about, of course, because I've always used Streamlit just as an open source application, is if they're raising money, they're obviously a business. And so I think the other thing that they're going to devote that effort into is the streamlet for teams, which in my understanding is some sort of sharing and a combination of like sharing and deploying in a secure way Streamlit apps that are actually sort of production applications and not just like demos or proof of concept sort of things or little tools and that sort of thing.

Yeah, I'm looking forward to seeing some of the other things they choose to do as they go into this new phase. So we may have to revisit with them at some point as they get some of this work done and they're able to use that capital. Well.

Changelog News is the best way to keep up with the ever changing world of software. We track, log and contextualize the coolest projects, the best practices and the biggest stories each and every week. Make changelog.com your daily destination or hit the snooze button and subscribe to our weekly newsletter that hits inboxes on Sunday mornings. Join more than 15,000 enthusiastic readers. It'll cost you exactly $0 and you can subscribe right@c chainsaw.com weekly.

So I know you had a topic that you wanted to go into, which I think is a good one. But before we do that, I just wanted to mention one other thing that actually just before recording today, I saw as I was scrolling through Twitter, which is GPU Accelerated Training now supported in Windows subsystem for Linux. And I have to admit I have not been a Windows user for quite some time, but in my understanding there are quite a few of them out there.

There are, yes, there are a few, yeah.

Quite a number. And I know, for example, like when I taught a couple courses at Purdue over the last few years, of course the lab machines there, Windows machines, or at least some of them. And so it was always a struggle for me to kind of figure out the best ways of like doing sort of AI experiments and programming in that environment. And mostly that's just my unfamiliarity with that whole world. But yeah, this is pretty cool. So I guess Windows Subsystem for Linux or WSL enables the users of Windows to run a native unmodified Linux kernel or Linux command line directly on Windows. So that's pretty cool in and of itself. But now I guess the step is that they're adding the GPU acceleration to that and you know, connecting up things nicely to Cuda and those sorts of things.

Yeah, and I think that's great. And like you, I have not recently been a Window. Once upon a time I was in Windows, moved away. But I've been hearing they're really embracing open source in recent years. And that's definitely brought me back around to being very, you know, I would consider them for a while before they kind of hit that approach, but so I total kudos to Microsoft for, for making that very hard. It's hard to steer a big organization in a very different direction, so been very impressed. I think it's a fantastic step forward to have the GPU support in that. And the funny thing is I keep running across Windows subsystem for Linux, being incredibly usable from people that are using it. And I work in an organization that has a lot of Windows users and so I'm getting really, really good feedback on the work they've done and being able to utilize that Linux kernel, it's not a second class citizen as I int understand it. It really does a good job. So now seeing that they have that support may change the landscape a little bit as that gets adopted over the next couple of years.

Yeah, I think that the like the sort of ability to run unmodified Linux things in Windows, that part sort of rings true right away for me. And it's cool that you could do the GPU accelerated stuff I guess in terms of my own workflows. Often, you know, I don't have the GPU like in my laptop or sitting on my desk, but I'm using it on either a remote computer or in the cloud. So in that case, if, if I was on Windows, I think the important thing would be this sort of command line stuff and scripting things and all that sort of things that I could do in the way that I'm used to. But I know also that people build a lot of great systems for also, you know, like gaming computers for example, that are Windows based. This is where my mind's kind of going with this I guess is that there's all of these like gaming computers out there with, with GPUs and games for the most part. And I'm also not a gamer, so I'm really speaking outside of my domain, but for the most part running on a, on a Windows system. So it seems like now this would make it maybe easier to buy a sort of off the shelf gaming computer or gaming laptop that's Windows, and then use the GPU on that for AI purposes. Whereas before maybe you have to like buy that and then install Linux and figure out all the drivers and blah blah blah blah. Maybe that makes that process easier, I'm not sure.

So I would agree. I'm not much of a gamer, but I think that makes a lot of sense. I actually think I'm probably going to try a Windows subsystem for Linux out in this context. So like yesterday didn't have a chance today, but yesterday I was logging into a dedicated DGX2 and I got all 16 GPUs for myself. And that was a lot of fun. Doing some work on there sounds like.

A lot of fun.

It was a lot of fun. And so I might have to pull out a Windows laptop and do the same thing. I did it from my Mac going in. But yeah, I think I'm going to give it a whirl.

You could write a blog post about Windows laptop versus DGX2.

There you go. There you go.

I figure There will be a clear winner, but it would be interesting to do the comparison.

Well, I can start on the Windows side and use that as a client and then log into the DGX and we'll use both systems. We can make that work.

Yeah, yeah, sure. Cool. Well, let us know. I'll be interested to hear from people if and when they start getting into this Windows mix of things. But moving on, I think you were mentioning a topic to me that I think is pretty interesting and oftentimes very confusing for people. And I know that we've touched on it before. You want to mention what you were thinking there?

Sure. So I do quite a bit of mentoring for people, not only at my employer, but just in general and people reach out and ask for advice and probably the thing that people ask about most often is they're trying to figure out how to orient their own careers on AI. ML Focus I've been pretty open that I came from the software development world and reoriented my own career some years back on this and it's completely doable. I think it's a myth that everybody in AI is a data scientist. I think it's a myth that you have to have a PhD or some other, you know, university based experience to get into this field. Certainly not the case. None of those are the case for me and a lot of people that I've worked with. And I think in a previous episode, I don't recall which one, but I mentioned the fact that because I've been in my career now for, I don't know, 25 years, ish in that, in that frame, I was around when the web was taking off. And that was the early part of my career was when the web went from the Internet with no web into the web. That was initially just academic and then took off. And I have observed as we've gone through this AI revolution that it follows many of the same trends of a brand new field that is exploding outward. And in the beginning people thought computer science was the thing. You had to have a computer science degree to do that. But we rapidly one role changed into many roles very rapidly. And there was a lot of diversity that got introduced as well as the skills you needed, the level of experience to do different roles. It got complicated and that's good. It's a sign of maturity and we're definitely seeing that in this field. And so a lot of people, when they're trying to figure out how do I do this, how do I fit into this new exciting AI world, that's where I really want to be in the years to come. But that's not where my education has been, that's not where my previous experience has been. And one of those things that I start with, with people that I wanted to address today is there's not one role out there that you have to find your way into. There's many ways. And actually it might be a role that you're already playing in a slightly different context. It may be that you can kind of evolve your way into this. And so, you know, if you're already working with databases and other data sources, data lakes, you know, that's one area that's now very involved in the kind of the big data input that goes into these AI models and stuff. So I really wanted to talk in a practical sense and have a conversation about what are different avenues people might be able to take to get into this fun field.

Yeah, and I think along with that, of course there's like you say, there's a lot of kind of jargon and job titles out there that people hear and might be confusing as to how they fit in, like data scientists versus machine learning engineer, research scientist or data engineer. But maybe it would be good to kind of talk about the various pieces of the AI workflow and where certain people might fit in in terms of a team of people working on these sorts of solutions.

That's a great idea.

From my perspective, when you're thinking about the workflow that often happens here, there is sort of an initial phase which involves a lot of kind of problem defining and scoping in terms of what may or may not be possible and what might be good to experiment with or try. And also an exploratory kind of phase of data gathering and pre processing and in an exploratory and interactive way, doing some model training and sort of proof of concept evaluation and validation of a certain process. So, you know, for example, if you're a manufacturing company and you say, oh, we've got this problem on our manufacturing line and we think maybe we could stick a camera, you know, in this location and detect this problem or something like that. You have to figure out, okay, well, what would I want as my input and output data? What's actually going to be fed in? Could this camera be placed? What would be the appropriate output that would actually make it useful? And then in an exploratory way, like could I actually gather some of the data which would allow me to train that sort of model? And if I could gather that data, what sort of model might I go after? And all of this stuff is very iterative and fuzzy. I guess this is the fuzzy phase. I don't know if you'd agree with me. I think a lot of these projects start out that, that sort of way it does.

There is expertise required on the front end. In real life, you don't jump in to model development. I think there's this kind of perception of, you know, come join us, hop on, you know, pick an environment, whatever, you know, you care about, and, and build a model. But there's a whole lot of work that goes into it on the front end. Before you even get to exploring in the data context, you got to figure out what is it that you think you want to build and why. And why on earth would this particular approach be the right approach?

And why would AI bring value in versus some other solution?

Yeah, I mean, that's a great point in that there might be five different ways of approaching a solution to the problem. And if building a neural network is the most expensive approach to doing that, and when I say expensive, I mean the amount of effort and time and resources necessary to do it. Why would you do that? If you can get a result that's just as good from, you know, from some other algorithmic approach and you need whatever problem you're going to solve, you need expertise from as far as being a domain, you know, expert on that problem area. And that might mean working with the business side of your company on what it is that they're trying to provide for customers, because at the end of the day, that's what a company is there to do.

Yeah.

And we're just barely touching on the front end of this process. So there are so many ways to engage in this AI process that we're talking about that don't require that you have a PhD, you know, in data science from a, you know, from a top university and have 30 years of data behind your belt, under your belt.

Yeah, I think actually there's like, in this sort of category of contribution, I guess we could call it this problem defining, scoping, exploratory stuff. In fact, I think there is a sort of solution architect sort of role here where you do need some type of knowledge about AI systems and what is possible and what is feasible and what isn't feasible and what's sort of overkill and what's not overkill and appropriate usage and like scoping in terms of how long this is going to take or how much data we might need. But those are skills that you can pick up without knowing, like the Difference between LSTM and gru, right?

Yeah.

That level of detail is not required, I think, for this sort of thing. Although I may not be one of them. There are people out there that I think really enjoy that, like going into a situation or a problem, maybe dealing with a client on a shorter timescale, like you know, a few months and scoping out a potential solution and then passing that off to another team to actually do some more implementation and production related things.

Absolutely. I'm one of those people sometimes. Yeah. It's one of the things that I do in my own job. And I'll tell you, having built up some expertise in the field, if you can go talk to people on the front end and help them figure out what it is they should be thinking about what, what's going to serve the need. It can be quite fulfilling and it does take some, some understanding and expertise of the field to be able to do that successfully. If you go in and only do the kind of be a business analyst without any background at all and no interest in developing the background, you won't be as effective at being able to decide that. So strategy is a key part of the front end of this process. Yep.

I think once the problems start shaping up like this, seems like it's going to be a valuable thing to do. There's still that exploratory phase of like getting an initial proof of concept data set together, you know, proving out that this will actually work and produce the type of value that we want. And oftentimes in this stage of things, I think like getting a kind of brute force solution is kind of how I think about it in terms of this thing might not be optimized in every way, it might not have the exact accuracy or performance that we want, but all of the right things are sort of plumbed together and like the right type of data is coming in, the right type of pre processing is happening. The right type of model is producing, you know, some result which is then being used to create something of value. That kind of rough plumbing of those things together requires now some technical skill. But this doesn't have to be a fine tuned C application that, you know, runs with super high performance on an embedded device out in the field. This is like proving out that the thing works and developing the right type of solution. So I think it's a more technical level, but it's not as hardcore software engineering or data engineering as it could be.

When you say that, I agree with everything you just said and the way I would express that is that AI development fits very well into an agile software development process where you're having to iterate and you learn from that iteration and you make those adjustments and you go back. And that happens both at the model level and it also happens in terms of how you're going to choose to deploy and do the engineering you need to accomplish that. I very much, and I know that I'm going to say something slightly controversial I think, and that is that I think of AI development as a component of software development, which a lot of data scientists will say, no, it's not. No, it's not. But when I'm looking at it in production and I'm looking at us actually managing that, I see it in that larger context because all of those other activities are happening around it. So definitely.

We deserve a better Internet. And the Brave team has the recipe for bringing it to us. Start with Google Chrome. Keep the extensions, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits, we don't need them. Mix in ad and tracker blocking by default, quick access to the Tor network for true private browsing and an opt in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give tipping a try on changelog.com.

I really liked where you were headed with what you're saying, Chris. In terms of AI development being viewed as a sort of subcategory of software development, I think this fits very well into the kind of mindset of another person we had on the show, Joel Grus. We'll link to his episode from the Allen Institute for AI. I think he's mainly working on the Allen NLP project and I think he had a lot more things to say about that and why it's useful. I definitely think that like we kind of started talking about the explore more technical exploratory stuff where you're trying to figure out what you're going to do and start plumbing the right pieces together and validate a solution. You will see some difference in industry, at least from my perspective in terms of sometimes at an organization the people that are doing that are not the same people that are at the end of the day involved in producing the production system that's actually implemented. And then you'll see other organizations where at least there is some overlap between the team that does this sort of exploratory work and the team that actually produces the production systems. From my Perspective, the latter has a big advantage because if you have total separation between those groups, then you know, when something goes wrong in production, basically the production team will, you know, maybe in a non confrontational way, but basically at the end of the day they'll say, well, this is a problem with the solution and the model and the way it was developed, not a problem with our implementation. And then the problem, you know, the people that did the exploratory work and validated the solution will say, no, our solution's great, you know, there must be something in the implementation. No one's taking ownership of it and no one's taking ownership of the robustness of it in particular like, and how, how robust the solution is. So I think in a perfect world there is some overlap between the group that does those things.

No, I, I agree with you completely. And I think the reason that, to state it, the reason that that second group has the advantage is because to learn from those earlier processes. So if you have one group doing a prototype, they've gone through that process and they've learned what they need to know and if they're going to hand it off to a production only group, well, they're starting from zero again or from whatever documentation came out of that first thing. So there's certainly an advantage to the learning process, which is why AIML development is best served in a larger agile development process. And if you're in that software development world and you're hearing this, these should be familiar terms to you and those are all potential inroads for you and your career and your particular interest in this, to translate existing skills and existing interest into this AI world and be able to do that. And there's no point where you're ever done, you can continue to migrate across that space by always learning and always deciding where you want to go next and doing that. Yeah, I think that's a crucial for career development in general, but especially in this one.

Yeah, and even in the phase of this that's exploratory, I often use this analogy, which listeners will be familiar with, that a lot of AI development is more akin to cooking according to a recipe than it is, you know, some intense research and development. And so even in that exploratory phase, it's taking pieces of things that have been done before and putting them together in a unique solution which is very similar to software engineering. And if you were to produce a proof of concept in software engineering, the difference, I think, you know, there is a sort of tool set difference maybe that some software Engineers might be a little bit uncomfortable with like in this exploratory phase, you might have a, you know, a Jupyter notebook that shows here's how I ingress data and then here's how I pre process the data and then here's how I train my model and then here's how I do inference. And then when you move into the production side of things, maybe things, it gets a little bit more comfortable in terms of the tooling for software engineers where you would take that notebook and then say, well, I'm not going to run my notebook in production. I've got to take out this data gathering piece and make it a Docker container that's going to run in kubernetes on aws. And then I've got to take out this pre processing container and figure out how to run it in parallel over a large data set in the cloud. And then I've got to take my training piece and pull that out and dockerize that and figure out how to run it on some GPU accelerated infrastructure. And those pieces still carry through but the tool set and the way you go about it definitely changes.

Yeah, that's a great point there. And that is that at different points you may have different people involved in the maturation, the maturity aspect of this process. And so it's really common for software developers to look at a Jupyter notebook for the first time and scoff at it and say, no, I grew up in software development best practices. I'm looking at this, this Jupyter notebook and it's, you know, why would you do that? But you know, if you are the data scientist that's, that's trying to put the model together, it's a fantastic way of iterating rapidly. And your job at that point is not to produce production software, it's to test and try different things out. You may be implementing a transfer learning approach where you're then trying to customize that transfer learning into the specific solution you need. And likewise the data scientist needs to recognize when you deploy it, you're not deploying that notebook. You were using the notebook for what it's good for. But it has to be a software component. It's a model that's wrapped in a software component that's being deployed out into a larger software system at the end. And so there's a role for all of these things. And so leave your biases at the door, leave them there. Look for why each tool or each role is so important and recognize that. Because I've seen people fall down in that way many times.

Yep, I know. For example, we had a question in our slack recently and a discussion about, hey, I, you know, I hear all of this stuff about training and I'm able to run these examples, but then when I try to do this inference in production, the performance is so terrible. Why is no one talking about this or how. Why is it hard to find resources about this?

Great question.

There definitely are resources out there. And I think like the commenter said, it would be great to have a even a full episode about that side of things. And model optimization, that is another piece of the puzzle that changes when you kind of move later on into a project is if I'm running this in an edge device in a manufacturing plant, it's going to have concerns. If I'm doing it on a mobile device, it'll have different challenges. If I'm doing it on a beefy cloud instance, then you have maybe more flexibility, but you may have like latency issues you want to deal with or something in responding to people.

So that's a great question from the listener and I love how you led into that. And really I'm not sure it's an official term or not at this point, but we have conversations, I know in my own collection of colleagues about this all the time we refer to it as AI engineering. And I think the thing that is so crucial about that is to recognize that two years ago we were talking about the edge as kind of an exception case because people really deploying most often into servers or locally or whatever, and it was more of a kind of a standard, well known environment. But going forward, most things will be at the edge. As you make models and the utility of models pervasive in our society and our culture, you're going to see edge devices being the targets of that deployment in so many different ways. And so that requires that you rethink your engineering to accommodate that. Once upon a time, you know, deploying software was really, you know, it was kind of code centric and you'd think about just processors and stuff like that. But now it's all about data. If you are deploying to some sort of mobile platform, maybe it's an autonomous vehicle, you have telemetry from that vehicle, you have sensors in that vehicle, you have cameras in that vehicle. And to provide the level of performance, you need to be able to do real time inference on that requires special knowledge of engineering on getting the right data in the right way, to the right place at the right time, so that it can be acted upon and you no longer are doing static data that you're running through a server or something. So AI engineering is crucial for making this stuff actually work. It's later in the process than what we were talking about. But you know, after that data scientist has been working in the jupyter notebook, you got to either put it out there in the world or it's useless. It doesn't do anything for you.

Yep. Another piece of this puzzle is actually, I think so there's like the AI workflow and the different, you know, phases along a project all the way from, from kind of solution architecting or consulting to, you know, the very technical side of AI engineering things. But then there's also, I think, you know, you could look at that workflow in different domains or verticals and that's going to look very different. Of course, you know, in maybe the manufacturing world you're, you're going to be thinking a lot about computer vision and running things and edge devices in potentially hazardous conditions where they might have to, you know, you have a lot of device issues. In other cases like in web space, if you have a web app that you're dealing with or software as a service company, then you might be running your models a lot of time, you know, in the cloud and maybe you're dealing with a lot of natural language processing issues and dialogue related issues with customer service and all of that. And each of those sets of problems has its own tooling and its own methods and its own community and its own way of going about things. And so I think another thing to think about when you're thinking about the lay of the land is also the domain. And I think like you said, this happens in software engineering too. And people have specialized in certain areas of software engineering and AI I think will be no different. There's a lot of specialization that can happen.

Yeah, I think in my own experience it definitely bears that out. If I look at counting my current employment, my last three organizations that I've been a part of and all three had an AI role. In the first one we were working with clients and it was server based. It was kind of what I think of as a little bit old school now. Well, you know, it's funny that it doesn't take very long for something to become old school because it evolves so fast. But yes, we were deploying models into big servers that had, that were resource rich. And then in the next organization I went to, we were focused on warehouse spaces and introducing robotics and cameras and different things that make logistics work and that presented a different set of challenges that were specific to the domain. And then now I've moved into the defense industry and I focus on autonomous platforms and other adjacent technologies. And some of the previous things certainly had an effect. But this is a new domain that has its own specific constraints and challenges and that's the case. So we are definitely seeing diversity in how AI is conceived and implemented, depending on the context that you're using it in.

Yep. Well, one thing that's true across all of these workflows and domains is that definitely you're going to have to deal with bias in your data and model fairness. And this kind of brings us to the end of our conversation where we're going to share some learning resources with you and think in light of our current climate and things going on in our world, it's only natural to share some resources about bias in your data and model fairness. I think that one of those resources, which maybe is a good jumping off point, there's a nice write up in Google's machine learning crash course about fairness and types of bias and I thought this was pretty interesting and maybe certain branches of science have similar terminology around this sort of thing and think about like, like, you know, survey science, for example, thinks about bias a lot and populations and, and those sorts of things. So this was really helpful for me to kind of pick up some of this terminology and in examples they actually go through it talking about reporting bias, automation bias, selection bias, group attribution bias and others, and give examples of those types of biases and how they can creep into your data, which I thought was incredibly useful. I don't know how familiar you are with some of these things, Chris, but it was really helpful for me because I was not familiar with the sort of categories that you could think about bias in.

Yeah, totally. And bias in the involvement I have in the AI ethics space. Bias is a huge part of it. It's probably the concern that most people associate most with AI ethics. It's the thing that people think about the first. And so understanding those different types of bias and how they impact an outcome and how they can result in unexpected outcomes, which is, can be incredibly common, is pretty important. So that's, it's a, it's a first good way to get into that and kind of going back, I think it's particularly applicable as we have this episode at this particular time, given the large public response to injustice, to think about some of these tools I've already heard are being used in unexpected ways against protesters for instance even ones that are not breaking the law in any way. And it's you know just as a. As we think about different types of bias here, think about how do you want the application of these tools to be to be used? Do you you know facial recognition can occur long before or after a protest event. You know by following people through cameras and having do an automatic tracking. There's a lot of impact on how we may want to think about this.

I'd also encourage people just a couple more quick mentions here to take a look at IBM's fairness 360 website there. It just includes a really great sort of breakdown about various ways of that people are dealing with fairness. Both sort of pre processing of data in processing or model change actual changes to your model that you can make. Also like post processing monitoring of your predictions. They talk about a whole variety of things with great examples. So check that out. Also Google's responsible AI practices they have a great write up and discussion of fairness and bias. There's also a good project from Driven Data called Dion which includes a nice checklist if you like checklists that you can sort of start with a default checklist and update it to make sure that you're checking for certain things like bias and and fairness in a project and that can be embedded within your repository, within a jupyter notebook or other things. So we'll link to all of those in our show notes. I think it's well worth people's time to take a look at those things and make sure and educate themselves about how that can creep into your process.

Totally. There's one other one that I'll throw out that is it has been useful beyond the industry that it started in is that is the because of the process that the US Department of Defense entered into on their AI ethical principles and we a show where we addressed that in depth. Previously they went out into industry and academia and solicited feedback from many many different people in the space. Many of them were luminaries whose names you would recognize and you can actually go and do like if you Google DoD AI principles you'll find that they have their five and Google just like Google and Microsoft and all the other players do. But I've noticed recently that they're being adopted in completely different use cases because they're not specific necessarily to the industry that they were formed in. So that's a really good one that I end up interacting with quite a lot.

Awesome. Well it's been great to have a conversation with you again Chris. Great to have you back and looking forward to our future conversations and how those will be shaped with our ever changing world in the future. But appreciate our listeners hanging through us this spring with changes in our in our schedule and also changes in your life and being in different places than you normally would. Be glad that you've continued to stick with us and looking forward to more conversations.

Absolutely. And for my part, I just wanted to thank the listeners for bearing with us as we started the show and having my my sharing what had happened to me in the show notes. I'm also going to include a link to kind of my experience of COVID in a way. So if it's something you're interested in and want to know somebody that's actually dealt with it in a firsthand way, you can check that out in addition to the normal notes for the show. Thank you so much for listening.

Thank you for listening to this episode of Practical AI. People ask us all the time. They say, hey, how can I support your work? One easy way is to leave a five star review on Apple Podcasts. Tell folks why you listen and why they should too. It only takes about 30 seconds, but and believe it or not, those ratings and reviews really do help us rank higher in AI Related Search results. Practical AI is hosted by Daniel Whiteneck and Chris Benson. It's produced by Jared Santo, that's me. And our music is brought to you by the one and only Breakmaster Cylinder. We are sponsored by amazing people at companies who get it. Thanks again to Fastly, Linode and rollbar. Did you know we have a master feed of all Changelog podcasts? We do. It's your one stop shop for everything we produce. If you like this show, you'll love the Changelog Brain Science and go time. Check it out@changelog.com master or search for changelogmaster in your favorite podcast app. You'll find us. That's it for now. We'll talk to you again next week.