Learning to learn deep learning 📖

This is really the core about how people build careers. Now if you are thinking about how to get there, you have to build in constant lifelong learning into your process. And if you don't do that, you will fall behind. And as things are moving faster and faster forward, the ability to thrive in a digital world is really crucial to having it. Because anything you're learning right now in more of a formal context, if you're coming out of school, for instance, a lot of that, you know, will be obsolete. The basic math will be there, but the algorithms are going to change. How you're achieving problem solutions will change. And so you really have to be able to do that. I think that is a fundamental for being a digitally literate person in the job world these days.

Big thanks to our partners Linode, Fastly and launchdarkly. We love Lino. They keep it fast and simple. Check them out@lino.com changelog our bandwidth is provided by Fastly. Learn more at fastly.com and and get your feature flags powered by LaunchDarkly. Get a demo at launchdarkly.com this episode is brought to you by our friends at O'Reilly. Many of you know O'Reilly for their animal tech books and their conferences, but you may not know they have an online learning platform as well. The platform has all their books, all their videos and all their conference talks. Plus you can learn by doing with live online training courses and virtual conferences, certification practice exams and interactive sandboxes and scenarios to practice coding alongside what you're learning. They cover a ton of technology topics, machine learning, AI programming languages, DevOps, data science, cloud containers, security, and even soft skills like business management and presentation skills. You name it, it is all in there. If you need to keep your team or yourself up to speed on their tech skills, Then check out O'Reilly's online learning platform. Learn more and keep your team skills sharp@o'reilly.com changelog again o'reilly.com changelog.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changeaw.com community and follow us on Twitter. We're racticalaifm.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a strategist with Lockheed Martin. How you doing Chris?

I am doing very well, how's it going today, Daniel?

It's going good. For listeners who are in the US you'll know that this last week was Memorial Day weekend, which is good. It was a nice long weekend. It's beautiful weather here, but I'm feeling the amount of catch up I need to do in this short week. I don't know about yourself.

No. They give us a holiday, but then the amount of work for that week never changes. So you just have to cram it all in.

It's always the same, right? Yeah, I'm feeling it this week. But, you know, lots of good stuff to work on. But I'll sleep well over the weekend, I'm sure, and have a nice Saturday to sleep in, hopefully. So there you go.

Recovery time this weekend.

Recovery time this weekend. One thing is there's a lot of news and updates and AI stuff to catch up on. I know both Dean and I have probably been feeling that lots of things have been happening in the AI world. And so today on our episode, what we're going to do is we're just going to kick it back old school. And as our long term listeners will know, sometimes Chris and I like to do these episodes where we don't really have a specific guest speaking about a topic, but we kind of talk about a variety of topics that we've seen in the AI ecosystem we think are interesting, maybe of note, and discuss them here live. So you're up for that, Chris?

I'm totally up for that. We have fun with these episodes. I enjoy them. This is a little freeform for us.

Exactly. Yeah. And I'm sure it shows to our listeners how ignorant we are in certain cases. But that's okay because as everybody who's exploring this space, everybody has their sort of own little niche sphere of knowledge. And it seems like these other people are like very knowledgeable in all of AI, but they just have their own little niche of knowledge too. So it's good to sort of dip our toes into different areas. At least it is for me. I think.

You know, that is something that we talked about a little bit in the last episode for anybody that might have been listening was just the fact that there is too much stuff for everyone to know all of it. And so you kind of have to pick and choose where you're going to dive in. And that might be something worth talking about today because you have to kind of make some choices. And it's interesting how the different choices people make affects how a team might come together and, and what the team's capabilities are and stuff. So there's a lot of ramifications here.

Yeah, there's definitely the concept as well of like when you're putting together a data science team or an AI team or something like that. Actually, it's funny, we're actually bringing this up because we had one of these discussions internally actually even this week is, hey, what are the advantages of having a sort of organizationally separate, you know, data science and AI unit versus data science and AI people embedded in various teams throughout the organization? And also within those specific people, of course, there's a slant. Sometimes people sort of want to hire in data science people or AI people to just do the sort of modeling and analysis stuff and like, don't worry about the other things, just focus on that stuff and we'll handle the business logic and like pushing that out into products and all of that stuff. In other cases, people sort of very much like to, you know, there's even this term now full stack data science, where it's like, hey, I'm going to take care of doing this analysis piece, but I'm also going to figure out how to integrate. Integrate it into a product. So there's just so many different parameters there, I think in terms of how people put their teams together and it makes it hard for a new person to figure out where they fit in. That I think it sure does.

I mean, not only that, but it doesn't stand by itself. And that, you know, AI, if you're thinking of AI as deep learning, which we tend to in these last few years, and it has to be, you know, you have the algorithms, you have the, the need to deploy it out to be useful to your user community. It integrates in with other software, it integrates in with the microservices and the APIs. And so you have a huge area of target surface of things that have to get done. And I have never seen two organizations do it the same way. And the organizations that I've worked with, it's like everyone has their own way of doing it, which means there's not a normalized or standardized way to, to achieve that.

Yeah, yeah. And of course this is something because we have a podcast or we're interacting with a lot of different groups. Something that I'm asked a lot is, hey, I'm wanting to get into this field. Where do I need to focus? And oftentimes my response to that is, well, let me ask you a question, like, are you more interested in this sort of more engineering slant, or are you just wanting to do sort of researchy type things and analysis and that sort of thing. And because those two could take a very different path. And probably if you're focusing on one, you're not going to be well prepared for the other one. It's very rare that there's someone that sort of operates in both of those spaces.

Yeah, totally. It's a super hard challenge to figure out for someone new into it because I always ask people what they're already passionate about, or I ask them what kinds of things are they reading and watching videos on that has captured their imagination and then start building on that. And you can ask five different people and get five, five very, very different answers, which isn't necessarily a bad thing. You know, if they're each gravitating toward different areas that have to be addressed, then that's the makings for a team right there.

Yeah, maybe one thing I could suggest to promote my own, you know, biased opinion. I was thinking about this recently. I'm kind of involved to some respect over at Purdue University. Sometimes some of their students help my organization with some projects, like practicum type projects, or I help their students and working on projects or occasionally I teach some lectures and I was putting together some material for a course this time around. My thought process was really like for these students, if I was hiring them in, what would be the sort of set of things I would want them to go through or be able to go through that would really give me confidence in their abilities to perform, you know, on my data science or AI team. And so the thing, and you know, you could disagree with this, like you say every team's different. But what I did was I said, okay, I proposed a certain problem. The problem was question answering. So a piece of text comes in, there's a question, you have a model that actually extracts the answer out of the text. This is a well known problem. And there's models out there that do this because it's a short time that we have the students together. I was basically saying, hey, well this model exists, so figure out how to use it for inference. And then I would like to see three different sort of manifestations of that model. One that I can have a REST API and you know, send a request and get a response. One where there's a user interface where I type in my question and I type in my passage and get the answer, and one where I can use it in a batch way. So if I want to answer, I have a Data set of 20,000, you know, 50,000, 100,000 questions and passages and I want to answer them all at once. What's a way that I can do that in a batch sense? And I think thinking through those different sort of manifestations of how a model could operate was really, I think it was actually helpful for me in terms of thinking through, you know, how to structure the material. But also it, you know, I thought it was a good way to think about things and maybe something I'd share on the podcast because as people are getting into things, they want to sort of show a project or something and I would encourage people, you know, maybe it's worth, you create a model, but then you create these sort of different manifestations of the model in how it actually interacts with users or the end use case, something like that. Yeah, yeah.

So let me ask you a question about that. Was the model itself fundamentally designed differently or was it more of a software input output thing?

Yeah, so the model itself was basically the same in the three cases, although in a more batch sense.

I had a feeling you were going there.

One thing of course you can do is operate on, you know, instead of pushing one input into the model and getting one output over and over, of course you can push multiple in and then you can also parallelize across the data set and all those sorts of things. So there is a, maybe a different inference pattern. The model architecture itself wouldn't be fundamentally different. I think if there was more time I would have loved to add a kind of upfront component which would have been like fine tune the model, take a pre trained model, fine tune that model with just a little bit of data for a slightly different task or a slightly different domain and then sort of make it available in these three different ways, I think. I mean that's almost the sort of boot camp in and of itself. If you can sort of go through that process, I think you're really, you're well, you're at least well set up to talk the language and understand how people go about their AI and ML development. And you know, I'm sure there's people out there that disagree with me.

Yeah, yeah. I would argue that that's almost more of an engineering thing than it is truly a modeling thing. Because your model is not fundamentally changing, it's just how you're choosing to deploy it and integrate it in with the larger software environment that you're in.

Yeah, and I would probably go more, I'm naturally more on that engineering side probably in the way that I operate than I am in the sort of pure research or creating new Model architectures side of things. So if someone wants to go then and like be the next team developing the next cool model that is, you know, fundamentally, fundamentally different and that, you know, they spend five years on that to develop that model and really prove it out and probe all of the implications and all of that, that is also really needed. But it's sort of different skill set around research and experimentation.

You know, you raise a really interesting. There's a dimension to this I'd like to draw and tell me if you have a different sensation about this. But as I'm looking at the deep learning world over the last year or two, compared to the years prior to that, that it seems like we were seeing lots of completely new architectures coming through, through the late teens especially, you know, early and into the late teens. And then we kind of got into the 20s and we're really in an age of implementation. I don't know if that's because I'm in my bubble and I'm seeing tons and tons and tons of implementation, but not a whole lot of truly revolutionary new ideas coming to market. What is your impression? Am I trapped in my little bubble? Or do you think that we're seeing more implementation than we are research right now because so many more people are getting into deep learning?

It's a good question. I think part of it is maybe the, the maturity of the field in the sense that in maybe previous years everything seemed a little bit more new because nothing was really normalized yet. It was all this sort of storm of ambiguity and new things. Whereas there's, you know, really developed workflows around transfer learning and you know, even, you know, running really large scale jobs and multiple GPUs and other things and all of that sort of like people have developed workflows around that so maybe that those pieces of it seem less new. But I don't know. I don't know if I would say that, you know, the progression of new models and architectures has slowed or increased. I don't know. There's definitely still some of that going on, but the perception may be different.

It feels evolutionary at the moment to me because I was really wondering this, knowing that we were going to have this conversation among ourselves. I was coming into it, thinking about it and going, when's the last time I saw a completely new architecture sweep through the market and really take hold? You know, like, you know, back when we were looking at, you know, the sweep of convolutional neural networks. And then we hit the age of nlp really hitting its stride, followed immediately by another big jump with Transformers. And then we've seen reinforcement learning have some jumps in there as well. And I was thinking, when's the last time one of those big, big jumps leapt out at us? Can you think of what that might have been? What would you think of as the most recent big jump forward?

I don't know. I'm also pretty stuck in my little NLP bubble maybe. So obviously Transformers and large scale language models have been the thing.

Your bread and butter. Yeah, definitely.

That have really pushed the envelope forward. But that really has happened over the past few years.

Yes.

There is one thing I'd like to discuss in a bit that I think we can decide if it's new or not in that sense of new. It's new. But yeah, I don't know. I definitely think that there's more people getting into the field and because of the tooling and the maturity of their tooling, there's more people that are able to do very advanced and sophisticated things, whereas maybe that was restricted to a very elite before.

This episode is brought to you by Snowplow Analytics. Snowplow is the behavioral data management platform for data teams. Maximize the value of your behavioral data using Snowplow Insights, a managed data platform that's built on leading open source tech leveraged by tens of thousands of users. Capture and process high quality behavioral data from all your platforms and your products and deliver that data to your cloud destination of choice. When marketing needs to make data informed decisions, when product needs next level understanding, and when analytics needs rich and accurate data. Snowplow is a solution for data teams who want to manage the collection, processing and warehousing of data across all their platforms and products. Get started and experience Snowplow data for yourself@snowplow analytics.com again snowplow analytics.com.

Chris, you were kind of bringing up this topic of like new architectures and like what is a big leap in terms of new kind of models and what is like building on the shoulders of things before. It's to some degree, I mean everything's building things from before. But one of the things that I've really enjoyed sort of watching and have seen kind of have a lot of discussion on Twitter and in the circles that I'm in is this new model from Facebook, wave2vecU. I don't know if you've seen this. It's a pretty interesting model. It's a speech recognition model. And the interesting thing, well, I guess it sort of stands on the shoulders of Wave 2 VEC, which was a previous model, but it is very interesting in that it is unsupervised. That's what the U stands for. Okay. And I think what maybe shocked people or what got them really thinking is that this wave 2 vec u essentially learns how to create transcriptions of audio or speech using purely unpaired data or untranscribed data. So just putting audio in, which is kind of a weird thing if you think about it. Have you seen this or has it crossed your path at all?

I had seen the name, but I hadn't dug into it. And I'm looking at it while we're talking about it, and it came out a couple of weeks ago, so it's still very new. If I'm reading this correctly, it's still fresh.

Of course, speech recognition is very much a problem, or often the problem with speech recognition is the fact that you don't have a lot of data. We've talked to people like Jeff Adams or others on this podcast who worked in speech technology for some long time, and a lot of times the problem there is you need a good amount of high quality transcribed audio. Like, we're talking about thousands and thousands of hours of transcribed audio, and you need it sort of pretty high quality. You need a diversity, maybe of speakers, you need a diversity of accents. And gathering and curating all that data is incredibly labor intensive. And so this model, basically, it operates in a slightly different way than previous models in that what it's trying to do is take in unlabeled speech audio, just audio, and generate the phonemes corresponding to that audio. So phonemes is a word that I, you know, I'm not a linguist, so I learned this after joining with sil, and they were talking, started talking about phones, and I was confused about, you know, phones, cell phones, what type of phones are we talking about? But phonemes are phones here. We're talking about actual sounds. So phone phones are the sounds corresponding to the audio. And what you can do is you can actually try to create a model that takes in speech and generates phones from that or phonemes. And then you could take those phonemes and then map them to text. And it's that sort of generating from the audio to the phones that they did in an unsupervised way. And the way that they did this was actually in a sort of generative, adversarial way. So they had audio coming in in one stream to a generator model that generate, tried to generate the phones corresponding to that unlabeled data. And then over here, they had actually unlabeled phonemized text. So they took text that was already, you know, it was good text. It was like, you know, text that they knew was in the language that they're working with. They converted that to phones, and then they tried to see if a discriminator could tell the difference between the sort of, quote, good phones that were actually from actual text versus the phones that were generated from the generator model. And so it's interesting that they're working in this in two ways. They're working in this phone space, but then they're also using this idea of generative adversarial networks to allow them to solve this problem in an unsupervised way. So I don't know if this fits into, you know, after sort of talking through it, if this fits into fundamentally new, different, or just like incremental. I don't know.

I would say, I don't know that it's a whole new category, but it's certainly an interesting step forward in kind of that spans the natural language processing and generative adversarial network world together. I'm kind of curious. Could you talk about a use case where you could see this being applied just to give context for it?

Yeah, good question. So, of course, this also stood out to me because I work with a lot of local languages, and oftentimes you either have a very small amount of sort of transcribed audio for a language in one of these lower resource language, or you may have none. But oftentimes in, in, like, language survey and documentation and language preservation efforts, you do collect audio of the language. Right. It's just not transcribed, and in some cases, maybe it's actually never been written down. Right. Like, there's many languages out there, people might be surprised that are still just oral languages. They've never even been written down in any type of script. So I think that although this might not be applied, like out of the box in that scenario, I think it works us much closer to where, you know, people working with these sort of endangered languages and very extremely low resource languages who are able to gather audio, you know, of the language speakers, still they may have an advantage in terms of being able to actually work towards transcribing that audio more quickly than they would have before.

Fair enough. I'm really fascinated by the number of use cases. You know, we've just seen so many orders of magnitude exploding outward the last two or three years in terms of where, you know, when we started this podcast and we are closing in now on three years that we have been.

Doing this, that's kind of crazy, right?

The landscape, it. It is a little bit crazy. I agree. The landscape since we started has changed dramatically in terms of how widespread these technologies and specifically these models being deployed has been. It was still a little bit unusual to find use cases in the marketplace where you had deployed deep learning models that were actually in production. They were kind of here and there, whereas now it is everywhere. And really it is going back to some conversations that we had some time back. We are seeing, you know, it very much just being part of software development. The idea of model based engineering is just what engineering has become. It's not a separate thing anymore. It's not a call out. And I think it is unlikely these days to build any significant software system without any thought into model deployment as a service or microservice that can be utilized in that. Which brings us back to all of those folks out there who may either be trying to create a career in AI or deep learning, or however you want to call it, or even software and trying to figure out where exactly they fit in and how they should approach as well as a lot of organizations, it's now that the cost has been driven down so substantially from the early days when you, you know, very early when the widespread large things were being done by the big name companies. But now it's available to just about anyone. And all the major cloud providers have full tooling sets, you know, that cover not only the model creation and the model deployment, but the entire software stack with it integrated. So if you have somebody coming out of college right now, and maybe they have a computer science degree, maybe they don't, maybe they're in another area, but this has captured their imagination. How would they help their new employer get into this? In what different areas? How would you, if you were coming in. It's been a while since we talked about it from a complete newbie standpoint and I'm just curious, how has that changed now in the last couple of years?

So when you say that, do you mean like I am a data or AI person coming into a company that's not doing AI things? How do I go about creating that transformation? Or are you coming at it more from. I'm coming out of college and I want to step into this world of AI type stuff?

I think we should divide them. Let's hit both questions for a few minutes. And I think it's worth asking because you and I have been doing this for such a long time and it feels like, we've been there and done that many times. But in my day job, you know, outside of the podcast, I'm still having people come to me on a regular, you know, day to day basis and asking for mentorship and how do I do this? And it makes me realize how many people are still trying to find a path into it. That happens a lot.

Yeah, I think there's actually. And maybe we can go in terms of learning resources for those actually trying to get into the field as an AI person here in a second. But I think that the first of those that you mentioned about, like, how do you sort of seed or spawn AI work within an organization that's maybe not operating in that area currently, I think that's actually a very challenging problem. I would recommend. I don't know if you remember we were talking about how long a podcast has been going on, but way, way, way back at the beginning, in the first few episodes. I forget which of the first few episodes we had Mike Bugembe on the show.

Oh, yeah, I do remember that.

Yeah. He's sort of an expert in. I mean, I think I know he's an expert in this area. I don't know how you describe himself, but how. I sort of think of him as really, really having amazing expertise in this sort of behavioral economics of creating this sort of change within an organization. He has a book called Cracking the Data Code. So that's maybe a resource out there for those looking to create more of that organizational change. But he has a formula that he follows. But I think that part of that really comes down to making sure that you have and develop more and more empathy for the people in your organization and the sort of pain points that they're feeling, the more that I think they can feel truly listened to and that you understand their pain points and why they go about the way that they're solving those problems now and the challenges that they're having, the better chance you have of sort of coming beside them and, you know, bringing them in very early into the process and helping them sort of shape what a solution looks like. Because once you kind of, I think, get through a first cycle of this and if it is successful, other people will sort of start asking and momentum will build. But if you kind of get that wrong the first time and you develop a really bad taste in people's mouth for like, AI predictive things, then that's almost impossible to overcome.

I think that's a really good point and that was a really good answer actually to how to address that. One of the things that I'm seeing also is organizations trying to figure out how to kind of what we talked about in the very beginning a little bit, but how to address AI. Do they do it as a separate organization or do they, you know, they integrate it in with their data science or their software teams? And that can vary based on the domain expertise they have, what their employees are, what kind of business they're in and also what technologies they're focused on. I think early on I saw a lot of AI being broken out more and more as its own thing. And I think the people that drove that were very savvy on kind of internal marketing and getting budget for that. And certainly as we've seen it normalize over time a little bit, you're starting to see it roll back into more of the data science and DevOps side of things as well since it's, you know, it's just another model type. Why you know, divide up your model types into some or you know, some happen to use GPUs and some don't. That seems a very artificial. An artificial thing. Yeah.

And there's always going to be, I don't think there's any right, you know, perfect solution here because there's going to be challenges either way. If you try to centralize your data science and AI team within an organization, the challenges you're going to face are those where you have to be very intentional about having those people, you know, reach out and make sure that they connect with the end user, understand their pain points, understand their challenges and that they're really have the tentacles out in the organization. Whereas if you have everybody distributed everywhere, you're probably going to deal with more like duplication, type issue, duplication of work. How are they going to share resources? Maybe it's more efficient to have centralized infrastructure, but now it's more complicated because people are in all of these different organizational units and all of that sort of stuff. So I don't think any one particular model is perfect. Foreign.

We deserve a better Internet and the Brave team has the recipe for bringing it to us. Start with Google Chrome. Keep the extensions, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits, we don't need them. Mix in add and tracker blocking by default, quick access to the Tor network for true private browsing and an opt in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give tipping a try on changehog.com.

Well, as always on these full connected shows, I think that our conversation has led us exactly to where, you know, we often like to go in these types of episodes and that's to learning resources for people, maybe new ones that have come out or things that have been updated. And we are talking about two different scenarios. One where, you know, the challenge is kind of having an organization be transformed into one that uses AI or data science type techniques. And the other where maybe you're an individual and you're trying to break into an organization that's already using AI. Maybe. On that latter point I ran across something that I haven't gone through totally, but I very much want to. And I've seen sort of a bit of buzz around and this is this new book that is called Meta Learning. Have you, have you seen this?

I haven't, no.

Okay. So originally when I saw the book I thought, oh cool, it's a meta learning book. And of course that's a bit of a loaded term in the deep learning AI type world because meta learning we even have, I think an episode with, I'm not sure. Well, I know we talked to Cheryl Chen about AutoML, we think we talked to a couple people about meta learning where this is really learning to learn. Right. So you have some method that maybe learns an architecture or learns an optimization technique or something like that. You're sort of learning to learn in some way. Well, that's not the kind of metal. Well, maybe that's part of the book, I don't know. But this sort of learning to learn book is really about how to learn deep learning. So it is meta learning, it's how to learn deep learning.

It's the human learning, not the model learning in this case.

Yeah, the human learning, not a model learning how to do learning, but a human learning how to do deep learning. There's a nice title. Yeah, how to Learn Deep Learning and Thrive in the Digital World. And sorry if I get the name wrong, Radek Osmolski and looks like a really nice book. And I think that the focus here he talks about I learned to program and do deep learning using online resources. Most of my income over the last two years has come from deep learning roles. How did that happen? If you know, and he, he basically lays it out, how did he go from, you know, learning deep learning using online resources? And now he sort of makes his career from. Or his career is doing deep learning type things.

So this, you know, and I think both of us sort of ended up in that same path. Most of our learning has come from self learning.

I mean, certainly I'm grateful for the sort of math foundational things in my past and certain programming things, experience with doing experiments and that sort of thing in science. But I mean, physics isn't exactly deep learning. And I didn't really think about machine learning until I was in industry, so. And maybe you can remind people about sort of your status prior to working in AI.

Well, I was doing lots of software for years, but actually even that would be the same thing in that I learned how to do software fairly early in my career and my degree is actually in finance. So it has nothing to do directly with any of these things that we have been talking about, whether it be software or deep learning or the things that come next. So I think the beauty of a book like this, as I'm looking through the webpage about it, is that this is really the core about how people build careers. Now if you are thinking about how to get there, you have to build in constant lifelong learning into your process. And if you don't do that, you will fall behind. And I meet people all the time that are doing both. I meet people that are constantly learning. I know you and I are definitely two examples of that. I have some other good friends, good colleagues that are constantly learning as things are moving forward and figuring out where and how they're going to dig into those ideas incrementally as they go. And then you see people that don't do that and they do fall behind. And as things are moving faster and faster forward, the ability to thrive in a digital world, going back to the book subtitle, is really crucial to having it, because anything you're learning right now in more of a formal context, if you're coming out of school, for instance, a lot of that will be obsolete. The basic math will be there, but the algorithms are going to change. How you're achieving problem solutions will change. And so you really have to be able to do that. I think that is a fundamental for being a digitally literate person in the job world these days.

Yeah. And there's also a lot of what I've seen as maybe difficulty in parsing through. It's kind of a good thing and a bad thing. There's so much available online that you can use for your learning, but there's also so much available online for free or very inexpensively.

Yeah.

So like how. How do you figure out what to focus on when there's so much you could focus on? And in what order? Right. That's often the missing piece. I feel like something like this. And again, I'm expecting that the content of this book at least goes into a little bit of this, although I haven't explored it in detail, but it sounds like from the purpose of this, that part of it is, you know, kind of putting some framing around the resources that are online and helping people understand the different things that they might need to explore if they want to have this sort of role or. Or be part of this kind of world. And that sort of framing can be almost the missing piece. Right, because someone publishes a course over here on this topic, and someone publishes a course in this topic, and then there's, you know, a thousand courses and there's a route through those courses that makes sense, but if you don't know anything about the topic in the first place, it's hard to determine what that route is. And then you sort of get all confused and maybe discouraged.

Yeah, I think a really good tactic which actually comes straight out of the software world is the concept of scratching your own itch is that if you want to get into deep learning and into AI over time, as it continues to evolve, having a personally significant use case, something that you want to get done, and it doesn't have to be a day job thing, it can be something you're doing on your own, it could be doing something around the house. There's an infinite number of possibilities, and we've talked about these off and on over the three years of the show so far at various times in terms of things that we wanted to do, and having a use case and figuring out a path through that. And there can be an infinite number of paths through that. If your fascination is how do I create the model, then you spend your time and learning resources on what are the possibilities. Somebody else may be a lot more interested in the engineering side of it, and they're really just keen on taking advantage of transfer learning and finding a model that's already doing very similar to what you want, if not exactly what you want, and basically getting that into your context and deploying it out there. And those are very, very different paths that you might take to achieve essentially the same thing. But doing that also helps you find your own personal golden path on how you're going to come in to this AI world and be able to be productive and passionate about what you're doing. So really scratching your own itch is finding the things you want to do. If it seems a little intimidating, just try to dig in a Little bit. And try to figure out what those parts would be before you go, oh, this is too much, and break it down. And then continue to break each piece down and put them in the right order and get there. I find that I have a habit of the things that I've gravitated toward have usually been because of that. Because there was a personal fire I had going on that particular thing.

Yeah. The other interesting element of this sort of finding your path, and I don't know that this is explicitly detailed anywhere, maybe this would be a good resource that could be out there, but depending on where you want to end up in your career and who you're wanting to work with, the sort of stack that you're working with. And it can be very different. As we've talked about and certain of these courses online sort of assume just like a single lane, because it's all from one perspective in terms of the tech stack and the tooling as well. I'm thinking of. So I was looking up for some MLOps resources recently and just by the.

Way, for people who aren't Familiar with that, MLOps is kind of the machine learning version of DevOps, you know, development, operations, getting your stuff out there so people can use it in a reproducible and standardized way. Keep going. I just wanted to throw that out for those who aren't used to it.

Yep, very good. Yeah. So I was looking for, you know, some resources on this at different courses out there that teach ML Ops and all of them actually that I was looking at seem really good. But for example, I was looking at the deep learning AI new MLOps course, and I'm sort of looking through there and I think partly because I've been exposed to some of this stuff I kind of deduced, and partly from who's teaching it, you can sort of deduce, hey, this is a very tensorflow extended specific pipeline of tooling. Right. Like they're working with this framework called TensorFlow extended in terms of how they're doing both model serialization, optimization, deployment. Yep. Sort of from end to end. This is like the world that they're, that they're living in, which makes sense for, for a lot of companies, but there's a lot of companies that have very drastically different opinions on that sort of pipeline. And another one I was looking at was from Elvis Sarvia. He publishes a lot of great content and he has a site called Machine Learning Ops, a collection of resources on how to facilitate machine learning ops with GitHub. And so this is like a very Different. This is all about like, okay, I have code that does machine learning stuff. How can I automate the deployment and manage testing and all of those things with things like GitHub Actions, which is GitHub's sort of CICD framework, continuous integration, continuous deployment and how to like version things properly and all that stuff. And so yeah, but then there's another sort of world where like ML Ops, like if you're in a certain organization, they've bought into like a platform like you know, Domino, Data lab, Data robot, these sort of things that handle a lot of those things as well, but at an enterprise level. So yeah, there's just these different. I don't know how to help people navigate through things like that because if they, if they choose any one of those courses, it, it's that, that's good content, right, that they're going to learn a lot. But then if they walk into a company that's doing the other thing, you know, it's not that it's not relevant, but it's sort of a lot of, it's not going to be the same.

Yeah, there's that dichotomy that you are almost going to be required to buy in or invest in some sort of particular flow, an environment, if you will, an ecosystem. You have TensorFlow and Pytorch as two of the giant ecosystems and such, and there are others as well. And you know, we recently were talking about Apache on the show. And so at some point you have to kind of say, okay, well I have to pick something and you're going to pick the one that either your company's doing or that appeals to you, or you read an article, whatever. But it really helps to try to abstract back out a little bit about what it is that those things are trying to do. So you don't lock yourself in and be able to. And at some point you need to go look at some of the other ecosystems and try to map them to what you just learned in that first one. And that way you have the ability to understand how you're doing mlops. It's not limited to a single ecosystem or community on how they're doing it.

Yeah, I think that's a great point. I hope that this discussion, this sort of rambling discussion, has been useful for people that are trying to navigate this road. We'll provide links to all of the things that we talked about in the show. Notes. We'd really encourage people to join our Slack community by going to changelog.com, community, where you can share some of your favorite resources and talk about how you've been learning, deep learning and other things online. But yeah, that, that, that recording went pretty quick, Chris. It was a fast one. Yeah. I feel like there's a lot to talk about and catch up on, but.

Well, we'll have more fully connected shows coming up, more in the future.

Yeah. Thanks for the discussion. It was fun.

Thanks a lot. Daniel.

Thank you for listening to Practical AI. We appreciate your time and your attention. If you enjoyed this episode, help us out by spreading the word. Think of a friend, think of a colleague, somebody who would benefit from listening to it and send them a link. We'd really appreciate it. Practical AI is hosted by Chris Benson and Daniel Whitenack. It's produced by Jared Santo with music by Breakmaster Cylinder. Thanks again to our sponsors, Fast Forward, Linode and LaunchDarkly. That's our show. We hope you enjoyed it and we'll talk to you again next week.

Sam.