Celebrating episode 50 and the neural net!

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean, the simplest cloud platform out there and we're excited to share. They now offer dedicated virtual droplets and unlike standard droplets which use shared virtual CPU threads, their two performance plans general purp CPU optimized They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, cicd, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers or active front end web servers that need to be full duty CPU all day every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very Competitive starting at 40 bucks a month. Learn more and get started for free with a 100 of credit at.co changelog again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practicalai FM and now onto the show.

Welcome to another Fully Connected episode of Practical AI where my co host Chris and I keep you fully connected with everything that's happening in the AI community. We're going to take some time to discuss the latest AI news and we'll dig into some learning resources to help you level up your machine learning game. So I'm joined today by my co host Chris Benson, who's a chief strategist for AI High performance Computing and AI Ethics at Lockheed Martin. And I'm Daniel Whitenack. I'm a Data Scientist with SIL International. How's it going Chris?

It's going great. I'm excited today.

Yeah, me too. How's the week been for you?

The week's been good, just lots of stuff at work, was traveling about a week ago, was up in Boston at LiveWorx and been doing lots of interesting stuff in high performance computing and AI ethics and artificial intelligence as a field just keeps getting more and more interesting in terms of what we're doing and just at large and all the places you can go in it. So great time to be in it.

Yeah, well hopefully I'll survive. I'm working on a little bit of a jet Lag right now. As you know, last week I was in India, which was great. Was in Bangalore, but India was great. But getting back from India was quite a chore. So it turns out I was on an Air India flight, which they don't fly through Pakistani airspace because of obvious reasons.

Yeah.

But then while I was in India, this tension happened between the US And Iran, and the US put restrictions on planes coming to and fro over Iranian airspace, which is the reroute that Air India does, you know, as it kind of goes around Pakistan and then over the Arctic back to Chicago. So finding a route back to where I needed to get turned out to be rather interesting. So I got back a lot later than expected. And yeah, It's I think 2am to about 2am now in. In Bangalore. So hopefully bear with me if I start going off on a tangent.

Not a problem. We'll wake you up though, because this is a special episode for us. It's our 50th episode.

It is. Congratulations on 50th episode. Pretty crazy.

It does. So it's gone by so fast over the past year and just the idea that we've put out that much content and that we actually have people that still want to listen to us after doing that, I'm amazed by that every day.

Yeah, definitely. Thank you to all the listeners. It's been great to kind of gradually get more and more connected to the listeners and listeners engaging on our slack channel, on LinkedIn and other places. It's just really great to hear that you're appreciating some of the content, but also great to hear some of your ideas that we've been able to filter into the show. So keep those coming. Thank you so much for listening. We really appreciate all of you and really want to engage with all of you in our community. So make sure and check that out@changelog.com community but I just got to say.

When we get messages from people out there or people engaging us in the communities and stuff, it is just enormously exciting because it's kind of the reason that we're doing it. And the fact that people are out there, not only are they listening, but they're saying, hey, what about this? I'd love to hear that. And hey, here's a suggestion or hey, I know somebody who would be great on your show. It just, it makes the whole thing wonderful. It's. I know that sounds a little corny, but it's true.

Yeah, it is super encouraging. So keep that coming. We're really excited about episode 50. This is kind of a celebration for us. And so we were talking before the show of like, how what should we do to celebrate episode number 50? And what we came up with was kind of to loop all the way back to kind of where things started with AI and with practical AI. And that's to devote this kind of celebratory episode to one of our favorite things, which is the neural net.

Absolutely.

Yeah. So we thought we would. So we've talked about a lot of neural nets on the show, obviously, and many advanced sort of architectures and applications and all of that, but we've never actually just talked about the neural net itself, where it came from, and just kind of in brief and from scratch, what a neural net is, what makes it a neural net. And we thought this would be a great episode to kind of circle back to that starting point.

Yep, absolutely. One of the common comments that we get back that I've had conversations with several people about, including the young man that is at the Chinese restaurant two miles from my house, because he actually listens to the podcast, but he's not a data scientist. And he made some comments to me a while back. He said, you know, you guys are really good so long as you don't do jargon. And we've been. And we took that as a point to not to be very careful about that. But he said that, you know, sometimes we get a little bit out there for where he's at, and he's very interested in the topic, but we've never really done a true intro to neural net type of show. And it occurred to me that for those people out there who are trying to jump in and may find it a little bit intimidating, I can't think of a better way to celebrate kind of a milestone episode.

Yep, sounds great. So why don't we start with giving just a little bit of history about the neural net itself? So neural nets are not new. They've actually been around for quite some time. Do you know when sort of neural nets came onto the scene, Chris?

Sometime around World War II, if I recall correctly. Do you have the specifics?

Yeah. So if you just. You can just search Google for, like, neural net history, and, you know, there's several lists that come up that have varying, you know, variations of the various dates and facts and all of that. But generally people include kind of a date around the mid-1940s when the first kind of computational model for neural networks came out. It's a guy named. I'm sorry if I'm mispronouncing. This, I don't really hear this name too much, but Warren McCulloch and Walter Pitts created these computational models really that paved the way for both sort of modeling biological processes like actually neurons in our brain or neural networks in our brain, and then kind of more practical applications of neural networks.

Yeah, and then I think there was a kind of the next major step was when the perceptron was invented, and that was by a guy named Frank Rosenblatt in 1958. So we're getting, for me, we're getting a little bit closer to my year of birth. Not quite there yet, not quite that old. But that really set off kind of one of the early waves of research in this area.

Yeah, people are sometimes surprised because there's been a lot of talk about neural networks recently, but maybe they didn't hear it a while back. So these sorts of things have been around for quite some time in research. And like you were saying, moving up through the 60s and 70s, they were a topic of research. But I think that a big shift happened in the 1980s, kind of up to the mid mid 90s. And this is where things like deep learning and back propagation, so these kind of larger neural networks and, and applications to different types of data came, came around. So up until this point, people were researching neural networks, but they hadn't really figured out a way to kind of make them bigger and learn more complicated patterns. So before that they were pretty limited towards like linear divisions between class, you know, linear class boundaries and these different things. But as they saw that they needed to model more complicated relationships, they saw that kind of the size of the networks needed to increase, but they didn't really have a good way of training those sorts of neural networks. And that that kind of changed in the 80s.

That's true. And that's. There's. I have a special affinity for that time period because in 1992 is actually when I first became aware of neural networks. And that's before like the name deep learning was applied to it. And it was before anyone was calling them deep neural networks necessarily. My father worked for Lockheed Martin just like I do. He would have been shocked that I do at this point, but he worked there. And there was an event that really affected me in a very personal way, and that was that there was a fighter plane called the F22 at the time. It was the YF22, which is still kind of the world's top air superiority fighter, in other words, for dogfighting, you might say. And there were two prototypes and One of those prototypes was doing some testing at Edwards Air Force Base, and they were testing avionics on it, and there was a malfunction. And fortunately it was close to the ground and the plane slammed into the ground and went skidding in a fiery ball down the Runway for quite a ways. And the pilot got out, the test pilot got out and got away safely. But as the aftermath of that, both of my parents were on the F22 team, the core team that built the avionics for the plane. And my father was assigned to help solve that avionics thing. And as part of that, he was using neural networks of the day. And he started with feedforward and back propagation, which we'll talk about in a few minutes, and moved on to other architectures. But that was really special because he would come home and there's all sorts of classified stuff he would not talk about, but in terms of the actual science, we'd come home and he introduced me to neural networks and this was our dinner table talk for a while. And, and as he made progress into different areas and he would explain it to me at night and I would ask questions. But you know, before. So that's going back to when I, and I was a college student at the time. And so it was a really interesting way for me to kind of get into it in a very practical problem. And obviously the problem was solved and the F22 is, is in service today. And so anyway, but that's, that's how I originally came to be aware of them.

Yeah, and that's where you first started getting the ideas for this great podcast, Practical AI. I'm sure. I don't know if podcasting was not a thing, but. Yeah, this time was really, really where some interesting things came on the scene that was recognized actually this year with this year's Turing Award, which went to LeCun, Hinton and Bengio for things like back propagation and the sort of ideas around deep learning. And that was big news recently. But there was this kind of time period of the 1980s and kind of up to the mid-1990s where things were getting really exciting. And then there was a sort of die off of interest in these sorts of methods. Some people call this the, the AI winter.

Yeah.

Yeah, that kind of, that kind of led up almost to the, to the mid 2000s. So this was a time when kind of these, these methodologies were, were known. But the problem was that the kind of, as these networks got larger and larger, of course they had more parameters that needed to be fitted and that required more data and more compute. And so there was kind of this lag of the actual data and compute that was needed and along with the adoption that we've seen recently. So that really kicked into gear maybe in the mid 2000s and on where people really had access to a lot of compute, a lot of data, and really were able to plug that into these advanced methods.

Yeah, it really got kicked off by a guy who had been in the field for a while, kind of coming out of this AI Winter by Geoffrey Hinton, and he's kind of one of the legends in this field. And he started research and at first people as he kind of, and I think he continued through that AI Winter, but while everybody else was turning to other things. And I would argue that it was really some of his initial kind of in this latest wave since the mid 2000s that kind of kicked it off. And he is really, I credit him with coming out of the AI Winter and kind of being at the moment that we're at now.

Yeah, and recently of course, Google has switched kind of from a mobile first to AI first approach to, to their business in general. And that's kind of sparked a lot of interest from a lot of other industry leaders as well. So pretty much all the big tech companies now, along with, you know, a host of startups and smaller companies, have really switched to a focus on AI in terms of research and development and the methodologies that are powering their, their products. So AI has kind of at this point become a new layer in the software stack that's enabling new sorts of functionalities in, in applications. And you know, at the core of most all of those AI systems are neural networks. These things that started back in the 40s that were kind of envisioned and built up over time. But the, the core idea is there it is the neural network. Now not a lot of people will kind of argue about what AI encompasses and the sorts of methods that are AI and aren't AI. And there's certainly a lot of methods that aren't just kind of simple neural networks. There's non neural network methodologies, there's a lot of other machine learning type of methodologies, but really the neural network is kind of the core piece that's, that's powering a bunch of things in industry now and really is the focus of a lot of the, the AI research that's going on, which is why we're focusing on them.

That's true. And I have to say that was very well said because the reality is when you put different people in this field, data scientists and deep learning engineers, and you ask them what AI is, you're going to get a lot of different answers. And I was actually at an event where that it almost comically, you know, demonstrate itself in that way. It was an Adobe event which was a live broadcast on Facebook, and I was one of 10 people that came and participated. And there was a lot of stuff we agreed on. But the one thing that all of us had different viewpoints on was exactly what constituted artificial intelligence today. And without. Without delving any further, I just found that fascinating that they introduced us as experts, whether whether we were or not, but that we were, we were positioned in that way, and yet none of us could agree on the. The basic definition of the field.

The Data Engineering Podcast is a weekly deep dive on modern data management with the engineers and entrepreneurs who are shaping the industry.

Go behind the scenes on the tools.

Techniques, and difficulties of data engineering so you can learn and keep up with the knowledge to make you and your business successful.

Can you give a bit of an.

Outline about the motivation for choosing Jupyter Notebooks in particular, as the core interface.

For your data teams?

Yeah, and actually, when I first joined Netflix, it was sort of tossed at me and I was definitely like, are we crazy? And the answer was like, we might be a little crazy.

Go to dataengineeringpodcast.com to listen, subscribe, and share it with your friends and colleagues.

Okay, so we've talked about kind of a little bit of the history of neural networks, and we've talked about, you know, how they came onto the scene and really that they're powering a lot of these big tech innovations. But before we jump into kind of the very, very specifics of a neural network and what it is, I think it would be useful to kind of just give a real broad definition of the supervised learning. And there's a lot of different types of machine learning models out there, some of which are kind of unsupervised and semi supervised. But the bulk of models that people kind of get into when they're first getting into AI and machine learning are supervised learning models. And I think that would be a good framework for us to talk about neural networks within.

Yeah, I was just going to say, when I'm talking about neural networks, in an introductory thing, I may allude to some other things that are out there, but supervised learning is definitely the place to start. It's kind of the basics, and you learn the basics and then you can build on it in a Lot of different directions.

Yep. So let's kind of, when I'm teaching classes, I normally try to introduce some type of kind of model problem that people can have in the back of their minds. When I'm thinking about supervised learning, you might think about like, let's try to model the number of. Or let's say, let's try to model our sales for the month based on the number of users on our website or something like that. Now one way you could do that is by creating a sort of function, like a function in code that would take in your number of users and output your sales. And most often that would include some type of model definition and some parameters. So it's like you might input a number of users and then multiply that by a parameter or a coefficient and out comes your sales. So that's a model definition with a parameter. Now the big thing that separates kind of machine learning functions versus kind of regular code functions is that regular code functions that definition and parameters are kind of set by domain knowledge and someone coding them in. Whereas in a machine learning context, I like to think about those parameters being set by kind of trial and error or an iterative process of looking at a bunch of examples and trying to make predictions for all of those examples and then fitting or setting those parameters based on this sort of iterative process. Overall, that's kind of how I have the picture in my mind. Does that kind of make sense, Chris, or do you have a sort of different view?

No, I think I would see it the same way. I think one easy way to think about it is if you look at solving problems programmatically up until you get to deep learning. In other words, just using more traditional programming, you explicitly are going and giving the program commands on what you're going to do. And you might think of it in a very simplistic way as lots of if then type statements, lots of case statements, and you're having to think of all the things. Whereas this way of doing it, where machine learning the model is learning what it needs to do is sort of implicit. It's figuring it out for you. And it's kind of a different programming paradigm in large. In computer science beyond just, just the what deep learning is. So if you, if you think of it as the, the, the job of training the model is now to go figure out what it needs as opposed to being told what it needs. It kind of puts you in the right frame for learning this.

I would, I would say, you know, some people, when we use the words like you Know, the computer figures it out or the computer learns, they kind of have this visual view of like, oh, I'm going to go put my laptop like in the corner of my office and then kind of sprinkle some special fairy dust on it and it's going to kind of spontaneously start learning things about the world. Really at the, yeah, at the, you know, you don't have some of that laying around in your kitchen or something?

Yeah, I'll borrow it from my seven year old daughter.

Yeah, yeah, maybe. So in reality what there is, is there's always a sort of model definition. Remember thinking of our kind of users to sales, there's some definition and there's some number of parameters that parameterize that model definition. It might be the, you know, the coefficients multiplication or what we call a bias, which is kind of a number that we add on to the definition, but there's some model definition in those parameters. And what we mean by learning or training isn't just that it kind of our computer is at the right temperature and the right conditions and year of the month and the stars align and it starts learning. It's that these parameters are set through an iterative process of looking at a bunch of training examples of examples of what input is and should be and what output should be. So there's a bunch of examples of there's this input and this should be the output. There's this input and this should be the output. And there's a training process which is just another function written in code that iteratively looks over all of these examples and fits these parameters such that the model can then make predictions on new examples that it hasn't seen yet. So it isn't that this, there's kind of the spontaneous learning that happens. It's really kind of something much more benign. It's that there's a bunch of examples and computers are good at repetitive tasks, right? And so we just have the computer look at these examples over and over again and tweak these, these parameters until we, we get a good set of parameters to parameterize this model definition and then we can make new, new predictions. So that first process is called the training process. And then when we make new predictions, that's called the inference or prediction process.

So in the training process we've talked about making the little tweaks and that's called error correction. And so as, as Daniel was talking, talking about, we already, when we're in training, we already know what the ground truth is for any given example. And the model is essentially Trying to find that with where it's trained to up to that point. And then it actually, it says, okay, I have a result in this training cycle, and then I have the ground truth, and there's a difference in the two. And I'm going to use an error correction algorithm to say, what should I do? What tweaking should I do in this case, when my result isn't what I know to be the ground truth in the training set? So it is an algorithm that is driving that tweaking, but it is able to use that algorithm based on the data that it's come upon on that particular cycle.

So let's maybe make this a little bit more concrete now. So we've talked about supervised learning in general, and that there's this definition and parameters that are set. So what does that look like for a neural network? So in a neural network, there's these kind of subunits. I have an overall definition, and then I have a bunch of kind of sub definitions within that. Or you could think about it, if you're a programmer, like a function that calls a bunch of kind of sub functions underneath it. And these subunits or sub definitions are called neurons. And so each of these neurons kind of have its own inputs and outputs with its own definition and its own set of parameters. And these parameters for the neuron are often called weights and biases. So, again, you can kind of think of my overall definition of my model containing a bunch of these sub definitions of neurons that are linked together in some way, and that together, that assembly of neurons make up what's called a neural network architecture. So that architecture just means there's a bunch of these kind of subunits. Each of them have a definition and some parameters that can be set. Now, there's a lot of different ways that you can set up those neurons. So maybe we should look at a kind of common way to set up one of these neurons.

Yeah, like a fully connected feed forward is a good starting point.

Yep. So maybe. Do you want to. Do you want to start there, Chris?

Sure. So Daniel was just talking about these units of neurons. And if you're. If you want to paint a picture in your mind as you listen, you could think of each one of those. The way they're usually depicted graphically is as a little circle. And you could think of it as a little circle that has some stuff inside it, which we'll talk about in a moment. But you take each of those circles, and you line a few circles up into a row. And so you have a row of circles. And then at that point you line up a second row and maybe a third row. And so there's some number of rows, n number of rows that you have there. And there's some special relationships between each of those layers. So if you take for every neuron in that first layer, it is connected to each of the neurons in the second layer, but to none of the neurons in its own layer. And so in that second layer you recreate that. And so each neuron in a given layer is connected to all the neurons in the previous layer and all the neurons in the next layer, but none of the neurons in its own layer. And so you can kind of envision this mesh of rows of little circles in that way. And you start from one side to go in as an input and then you come out the other side. And that is the basic image in your mind of how you might think about a fully connected feed forward network. And I'll note one other thing is these shows where Daniel and I talk about topics on our own without a guest, you may have noticed that they're called fully connected episodes. This is what we're referring to. It was named after this.

How clever. You mentioned that each of these nodes or neurons is fully connected in this sort of, in this sort of network, and each one has its own inputs and outputs. Now if we dig into one of these neurons to think about kind of what's inside of that bubble, and again, you know, you can think about that visually like a, like a bubble or a node, or if you're kind of a programmer, you might think about it as one of these kind of sub functions under, under a big function. But it has its own inputs and outputs. And if we think about it maybe as just having a couple inputs, let's say X1 and 2. Now what happens inside of that circle or inside of that neuron? Well, there's some kind of simple things that happen often. So one way we could think about processing these inputs in the neuron is to just add them up, right? And so in a kind of linear regression sort of way, we could multiply each of my inputs x1 and x2 by a couple coefficients. Let's say w1 and w2, those are often called weights. So I just add up the two things after I multiply them by coefficients, and then I might add in like a, an intercept or a constant. And you know, so just x1, x2 plus something, and that's often called a bias. So in this case I would have like three parameters that I, that I parameterize the way I'm combining these inputs and so each of my X1 and X2 come in, I combine them together in this way and, and that's all good and fine, except, you know, most relationships in our world aren't linear. So it might be good to introduce some non linearity into this combination. And that's where a thing called an activation function comes in, which is just a nonlinear function that's kind of applied to this combination of inputs to give it some non linearity. And common functions that are used are like sigmoid or relu or other functions, hyperbolic tangent, that are applied to this combination of inputs. So when my inputs come into that node or that circle, they're just kind of added up in a special way and then output out the other end. And all of my neurons in my network kind of do similar sorts of simple operations that are parameterized in a similar way. So each neuron has a certain number of inputs, they're combined together using some parameters, and then output is a number. And that's kind of what each neuron does.

That was a very good explanation there. And so as these inputs start flowing through these layers and they're doing this concurrently, so the inputs come in, it hits all the neurons in that first layer simultaneously. Everything that Daniel was just talking about happens in each, each of those neurons in that first layer. And as they go through their transfer function, that adds the non linearity and then they go out. And the output of each one of those neurons in that first layer goes to all of the neurons in the second layer. And as combined, remember, since they're fully connected, there's lots of inputs potentially coming in and they're all summed up again in each neuron just the way Daniel described. And so this happens in concurrent currency at each layer, and it goes through layer by layer by layer until you get to an output. And then you either, you discover at that point that if you're, while you're going through this training process that you have some values coming out and you compare that against what you know to be the ground truth that's in your training data set, you know what the result is while you're trying to train. And that's when your error correction comes in where you have to say, okay, well I've ended up with an output and it's not quite what I was hoping it would be, so I need to change the values throughout the architecture. The initial thing that most people learn and is most widely used is called back propagation. And that's where you work your way back through the layers through a set of algorithms that make little tweaks all the way through your layers. And then, hey, guess what? You've done one full cycle and it's time to go to the next row of your data to train that. And you do that whole process over and over again.

Yep. It's kind of like you might think about if you're trying to set these weights and biases manually as a human, what we would do is just kind of try to make, you know, make an initial choice for them, try to make some predictions and then see if our predictions were good or bad and kind of adjust the parameters accordingly and then just do that over and over. So that's what the computer is doing is essentially a bunch of trial and error. It's making some predictions. And of course there's more sophisticated ways of updating the weights and biases rather than just kind of randomly making choices for updates. And that's where kind of this gradient descent comes in. But essentially we're just making those corrections now. I think an interesting thing to add in here is we're always talking about models and like what, you know, we have a neural network model, we have this type of model. So here we've talked about the kind of definition of the neural network. We've talked about all of the parameters that need to be fit for this neural network. We've talked about the training process that trains or fits all of these parameters. And then we've talked about the inference or prediction phase in which we used all of that to make predictions. So I'm kind of curious, in your mind, Chris, what do you consider the model? Like, what is the model in your mind amongst all of that?

So the way I would think of a model is I think of it when you start out with these layers of neurons lined up. And we're talking about the simplest use case, obviously. And there are. You can add a lot of different complexity to this over time to achieve different architectures. And there are many, many architectures out there. When someone talks about a model, though, I typically think of a trained architecture. If you think of a fully connected feed forward architecture as being something you're training, when it gets done, it has a purpose it's trying to create. Its purpose is to make inferences about a particular set of inputs to give you an output. And that's what I would call a trained model. It's the architecture at work that is deployable. One thing that we did mention briefly is that when you're training a model, how do you know when you've gotten there? And I just wanted to note that it's arbitrary based on your use case, in that we've been talking about the fact that when you get to the end of each cycle in training, you have some sort of delta between what you have and what you know to be the truth. And so that is an error that you have there. It's a degree of error. And you have to decide for your use case how much error can you tolerate? If you can tolerate more error because it's not a very critical need, and if it happened to be wrong, it might not be a terrible thing, then you can probably achieve training quicker and deploy because it's not. If it's a life and death thing and it has to be extremely accurate, then you need a very small error in your final product and therefore you may take quite a bit more training to achieve that. And I just wanted to note that's how you know when your training is over, is that you have achieved an acceptable level of error for your use case.

Chris, you were just talking about kind of the acceptable level of error with a neural network. And I think something that, you know, needs to be understood here is that these nodes or these neurons can be assembled in all sorts of, from simple to very complicated ways. And you could have sort of layer after layer of these that might be fully connected, might not be fully connected. But as soon as you start adding these things up or putting them, assembling them in all sorts of complicated ways, which is really what's done in deep learning, then you start accumulating a ton of parameters. So in some of these, you know, in some of these recent models, let's say, like transformer models that have come out for language, there's millions, in fact, hundreds of millions of parameters that need to be set. And so when you're thinking about the compute and the data that's needed to actually train these models or fit all of those parameters, now you can kind of understand why a lot of data and a lot of compute is needed. Because you can't have like 3,300 million parameters and then like 2,000 training examples and call it good and say that's going to set all of your parameters. You have to have a significant amount of data for you to be able to kind of learn the complicated patterns and fit all these parameters. So a ton of compute and a ton of data is needed.

Absolutely. I think in calling out the scale that you're talking about there is important because it is a distinguishing factor between this particular tool in data science and other tools that we've all worked with previously. I think it's, you know, and people say, well, okay, I understand that. And they're, they. Shortly upon coming into the field, you learn that there is special hardware used for the computation. And people have often asked me, why is that? I've heard GPU and stuff like that. And, and that is the fact that to do these computations which are actually, they're, they're not, they're not complex, but it is a field of linear algebra. It's called matrix multiplication. And as Daniel just pointed out when he was talking about the scale of the parameters. And, and you might have very large architectures with many, many neurons in that that are all concurrently doing these mathematical operations. You, it, it lends, it lends to efficiency to have hardware that is able to do this type of computation much faster than the hardware that came before. And that's why you hear about GPUs and TPUs versus something that we may have all grown up with, which was the CPU driving our laptop and stuff. And that is that this hardware enables the mathematical operations that have to happen at such scale. And the fact that you have that relationship there is, really distinguishes this particular data science toolbox from others and makes.

It expensive in some cases.

Oh, boy.

Yeah. So we've talked about the neural or neurons, we've talked about architectures or combinations of these neurons. We've talked about what it takes to fit all of these parameters of the neurons, but we haven't actually got to maybe what's the most important, important point, which is why do neural networks work? So if you think about kind of what we've done, it's somewhat arbitrary in some ways in the sense that we've just put a bunch of functions all together in a row that combine things over and over. That's kind of simplifying things, but it's really what we're doing. There's inputs, and those are fed through a bunch of things that combine them over and over and then output something that combines that output over and over. And, you know, why does that sort of thing work? And the way I kind of like to think about it, I'm curious about, you know, how you think about it, and I know there's more formalisms we can put around it, but the way I like to think about it is, you know, if I have a relationship, let's say, between some input and output, and I'm thinking of like, again, the users and Sales example that might be a fairly, you know, simple relationship. It might just be a proportional one that I can, that I can model via one or two parameters. And I just put that in and there's a simple relationship there. But there's a lot more complicated relationships in our world. Like if I'm trying to detect a face in an image, there's a lot of important things there, from color to edges to certain features of the face. And it's really hard for me to write down a definition using my own domain knowledge. That kind of is the definition of a model of a face. And so the way I think about neural networks is kind of just saying, well, okay, I'm not even going to try to write down this sort of domain knowledge definition. What I'm going to do is make my model definition as complicated as it needs to be, such that whatever the relationships are between my input and output, whatever those happen to be, I'm able to account for those complexities because my model is parameterized in such a complex way. And so this takes some of the burden off of the programmer, the domain expert, and really puts it on the computer in terms of computation and data. Because all the assumption I'm making is that there is a relationship between my input and output. And if my definition is complicated enough, I'm going to be able to parameterize it to model that.

That is actually a great explanation. I really like how you said that. And I think it differentiates from a number of other approaches one might take. And so when we are using neural networks to solve really complex problems, we'll try. There's a balancing act that we're trying to do. The bigger the architecture, the more computation you're introducing into it by default. But you need it and you can actually have. There's actually mathematically the ability you could have a feed forward network with a single hidden layer. And since we haven't specifically mentioned the word hidden, think about this neural network architecture that we talked about. And you had that input layer of neurons, and then the second layer only takes the output from the first layer and then it passes its output to a third layer, which is your output. So you have a hidden, I'm sorry, an input, a hidden and an output. And there is a mathematical equation called the universal approximation theorem. So you can go look it up on Wikipedia, which. And it notes that a feedforward network with a single hidden layer containing a finite number of neurons can approximate continuous functions. And that sounds like not a very impressive statement to make, but I think it's it's pretty amazing in that it's saying you can approximate all sorts of different functions out there. And I think that's really important because it lends itself to why this is so powerful. And going back to what you said a moment ago, Daniel, you mentioned the fact that we'll add complexity because if you have a really complex function, that one hidden layer might eventually get there, but it may be unreasonable in terms of the time it takes to train it to get there, to get within what is an allowable error for you. And so the way we get around that is we either add more neurons or we add more layers to it. So we deliberately add complexity before we know what the solution is. And in doing that, it gives this matrix multiplication a lot more options on finding all those little things. Here's a line, and here's a line with another line that creates a shape and, and lo and behold, it turns into part of the face or something eventually. And so by having these layers, that complexity allows you to pick apart pieces of it and do it. And so you're balancing how big of a network do I want for computational expense versus what does my problem require? And so when you get into this field, you learn that you have to balance that and then obviously have various architectures that lend themselves to solving particular problems.

Speaking of getting into this field, I think maybe with the few minutes that we have left here, Chris, it might be good to just talk about if you're getting into this field or if you really like, you've done some tutorials with neural networks, but you don't kind of have this fundamental understanding of how they operate. How can you get some of that intuition about how neural networks operate? I know one of the things that, that I did in the past that was really, really helpful for me was implementing kind of a simple feed forward neural network from scratch. And I did that just for the iris classification problem, which is kind of a very well defined, classic problem in machine learning where you're trying to classify types of iris flowers based on the measurements of their petals. And I did this in, in the go language because I was also interested in that and using that. But I think whatever language you use, it doesn't really matter. But just kind of introducing each of these components, like the neuron, the activation function, this loop of training is really, really useful to gain a fundamental understanding. And one one way, if that's kind of intimidating to you, I might recommend the great book from Joel Grus called Data Science from Scratch. He just released a second edition of that book. And he added in a bunch of things about neural networks, deep learning, recurrent neural networks. But in that book he kind of walks you through some implementations of neural networks from scratch using Python. And so I think that's a, it's a really great way to gain this fundamental intuition and something that I think would be even good for me to do occasionally in different languages or in different ways to kind of help me keep that intuition.

Yeah, not only that there are so many approaches. I really think it's such a great time to get into this field right now. I won't call it mature, but it has matured a lot in the last few years. And back when you and I were first looking at it, incidentally, that's what I did as well. Same programming language. I created a toy neural network in Go just to make sure that I understood where I wanted to start from. And all the pieces made sense to me. It was more of a science experiment kind of thing before moving into frameworks, which is where the real action is. But there's a lot of learning. So if you're into books, there's all sorts of different books. There's the deep learning textbook, which was written by several of the luminaries in the field. You got to love your math if you want to jump into that one. If you're very comfortable with your linear algebra and your calculus, then that's a great place to go. If you're not. So then it's a good reference to try to work toward. But you might want to find some books that cater to whatever your knowledge level is. And also there's a whole bunch of really fantastic courses online. Coursera has them. Microsoft, Google, there's a bunch out there. And so whatever your approach to learning is, however you consume new information best, I can almost guarantee there's a high value way of doing that that you can cater it around yourself. I know that didn't really exist when we were doing ours originally, but the last two, three years it's just exploded.

Yeah, there's great online resources. I really like the machine learning crash course from Google. There's of course the fast AI material that's all online that people love. So it's a great time to get into the field. And this is, hopefully this has given you a sense of what neural networks are or given you a refresher in that to really encourage you that like we, we can, you know, get some intuition about what's going on under the hood here and, and that's not too far away. From you. It's within reach. So if you have a passion about this stuff, you know, get involved, dive into some resources. Let us know if you need help finding those resources. And yeah, I'm, I'm just excited about the next 50 episodes that we get to dive into. More about AI Chris.

I am, too. I hope people listening out there will join us in the various communities. We're on slack, we're on LinkedIn, we're on Twitter, and we really do have a lot of great conversations. And as we look toward the next 50 episodes, we really want your input. What do you want to hear about? Who do you want to hear from? What topics are of interest to you? And we really want to build the next 50 episodes around you. Yep.

And congrats again, Chris. Great. Great to be doing this with you and looking forward to the future episodes. We'll see you next week.

See you next week. Thank you.

All right, thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor. Go on itunes and give us a rating. Go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend. Whatever you got to do. Share the show with a friend if you enjoyed it. And bandwidth for Changelog is provided by fastly. Learn more@fastly.com and we catch our errors before our users do here at Changelog because of rollbar. Check them out@rollbar.com changelog and we're hosted on Linode cloud servers. Head to linode.com changelog check them out. Support this show this episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@change law.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.