Stanford CS230 | Autumn 2025 | Lecture 1: Introduction to Deep Learning

What I want to do today is um, give an overview of this class. Uh, I'm, I'm actually curious before we get started. Um, this is now September 2025. Um, how many of you you know, just started at Stanford? Raise your hand. Wow, cool. Awesome. So others pay attention to who just raised their hands and do say hi to them and you know, know, welcome all of the people that just joined Stanford in whatever program. Um, CS230 is a class that we offer in the flipped classroom format. And what that means is that instead of listening to me or Qian, my co instructor that you meet, that you meet next week instead of listening to us lecture at you for an hour, hour, 20 minutes or whatever, um, where she asks you to watch a lot of the video lectures online so that we can then make use of the precious in classroom time for much richer, deeper discussions. So both today and for the entire quarter I would really warmly welcome anyone raising questions, raise your hand and in fact I, I, I find that um, instead of you sitting there, oh, and even though we have a longer session scheduled by the registrar, uh, we'll use usually only up to an hour and 20 minutes for this course. Um, and the goal is to it it turns out that a lot of Stanford students were watching, you know, the lectures on Seagull on the online videos anyway. So rather than us delivering that lecture same lecture year after year, we put a lot more effort to put very high quality lecture videos online. And we'll ask you to just watch that online, which people are doing anyway but just highly edited for offline watching. Spend and spend the classroom time, you know, you know, doing the things that make sense for us to get together in person too. Um, so because you haven't watched any of the lectures yet, I assume most of you have not today we'll even maybe have a slightly shorter session to introduce the class, talk over logistics and so on. Right. Um, as many of you know, deep learning is one of the latest, hottest trends technologies of computer science and AI. Um, if we look at you know say our PhD admissions or, or even Master's admissions, a very large fraction of all students that come to Stanford or apply to come to Stanford want to work on AI. Right? I'm not sure I'm allowed to say the numbers but they are, you know, extremely high as you can imagine. And so my goal and my, my co instructor Kian's goals, our collective goal is through this quarter to help you get to near or at pretty much state of the art with regard to deep learning. Um, and make sure that all of you walk away from this class highly skilled at applying deep learning. Um, and so it turns out that a lot of progress in AI over the last, I don't know, decade, maybe 10, 15 years, was made by scaling. And one of the reasons why deep learning was so successful was because it's good at absorbing a lot of data. So, you know, if I draw a figure where on the x axis, I plot the amount of data.

We have for a problem, um, then using more traditional machine learning algorithms, so, you know, logistic regression, maybe decision trees, using all the generations of AI machine learning algorithms, um.

As you gave it more and more data, the performance of the accuracy of the more traditional algorithms would plateau. Right? It was as if, so take speech recognition. With all the generations of algorithms, even as you fed it more and more data, hundreds and thousands and tens of thousands of hours of speech data, the accuracy would often plateau. And it was as if the older generations algorithms didn't know what to do of all the data that we now have. But what we start to find about 10, 15 years ago was that if you train a small.

Neural network, also known as a small deep learning model, its performance would kind of maybe get better and better. And if you train a medium sized one, and if you train a very large neural network, the performance just keeps getting better and better. And I think the reason that deep learning has dominated the AI scene for the last 10, 15 years is because there is a recipe for training very large neural networks that we can then shove a lot of data into that results in exceptional performance. So I think we started to see this because of some, frankly, some Stanford research papers about 15 years ago. When did the first early work on using Cuda programming and GPUs to scale up deep learning? Oh, by the way, actually, just one fun fact. Um, uh, the first, my first GPU machine used to train neural networks using cuda, which is a controversial thing at the time, it was built by a Stanford undergrad in his dorm room. Right. His name was Ian Goodfellow. But, um, I think that compute server built in the Stanford undergrad dorm room, um, allowed us at Stanford to lay the early foundations of using CUDA at that time, a modern language for training GPUs to train large neural networks. And then, you know, obviously that influenced a lot of people and helped scaling up deep learning takeoff. So I tell that story because sometimes the work that some of you can do in a dorm room or graduate student housing or whatever, or in a lab at Stanford, looking back over some number of years, it can really have a huge impact. And maybe in this class, some of you will do work as impactful as that someday as well. But, so what we started to find was that as you train larger and larger neural networks, they could soak up lots of data and drive exceptional performance. Um, and then there was a research paper out of Baidu which showed that as you scale up neural networks, as you scale up deep learning algorithms, the performance gains are actually quite predictable. So you can forecast, if you buy this many GPUs, throw this compute and this much data at it, what would the performance be? Um, and then later, OpenAI popularized the idea with a really influential paper on scaling laws. Um, and that predictability of how deep learning gets better in performance then drove a lot of the investments in, you know, data centers and, and building very large AI models of lots of data. Um, and so let's see in terms of where this class sits. So in computer science, all of us build on each other's work, right? A lot of the way that computer science and AI has made progress is we build on top of other ideas that are in turn build on top of other ideas. They in turn build on top of other ideas. So maybe I want to give you a little map to show you maybe where deep learning sits.

So I think there are.

You know, machine learning is built on top of computer science, so I think it's actually helpful to learn CS fundamentals. And even though I use, and I suspect vast majority of use AI assisted coding, be it tools like cloud code or Gemini, CLI or opi, codecs or Cursor or Windsurf or whatever, I find that people that know CS fundamentals, that really understand computer science, that really understand how computers work, rather than, like, I'm gonna bytecode this, you know, like, when you understand CS fundamentals, you get things to work much better. But on top of CS fundamentals, there's a set of.

Machine learning skills.

So how do you build algorithms that can learn from data? Um, and then deep learning is a special type of machine learning. Well, that's really the most effective type of machine learning, as far as you can tell, in which we train neural networks, which train certain types of algorithms to learn from large amounts of data. So far in the last 10 minutes, you probably heard me use the words deep learning and neural networks. And I think today those two terms are almost interchangeable. Uh, some, some purists will insist on some technical differences. Raw practice, they mean the same thing. But what happened was the term neural networks had been around for decades. But around 10, 15 years ago, a number of us realized that, you know, deep learning, it was just a much better brand. And so even though neural networks have been around for decades, starting about 10, 15 years ago, it was deep learning that took off. Because who doesn't want learning that is really deep, right? It's just a good brand. Um, but you hear me use those terms kind of interchangeably, but deep learning algorithms, neural networks, they give us a way to take advantage of more and more and more compute capacity so they can build very large AI models with a lot of parameters to soak up the large amounts of data to get more and more intelligent, or to make better and better predictions, or to generate more and more accurate outputs using the large amount of data that's available to us. Ever since I was a teenager, my, my mom's been trying to convince me to stop mumbling.

But now, many years later, I still struggle with that. So I'll, I'll, I'll try my best, but please wave at me or kind of let me know if I start to drift lower again. Yeah, all right. I think my mother would be very happy to have to practice like this now.

Um, all right, so, CS fundamentals, machine learning, deep learning, and then the recent generative AI revolution.

You know, generative AI. Sorry, bad handwriting. Generative AI.

Which is mostly built by a specific type of neural network called the transformer neural network, which you learned about in this class, you actually learn what is the transformer architecture later in this quarter as well, is in turn built on top of deep learning. Right. So I assume all of you, you know, are regularly prompting oms, large language models, to help you get work done. Um, and what I find is that while I use OMs all the time for a lot of applications, just prompting LLMs, it doesn't cut it. There are a lot of things that I cannot get to work just by prompting an LLM. Um, and so I often have to go a layer deeper into the deep learning layer of abstraction and fiddle with deep learning algorithms in order to get certain things to work. And in fact, so what this class covers is we'll try to make sure that you are, you know, expert, near expert in deep learning by the end of this class. But we'll also, um, kind of dip a little bit into machine learning concepts. We'll talk a lot about objective functions and tips and tricks for optimizing parameters in an efficient way. Um, and then we'll also actually reach up a little bit to cover some gen AI. In particular, we'll talk about what is the transformer network. And then through this quarter Qian and I will also chat a little bit about the job landscape as it relates to Gen AI and deep learning and how to use how deep learning is enabling certain types of gen AI applications. Okay, um, I have more to say about this but let me just pause for a second and see if you have any questions. Yeah, go for it.

Oh, good question. When I say prereq by congratulations, you've won the prize for the first question asked in CS230, 2035. So well done.

Yeah. Um, so is machine learning a prereq to this course? Uh, not really. So, So I think two common entry points to AI at Stanford are CS. Well a few common entry points are CS129, CS229 and CS230. Um, if you don't know any machine learning, this course may end up going a little bit fast. May seem like it's going a little bit fast in the first, I don't know, two or three weeks, but some people do pick things up quickly that way and, and maybe actually I should. So a few clauses that you may hear about. Um.

So 129 is a relatively easy entry point into machine learning that slow tends to, you know, takes a longer time to go through the core concepts of machine learning like what's the optimization objective, how do you implement gradient descent, what is logistic regression, what's a very basic neural network. So this is a relatively applied, easiest of the, of, of, of this list. Um, CS229 which I'm also involved in co teaching is uh, much more mathematical and theoretical, very high paced, very intense and very mathematical. And this is less applied than 129 and 230 but this will go over a lot more of the theory and the math derivations behind machine learning algorithms. So for example, if you ever want to learn how to do um, calculus not using real numbers but calculus using matrices and vectors, you know, that's a bunch of sort of, I don't know, mildly complex math that's worth knowing. Um, CS229 goes over that. CS230, this class is relatively applied and it focuses just on deep learning. So of all the machine learning, there are a lot of machine learning algorithms out there, right? Supervised learning, unsupervised learning, a lot of machine learning algorithms out there and many, many of them are very useful. But, and, and so they're all worth learning about. But of all the machine learning algorithms out there, the one category that Is most useful is, you know, that's taken off the most is deep learning and this class focuses just on that. But the other algorithms also worth learning about. So 129 is the easiest on ramp. Um, but if you've done either 229 or 230, I would probably skip 129 at that point. But if you are getting started, there are, there are multiple on them. Yeah.

Can we take 229 and 230 together or do you think it's redundant?

Oh yeah. Can you take 239 and 230 together? Yep. Thank you Price for the second question.

If you look at my clap. Sure, go for it.

Um, yes, you can take CS230 and CS239 together. Um, we designed the two curricula to be relatively low in overlap. So the very small amount of. Oh, opportunities too. Right. All right. We won't clap for every single question, but yeah, go ahead.

Are you going to cover more like recent deep learning algorithms that are used in like recent, like LLM developments?

Like.

Yeah, so when we cover the recent LLM developments we will touch on the transformer neural network but.

Not the latest LM variations in this course. I think a lot of the it turns out that when you go to get a job, well, maybe assuming you go and work in industry, the number of people training LLMs is actually very small. Right. Some of those jobs tend to be incredibly well paid. So we hear about kind of very high salaries in news media. But the vast majority of application builders end up sometimes working at the gen AI level. Not that often training a transformer from scratch, but then often using deep learning tools as well. So maybe one example something that many of my teams have done is we have trained transform model foundation models from scratch with relatively small ones in say startups. But one thing we do do quite a lot is take a pre trained transformer network, uh, and then engineer our own data to further fine tune it. Right. Uh, sorry if I'm using words that you may not totally understand. You know, pre trained, fine tuned, you know what all those terms are by the end of this quarter. So those are things that we actually do, um, uh, kind of day to day. This is important for getting a bunch of practice of work so you will gain the foundations needed to do type of work in this course. Um, the one thing, one thing we don't do is, is talk a lot about how to train the largest cutting edge transformer networks. I think that is a very important skillset is a Relatively niche one for which some people are, you know, getting paid really, really well. But the number of people doing that in the world is actually small. Whereas the number of people building applications with you know, this set of skills is, is, is very large.

Dynamic course is covering that. I think Percy Liang was thinking about doing something, but I don't remember his doing it. Quarter few people are think thinking about doing something like that. Yeah.

This is a question about the course itself. Do you know about what proportion of the course is going to involve like mathematical analysis and what proportions coding?

So just repeating for the, for the mic, for the home viewers, what portion is coding versus what proportion is mathematical analysis? This course is relatively math light. Um, sorry, maybe that was too strong a statement, but I think this, this course is very practical. Um, I, I, I, I remember many years back I was speaking with a mathematician, um, and you know, we were just chatting and he was asking, he was just talking about his career, why he chose to be a mathematician. And I, I still remember, you know, this, he had, he had like stars in his eyes when he told me that he chose his career path because he felt his role is to pursue truth and beauty in the universe and that's why he became a mathematician in this course. I'm not going to do any truth and beauty stuff. Right.

Truth and beauty is good. But you find that I want to take a very practical approach to um, talking about how to build applications and build software that works. Um, yeah. Cool. Anything else?

Right, cool. Awesome. All right, thank you for all the questions. Please keep them coming and feel free to interrupt me or Kian throughout this quarter as well with questions. Uh, love it. Um, so just to, just to flesh this out a little bit more, this is what I see in terms of teams building practical applications and I'm excited about applications because with improving machine learning algorithms, deep learning algorithms, Jenny, algorithm. There are a lot of applications that you can build now that just were impossible or really inaccessible to any person to build even a few years ago.

I find that when I prompt Genai, it works really well for a lot of text based applications. And there's work on multimodal LLMs, large multimodal models, so making inroads into vision, making inroads into audio. But really Genai algorithms, especially transformer networks trained to output text, you know like ChatGPT, Claude Gemini and so on. Really fantastic for text based applications. Right.

And I find myself.

Regularly working with deep learning algorithms directly when I am working with audio data.

Image and video data, and then also a lot of Structured data.

Ah, sorry, my handwriting is painful. Right, so structured data refers to large tables of numbers, right? Like giant, you know, Excel or Google sheets, spreadsheets. So that's, that's structured data. Unstructured data refers to text, audio, images, maybe video. Um, and because a lot of Genai large language models like ChatGPT had grown up being text in, text out kinds of machines, they are remarkable for a lot of text processing applications. But for other types of data, I end up often, you know, dipping down directly to use various deep learning algorithms. Um, and then it turns out that for text based data, if all you do is prompting, you could usually go quite far. So a lot of applications are built by prompting oms. But, um, uh, but I've been on, you know, quite a few teams where after fiddling with the prompts for a month, you just can't get the performance to be better just by tuning the prompts. Um, or another good problem to have. It turns out use of Genai tools are relatively inexpensive when you're prototyping, right? It's like, you know, a few dollars per million tokens. You can do a lot. Um, but sometimes if you're lucky enough to be on a product that his product market fit and a lot of users want to use, you know, multiple times I've been on teams where we basically did not care about our large language model bill, right? It was like whatever, you know, $20 a month or $100 a month was fine. But when more and more users start using it, then to your team's kind of positive surprise, your AI bill really starts to skyrocket. And then at some point you look at how much you're paying for your AI bill for the large language models and to bend the cost curve back down often while the techniques in deep learning become very relevant as well. So there are, I don't know, I'm thinking just some of our bills were really breathtaking. I don't want to say the numbers, but just kind of definitely more than we want to pay. As much as we love the companies providing OLMs, our bills that we're paying them were significantly larger than I wanted to pay. And then knowing how to use deep learning to fine tune smaller models, that was really the critical skill set that just bent the cost curve back and just made the whole thing affordable to keep on providing a service. Right. So, um, yeah, right. So that's what, so that's the skill set I hope you get from this course.

Let's see.

All right. Um.

And to give a.

Sorry Let me use this one.

So to give a fi. To give a quick overview, this course, the online materials is broken down into five modules. So just give you a overview of the five of them. First one is on the basics of neural networks. NN Neural Networks and Dr. Deep Learning. Right. So you learn how to build a neural network or deep learning algorithm from scratch in Python. Um, I find that sometimes if you use the frameworks like TensorFlow or PyTorch, it hides a lot of the details. So actually work through how to build a basic neural network and how to build a basic deep learning algorithm just in RAW Python so you really understand it. Um, and then the second module, the second mini course will be on, um, how to improve how to tune your neural networks. So you may have heard that when you're training a neural network there are a lot of parameters, or we call them hyperparameters. Hyperparameters are parameters that control the parameters. Right. So the weights are parameters. Hyperparameters are things like the learning rate or what's the size of a neural network. And so there are actually a lot of hyperparameters that we end up tuning and try to give you a way sense of what are the most important ones and practical skills for tuning them. It turns out that.

If I look at, you know, like my PhD students, I think every one of them that. Right. Well, I think, I think that probably every one of them, definitely everyone that every PhD student I know that became great, I think at some point wound up up at 2am tuning hyperparameters. Right. Um, and I still have very clear recollections of like being in the office, you know, 2:00am, 3:00am filling the parameters, trying to get it to work. And it turns out that literally your skill at tuning hyperparameters, it really makes a difference. So there were some evenings that I knew my skill at tuning hyperparameters. You know, frankly it made the difference between whether I went home to sleep at 3:00am versus where I went home to sleep at 7:00am Maybe there's not, maybe don't do what I do. I'm not encouraging that behavior. But, but this is just my personal experiences. But it really makes a big difference your practical skill at how quickly you can figure out the recipe to get these neural networks to train. Um.

And.

And related to that is.

One thing we chat a lot about in this course is strategies for building machine learning projects. So it turns out that if you build a complex system, you know, let's say you built this one example, go to later this quarter. So say you want to build a system that recognizes your face, a camera that recognizes your face, your friend's face, unlock a door, right? Security, safety, whatever. So something like that. You know, I've worked on systems like that. Chance works on that. These are complex systems with multiple components. There's a camera, there's, you know, do you subtract, clean up the image? Do you cut out the face? How do you register the face? How do you compare a face? How do you decide to take another picture before you unlock the door? Or just, you know, or someone trying to fake hold up a picture printed on a piece of paper? There are actually a lot of decisions. And so what I find is that the biggest difference between a team that knows how to drive forward a project like this well and get it done in days rather than weeks, or weeks rather than many months, is the ability to drive a day. Disciplined development process. It turns out when you have a complex system, less experienced teams will often almost pick things at random to work on, right? They'll read one research paper and say, oh, I read in the research paper we should get more data, you know, well, because like some newspaper said, AI needs lots of data. So let's go spend six months to collect more data. Turns out a lot of the time collecting more data does not help your application. Um, but sometimes it's a huge help. So given your application, how do you decide? Should you spend more time collecting data? Maybe you should buy more GPUs. I, I, I should definitely know people that read in the news a lot of GPUs are helpful, right? And then I've literally met, you know, fairly senior business leaders that have spent a very large amount of money buying GPUs. And then, you know.

Some funny stories and then, and then I go talk to them and say, what are you doing with these GPUs? And then sometimes, you know, there was one meeting I was in where literally a very large family run business had bought a lot of GPUs. And the CTO then pointed to his nephew who was a, who was a current college student, undergrad and circumstances. Oh, my nephew knows AI. I'm giving him this very large budget in GPUs and I think he'll do AI for me, right? And so, but, so I think that knowing how to make these decisions and not just buying into the hype that you read about in the newspapers is really important. Um, and one thing I hope to do in this course is share with you what driving a disciplined development process looks like, because this is one of the things that really makes a 10x difference and the speed with which you can get something to work. I've literally seen teams, you know, like take six months or 10 months pursuing an approach that experienced engineers would go in and go, you know what, I could have told you six months ago that spending all this time collecting data, this was not going to get your application to where you wanted to go. Right. But so how do you examine an application and figure out the diagnostics to figure out what are the productive things to do for your application? We actually spend a lot of time on that and in fact, I'm excited about doing some simulation exercises in this classroom with you later this quarter where I'll invite you later this quarter, you know, to, to, to say in this, in this scenario, what would you do and, and see if you could, you know, make decisions in a more systematic way. Right, um, all right, um, then course four, we'll talk about con. Convolutional networks. Um, very useful for computer vision applications. Um, and then.

So convnets are specialized models most u used mostly for, uh, vision applications dealing with images. And then we'll talk finally about sequence models.

So sequences could be time series or sequences of text like words. So also touching the transformer network that, you know, that power law, the Genai revolution. Okay. Um, and so throughout learning, through learning all of these things, I hope that you gain a large tool chest, um, that will enable you to tackle an almost bewildering range of applications. I think one of the things I've most enjoyed as an AI person is it turns out when you work in AI, there are so many other teams that have data and they could use our home. Right. So I feel like as an AI person, you know, I somehow bizarrely had a right to play in, you know, building autonomous helicopters or hoping companies, I don't know, place more relevant ads. Maybe not the most inspiring thing I've worked on, but certainly very lucrative for some companies. Right. Or improve web search rankings or improve safety, get rid of the kind of negative toxic results. You may not want search results to come back on. Or improve e commerce retailing, or improve speech recognition or help ships be more fuel efficient. Right. All these are real examples. Or fight fraud, which is actually really exciting when you're fighting financial fraud, which is obviously a bad thing. But when you, sometimes you, when you're fighting fraud, sometimes you wake up in the morning and your team's alerted you that there's a new scam and then you just have to Go and fight them and build algorithms in real time. And you know that every hour you take, you know, more, more money actually leaves on the financial system. So it's kind of awful that there's financial fraud, but it's actually one of the most exhilarating things I've worked on because literally every hour you are slower to, you know, formulate response, you know, more dollars are leaking up every hour. So anyway, so, so somehow when you have this tool set of deep learning, you just have a bewildering right to play or ability to play in a huge range of applications that could use your hope. Right. And I think on campus too, there's so many departments.

Across campus in the sciences, engineering, humanities, business, that have interesting data where your skill set will let you, if you choose, collaborate with them to do interesting projects. And I find, for example, bizarrely.

Some of my PhD students are working on climate science. It's like, what do I know about climate science? Right. I wish I knew more. But using machine learning tools, you know, we could actually work on climate modeling and geoengineering and just play in all of these important.

Important, hopefully important and interesting places. I hope that you have that skillset as well by the end of this quarter.

Yeah, I don't know if you have enough data for neural network versus like, it's better suited for other work like machine learning.

Yeah. So how do you know if you have, how do you know if you have enough data for a neural network? Um, it turns out to be really difficult to know. Uh, so if it's a application that others have worked on or that you've worked on before, then you may have a sense. So for example? Uh, no, I, because I worked on face recognition, you kind of have a sense. If you want to train a face recognition system from scratch, having like 50,000 images, 50, 50,000 unique phases is pretty good, right? Or so if you worked on it, or if you read the research literature for something people have worked on, that would give you a gut sense for how much data could be enough to get you started. But then for green field, brand new projects that no one in the world has worked on before, if you can't find parallel projects that are kind of comparable, it could be really hard to tell. And so common advice for completely green field. Sorry, green field. I mean, a brand new project, dissimilar than things. So for example, if someone has invented a new medical device and no one has collected this type of blood specimen data before, it's really hard to tell. And in that case the Most common advice is get a little bit of data and just try training a model and the degree to which your initial model works or does not work work. That will help you hone your perspective on how much data may be needed. And you may be surprised. Maybe 100 data points is all you need, right? Sometimes we've been surprised by that, and then sometimes we've also worked on applications where, you know, 100 billion data points later, we're still trying to get a lot more data. And I. I find it really difficult to tell. Good question, though. Anything else?

Oh, so this is a good time for me to pause and take questions because so what, what I'm going to do is.

Right. What I'm going to do after this is switch tracks and talk a little bit about exciting trends in AI, kind of recent trends in AI that I'm excited about and how I view the AI landscape. So this is actually a good breakpoint to see if people have other questions before I talk about some trends in AI. Anything else? Yeah, please.

When you were differentiating AI and deep learning those different tasks, what were like the distinguishing features? Transformer versus non transformer, or how would you describe the difference?

So I guess, let's see all of these. Many of these terms are blurry and kind of fuzz a little bit into each other. But when I refer to generative AI, generative AI is this body of work that generates text and sometimes also images, sometimes also audio using deep learning algorithms in certain ways. So I think Genai refers to this body of work with most of the center of gravity on generation generating text, you know, maybe also images. And the text generation algorithms have been mostly implemented using transformer neural networks trained on large amounts of data, you know, straight off the Internet and elsewhere. So when I refer to Genai, I guess that's one particular application of deep learning models that has given us large language models like ChatGPT and Claude and Gemini and Metalama and so on. That makes sense. Yeah. Cool. All right. Anything else?

Cool. All right, so let me. Could we go to the slides, please? One of the nice things about this class is Ken and I can occasionally just share a view of things we're seeing in the broader world. Hey, while doing that, I'm actually curious how many of you use a specialized AI assisted coding tool like Cloud Code, Cursor, Genici Codex, Kurt Winserf. Awesome. Almost everyone, but not everyone. Interesting, huh? Interesting. Oh, I thought, interesting. Okay, cool. So, you know, one of the most exciting things that's happened in programming is AI assisted coding. And I feel like I personally hope I never ever have to go back to coding by hand. Right. It's just, it's actually interesting. I often work in coffee shops on the weekend and a few weekends ago I was sitting in a coffee shop and sitting next to me was someone that was, you know, coding by hand and looks so strange. I just asked them, what are you doing?

In a respectful way, you know, in a time. Turned out that they're doing homework from some other university that required the code by hand. But one of the things I find exciting is that.

Individual programmer productivity is much higher than it ever used to be. Right. And maybe I want to share with you just one thing, what I see. So I find that in the software work that I do, I maybe categorize it into two buckets. One one is building quick and dirty prototypes to see if something works. And then sometimes I, you know, write production grade, enterprise grade, robust, reliable software that has the scale. Right. Um, and I find that it's where AI assisted coding has made the biggest difference is building the quick and dirty prototypes. Um, whereas I think actually literally one of my collaborators used one of the agent decoders did I name, but I won't say which one. But literally this most morning he sent me a slack message saying, sorry, you know, this agent encoder had a database migration error and we just wiped out all of the database records, fortunately for a test application with like five users. But it did happen. So I find that. Oh, oh, thank you. So I find that my use of the agent decoders, you know, for the production grade software is more careful, whereas the for building quick and dirty prototypes, it kind of. So unless you're not shipping software in an irresponsible way, quick and dirty prototypes have lot fewer dependencies. You don't need to integrate with legacy data infrastructure. And then, you know, I'm going to say something that feels like something I'm not supposed to say, but I'll say it anyway, which is I find that.

When I am running code, I find that people often worry about safety and reliability of software or security of software. So one thing I often say to my teams is if you are building a prototype that only runs on your own laptop and doesn't use any sensitive information, so there's no risk to sensitive information then unless you are planning to, you know, maliciously hack into your own laptop. Right. The security requirements can be lower. And so I find that when building quick and dirty prototypes.

Having a sandbox environment that lets you operate within it quickly means that you can just make a lot of decisions faster without worrying as much of a scalability or security or reliability. So unless the sandbox environment means this stuff isn't going to get out there or leak information or create a security loophole. So that's part of what lets us move much faster. And so I find that because of the speed of prototyping, so less you can do so in a responsible way. To pursue innovative ideas, my teams will increasingly, you know, try 20 things and see what sticks. And because I know that a lot of teams are lamenting that many proof of concepts never make into production, right? You try something and it doesn't work. And I know some teams are feeling angst about that. I actually have a different view. I think if the cost of a proof of concept is low enough, then who cares if you have to do 20 of them? And that's the price for finding the one or two things that works really well. So one thing you hear about in this course is both when you're building a machine learning application, you usually don't know what's gonna happen. Um, and there's specific reason for that. The reason is the output of a machine learning algorithm. It depends both on the code you write as well as on the data you're training on. And while you control the code 100%, you don't really know usually what's really in the data, in the weird and wonderful data that, that the world has given you. So, for example, worked on speech recognition a lot in multiple companies in multiple contexts. And even now when I work on speech recognition, I'm still, you know, sometimes a little bit surprised that, oh, this data has people of a certain accent more than I realize, or people somehow speak faster or, boy, there's a lot of background noise when people use it in a car. Right. So even though I work on speech recognition multiple times, oh, and actually one recent example application, I was actually surprised by the number of background speakers. They talk to us, they come and talk to a different person, then they talk to us and then we get confused. Right. So I find that the data that the world gives us is often weird and wonderful. And so it is only by building a system that you then start to discover these things in the data that lets you make progress. Um, and with a lot of software applications as well, separate from machine learning applications, a lot of what I end up having to discover is what do users actually want, Right? So again, I control my code 100%. I can write whatever code I want, I control that. But you don't get to control how your users will react to your system. And I find that our ability to build quick and dirty prototypes rapidly, um, both to discover what's in the data and also to tickle the users to see if they like it, that allows us to drive faster feedback loops than was ever possible before to then help us build more and more valuable software. Right. Um, and I think, um, you know, I, I, I, I know that the mantra move fast and break things, right? Got a bad rep because it broke things. And I find that some teams took away from this that we should not move fast, but I think that's a mistake. So what I usually tell my teams is move fast and be responsible. And despite all the hype about, you know, AI extinction risk, AI curls all that somewhat bizarre hype in my opinion, I find that when teams move really fast, we can then implement things, test it out in a responsible way and much more quickly identify problems and fix them. So I find that a lot of the most responsible teams I know, teams able to really get stuff to work really quickly. They tend to be some of the fastest moving teams because it's that speed of execution that lets you finally implement it. Figure out what's in your data, figure out what uses one, and that's the best way to figure out what could actually go wrong to then make sure things don't actually go wrong. Right. And.

Somewhat related to that is.

AI coding assistance. And I, I, I assume you know, almost everyone or everyone in this class knows how to code. Um, uh, if you, if you are well, you haven't learned to code yet, you probably might not want to take this class yet. Um, but I find that there have been people, including very senior, right, business leaders, advising others to not learn to code on the grounds that AI will automate it. And I want to share this with you not because I think you need to learn to code, but because I hope you help me spread the word. Right? Go to all of your friends in other departments to tell them this advice and not learn to code. I think we'll look back on this as some of the worst career advice ever given.

And the reason is when coding becomes easier, more people should do it rather than fewer. So when humanity went from punch cards to keyboard and terminal, that made coding easier. And so more people learned to code. Um, when we went from assembly language to modern, well at that time, modern programming languages that made coding easier, more people learned to code. I, I went back, I actually found these papers on um, when uh, these articles on Wen Cobo very Old school programming language, right, was invented and there were actually people that said oh wow, now we have the COBO programming language. Coding is so easy, who needs programmers anymore, right? And obviously the opposite happened. Went from text editors to IDEs and then you know, AI assisted coding, as coding comes easier, people should code a lot more. A lot more people should learn to code. And the other thing I'm seeing is I, I just add just something like this on people's minds. I know that unemployment of recent computer science graduates has ticked up to higher than it's been, you know, compared to I think the last decade. Um, and so I know that to people learning CS that has caused some consternation. So I'm going to share my view on that. So, um, it turns out that what I see in Silicon Valley and beyond Silicon Valley value is we just can't find enough people with these skills. Right? I know many businesses that would, well I know large businesses that love to hire 1000 people with skills in gen AI and or deep learning and or machine learning, but that are struggling to find people with these skills. Conversely, there are still, you know, universities with curricula that has not changed since, has not changed much, right. For the last like, I don't know, since 2002 before the rise of Genai. And so I do see that many new CS grads, not from Stanford but from around the country are struggling with finding jobs because unfortunately that older non AI enabled skill set that is, you know, not as much in demand, right? And maybe just for myself today, I will not hire someone, I will not hire a software engine that doesn't know how to use AI to help them with clothing. It just doesn't make sense. Same reason why I just won't hire someone that uses a punch card instead of keyboard and terminal. Right? And I think when the world evolved from punch card to keyboard and terminal, people still had punch card jobs for a while. But eventually the punch card jobs just went away. It just doesn't make sense anymore. So today there are still coding by hand jobs around maybe some, some specialty, some very low level coding where AI is not very good. It turns out AI is not very good at sometimes the GPU programming. You know, there are some niches with coding by hand actually still makes sense but for building applications.

You know. Oh. So I, I, I actually I, I, I remember this was a few months ago now, many months ago now where I interviewed two engineers back to back. One had not yet graduated from college but was highly on top of gen AI coding. So you know Spoke with that candidate, knew how to use AI, build programs, get set up quickly. Right after that I also interviewed someone with 10 years of experience as a full stack engineer, but whose skillset was exactly the same as the 2002 skillset, had not tried out any AI assisted coding, really good skill, full stack engineer with 10 years of experience. And it was actually really clear to me I picked the fresh college grad. Well, he was about to graduate over someone with 10 years of experience. I think making sure you master these skills are really important. What I'm seeing is there is a very large gap that businesses are having a hard time filling for people with these skills. Um, but the demand for the 2022 skill set, software engineering, full stack engineering skill set, that is not there. Right. Um, so, so I think, and then in terms of AI assisted coding, um, I find that CS fundamentals really are important. So in addition, so I know I have quite hired someone fresh college grad over someone 10 years experience. Right? True story. There's actually one other part to this story which is with, with, with respect to all of you about the graduate from college, the best programs I know are also not fresh college grad, no disrespect intended to fresh college grad, some above the graduate from Stanford. The best programs I know are really on top of AI assisted coding and additionally deeply understand computer science fundamentals. Right? So it turns out, maybe I'll illustrate this with a quick story. When I teach an online course, my team wanted to generate background pictures like this, you know, just for decoration. So when I was working on this, this is of course generative AI for everyone. I was working with a collaborator, Tommy Nelson, that understood art history. And so my collaborator knew the language of art. He knew the artistic genre, inspiration, the palette, so he could prompt midjourney AI image generation with the language and art, and so he could generate beautiful pictures like these. In contrast, I don't know art history, I wish I did. And so all I could do was go to midjourney or AI image generation and I type, please make pretty pictures of robots for me. And I could never get the control that my collaborator Tommy could to generate pictures like these, which is why we use all of his pictures and none of mine. Right? And I'm seeing the same thing in computer science. Um.

One of the most important skills for the future is to understand how computers work and understand how genai and deep learning and machine learning work so that you can use the language of AI, use the language of these tools to tell a computer exactly what you want so the computer can do it for you. And there is actually a huge difference in performance between, you know, someone that's learned to just prompt an LLM without understanding how computers or how AI really works versus people they can look at the problem, analyze it, and then with AI assisted coding, you know, tell a computer who to take the next steps. Which is why I think that CS fundamentals is very valuable. CS fundamentals, Machine learning fundamentals, deep learning fundamentals. I and my teams, we use that knowledge, like every day, right. In making pretty consequential decisions. Right. So I hope that you get that from this class and the many other classes at Stanford as well. Great. Um, all right, I think I might leave the rest there. There's more I could say on trends in AI, but I find that.

AI assisted. Oh, but, but one thing I hope you do really, you know, go to all your friends in all the departments across campus and encourage them to be a builder. Because the other thing I'm seeing is, um, clearly for computer science professionals, you know, use AI statistic coding, no, CS fundamentals build cool stuff. But for other disciplines as well, that is not computer science or not AI. I'm finding that, you know, the education professional or the climate scientist or the mechanical engineer, the ones that know how to build software are just more productive and get a lot more done. And the barrier to entry to AI, to coding is the lowest it's ever been, you know, in our lives. And so this is a good time, frankly, for I, I, I wish every single Stanford student, right, will learn to build software with AI assistance. So I hope you go help your friends across campus to master those skills as well. Okay.

Yeah. Any, any other questions? Yes.

Someone knowing.

AI, I heard a lot of, like, a lot of the industry folks talking about how they would rather hire someone with 10 years of experience who knows, just train AI as is like just using coding into the system rather than fresh undergrad that does come like that deep understanding of deep learning, machine learning and like.

Just because.

They have more experience.

Really?

Do you see that also?

Yeah. So let me rank productivity, right? And I'll give you four levels, I think. And again, and I say this with a lot of respect for individuals, if I talk about productivity, it's not with any disrespect or any lack of affection for anyone or for their work. Right. But I think, you know, least productive are people with no experience and don't know AI. Right. One step on top of that is people with less experience. But on, on, sorry, one step on top of that is people with say, a decade of experience but that don't know AI. Um, on top of that, I would rather take a fresh college grad that does know AI, but then even more productive is someone with, you know, a decade of experience and also really on top of AI. So I think between the two factors, really understanding AI is very important, but experience is also important. And so the best developers I know, we just work and we just ship code like no one's ever done. I think even two, three years ago are very experienced developers. They're also very on top of how they use the latest AI technologies.

Please. Oh, oh, sorry. And just one other thing about the job market. I find that a lot of employers have not yet figured out how to hire appropriately. This is contributing, frankly, a lot of employers. You know, if a company has no one that knows Genai, how do they even know how to interview appropriately? So that is a problem that we need to solve as well.

For example, passes at CS107, 111 where we're not encouraged to use and we're encouraged. Is that true of the fundamentals?

And do you think that's necessary? Yeah. Boy. So CS17, CS111 are great. So do take them if you're considering them. I find that the fundamentals are important.

How to put it? Yeah. Not sure what else to say other than that. Um, I think, I think honestly Stanford were known for CS department were known for. Really. I know I'm biased, but I want to say like probably the best entry level CS proven classes of any university in the world. I'm pretty biased, so maybe I shouldn't say that. But I think the excellent course if you want to learn the fundamentals in a really solid way. And I know the instructors are routinely thinking about how to update the curriculum and realities of Gen AI, so I, I think they do an excellent job with that mix.

Anything else, please.

Compare it to person. Just pull out the cursor and handle prompt.

Yeah. How do I define someone that really knows Genai compared to someone that just uses a tool in terms of prompts? So I feel like in Genai there are two buckets of skill. And again, by the way, deep learning is not Genai. Deep learning is also a very valuable skill, but centralized by Genai. I think there are two things. One is I find it really useful to know how to use AI coding assistance. That's really valuable. But having, you know, kind of fundamental knowledge helps. Right, you do that. And then the other thing is in Genai, there's A number of emerging tools. I'm going to toss up some buzzwords. Okay, so if you don't know what any of these words mean, don't worry about it. But. But I think there are emerging tools like RAG retrieve, augmented generation or how to vector databases. How to do evals and error analysis, how to build guardrails, how to use knowledge draws and interface that with your om. Maybe how to do multimodal oms, how to fine tune a model. What else? I'm probably blanking on something. Oh, how to build agentic workflows. But I feel like there are these categories of new techniques built on top of gen AI that are like a useful bag of tools for building applications. So when, well frankly when I'm interviewing candidates, you know we still do a fair amount of as far gen AI role is these skills. I. I tend to look for this set of tools as well as AI assisted coding and then. Yeah.

Please.

I don't know if this question is best posed to you or if I'm considering taking 29 and 2:30 at the same time. Classes and final projects. So how would you recommend thinking about ideas to pursue?

Yeah. So let me just give One tip about CS courses in general, not just 229 and CS230 which is. I encourage you to think of AI courses at Stanford. A bit like Pokemon. You got to catch them all.

But in all seriousness, I think taking more CS courses in AI it is a good thing to do. Definitely encourage you to take multiple courses. I think in some years we've had students do joint projects where the standard is higher. Higher expectations for sure. But that's one option to look at. Yeah. All right, go for.

Oh, so let's see. 229 and 230 are very different courses. Um, 229. Well in 229 you, you. We have live instructors doing the lectures in person rather than online content. So it's not the flipped classroom. So in CS230 um, we have most of materials prepared kind of in online videos, highly edited online videos. Um, uh, and then.

I think the big, the other biggest difference is CS229 is more mathematical, theoretical, but it's important Math. Right. Whereas CS230 is more practical. And so I don't know if we do even want a single maybe we do like one proof somewhere in the lectures but we just don't do a lot of math in CS230. And it's very, A lot of machine learning is very empirical. You should try to see what works. But at end, the disciplined approach for exploring what works, we focus a lot more that in CS230. And. Oh, and 229 covers a lot more techniques, right? So there are a lot of machine learning techniques, supervised learning, unsupervised learning, you know, I don't know, decision trees. Boosting K means clustering. So CS229 covers a much broader survey of a lot of machine learning techniques. Whereas CS230 is just one thing. Well, deep. We go really deep into deep learning.