AI code that facilitates good science

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean, the simplest cloud platform out there and we're excited to share. They now offer dedicated virtual droplets and unlike standard droplets which use shared virtual CPU threads, their two performance plans general purp CPU optimized They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, cicd, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers or active front end web servers that need to be full duty CPU all day every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive starting at 40 bucks a month. Learn more and get started for free with a 100 of credit at.co changelog again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're at Practical AIFM and now onto the show.

Well, welcome to another episode of Practical AI. I'm Daniel Whitenack, I'm a data scientist and I'm joined by Chris Benson, who is a chief strategist for AI and high performance computing at Lockheed Martin. Hey Chris, how you doing?

I'm doing good. How are you today?

Doing pretty good. Made it through the, through the fourth of July without blowing myself up or anything like that. Always a good thing. How about, how about you? Did you have a good holiday?

I did. Just family stuff. I got kissed by a llama, I'll have you know. I went to a friend's house, they had llamas and the llama came up and implanted one right on my face. So that was an interesting experience. I was not sure what to do about that. I declined to take it further with the llama. I decided that I that was not the relationship I wanted to pursue, but that was, that was the highlight of my force. Sadly.

Probably, probably a good thing. But yeah, well, on on that note, and maybe a more more relevant note given that this is, this is practical AI in our in our community on Slack, which you can find by going to changelog.com community but oftentimes we're asked maybe what are good practical ways to start learning about AI? Maybe from. From software engineers or those with an engineering background who are getting into AI and data science. And pretty much, I always tell them that a great place to start is with Joel Grus's book, Data Science from Scratch. That was one of the kind of instrumental things in helping me understand what data science is. And I frequently go back and reference that book. Well, Joel has joined us today. I'm really excited to talk to him about his book, which now has a second edition, and also some things he's doing at the Allen Institute for AI in Seattle. So welcome, Joel.

Thanks for having me here.

Yeah.

Maybe you could start by just giving us a little bit of background about yourself, how you got into data science, how you got to Allen AI and what you're working on now.

Right. So originally I studied math and economics. This was a long time ago, well before data science was a thing. And so what I did, what a lot of people with math backgrounds do, is that I went into quantitative finance, options pricing, financial risk, things like that. And I discovered I didn't like it very much. And so I was working at a hedge fund and it went out of business and I got laid off. And I was lucky enough, and this was 2006, to get hooked up with a startup called Faircast, which was an online travel site, doing basically price predictions on airfare. So they'd use machine learning, and you'd say, I want to fly from Seattle to Los Angeles on these dates. It would tell you the best price is 300 bucks on Alaska. And we predict the price is going up, so you should buy now. Or we predict the price is going down, so you should wait. So they were doing a lot of machine learning before putting machine learning and consumer products was particularly popular. I was not doing machine learning. I was doing bi writing, SQL queries, building spreadsheets, things like that. But as time went on, I started getting a little bit into the machine learning side of things, a little bit into the Python scripting side of things, and just started to grow my skills in that area and learn by osmosis from some of the data miners, as we called them. By the time it was 2011 or so and data science was just starting to become a thing. I said, you know, that's like what I've been doing, but kind of better, and now there's a name for it. So I want to become a data scientist. And I sort of pushed myself in that direction, managed to talk my way into a Data science job and spent three years basically running data science at a startup called Volumetrics. While I was at Volumetrics, I ended up writing a fair amount of production code, for better or for worse, because that was a startup doing analytics on enterprise collaboration data. And so the data science kind of was the product. And so that meant that the data scientists ended up writing a lot of production code. And I found that I really liked it. I liked building things. That was a lot of fun for me. And so I wanted to look for opportunities to really grow out that skill set. And I had the opportunity to interview at Google. So I sort of crammed for a while teaching myself all the computer science that I should have learned but never did because I didn't study computer science. Somehow passed the interview, went to Google for a couple years and really kind of leveled up. And in terms of software engineering, you know, writing code, building things. But after a couple of years there, I sort of felt like I wasn't excited about the projects I was working on. You know, building tools to help ad salespeople sell more ads, or writing back end services, and C to replace spreadsheets, things like that. And so, you know, I had the opportunity to interview AI too, which is a research nonprofit doing, you know, fundamental AI research. And I decided to move over here. I've worked on a couple different projects here. The first one was called Aristo, which is basically teaching computers to pass science exams. And now I work on a team called Allen nlp, which does fundamental NLP research as well as we build a library called Allen nlp, which is a deep learning library for doing NLP research. And so most of my job is working on building that library, supporting people, using that library, adding features, adding models, fixing bugs, giving tutorials, giving talks, so on and so forth.

Yeah, it's interesting that you mentioned kind of having to level up your computer science skills. I remember kind of like getting into data science from a physics background. And I feel kind of similar to you in that, like it was around, around the same time a little bit later, but data science was still kind of, the hype was growing and I felt like similar that I kind of talked myself into a position and then learned a bunch of these coding things later and like learned how to do a bunch of things like on, on the fly. I'm not sure now, like if I was in the same place. Things are so competitive now in like data science and like there's really hard interviews and all of these things. I'm not really sure if I would have like Made it in. Given. Given where I was back, back in the day. So it's interesting that you had a similar experience.

Yeah. So basically my interview to be basically the data scientist at Volumetrics was me meeting with the CEO, him handing me a printed out SQL query and asking, do you understand this? And I said yes. And then he hired me. He hired me basically in a contract to hire position because it was so early in the company that they weren't set up to have full time employees yet. So in that sense he wasn't taking a ton of risk because he could have easily not converted me to full time employee. But still it was not a very rigorous process.

You don't happen to remember that SQL Query, do you?

It was about five pages long.

Oh boy. Okay. Serious SQL right there. So I guess to turn toward the book. I guess I'd like to start off by just asking you, why do you think it's important to think about data science from scratch, since you used from scratch in the title of the book and why is that approach useful to get people into data science and AI? What's your take on it?

So the practical answer is that it's very easy to install scikit, learn point, random forest classification at some data set and spit out a result and have no idea what's going on. Which you know is good in some ways. And it's also bad in some ways that if it goes wrong, it's very hard for you to figure out why it's gone wrong, or it's possible you've made some conceptual mistake or you're cheating by looking at the test data or things like that. And so if you don't know those things, it's very easy to lead yourself astray. So that's kind of the practical answer. The moral answer is that as I mentioned, my background is mathematics. And one thing that's a very strong almost moral principle in mathematics is that you're not allowed to use theorems if you haven't proved them yet. So math classes are set up so that in the first quarter we'll prove these theorems and then the second quarter we'll build on them and third quarter will build on them. So much so that when I was in grad school I had a year long class, but the professor changed between the first semester and the second semester and the second semester he came in on the first day and he says, I'm so glad you proved Theorem X because I need to use it. And with the premise being that if we hadn't gone through that proof he couldn't use it, and then we'd be in a bad spot. And so somehow that burned really deeply into me, this idea that you shouldn't use things like that if you don't really understand how they work. And so when I want to sit down and say, what is linear regression? What is a random forest? What is a neural network? I feel like I need to really understand under the covers what's going on here, what is it doing behind the scenes? Otherwise, I don't feel comfortable using those things. And so that's kind of the, maybe moral, you'd call it, reason for doing it that way. I feel like it's a good approach.

Yeah. So, I mean, I definitely get what you're saying. I remember someone telling me when I was in physics, like, one of the, one of the ways that you can tell, like, the difference between, like, a math physics person and, like, a physics person. So there's this, like, group of physics people called math physics people, is that, like, physics people are okay with just, like, trying something, and if it works, they're, they're happy. Whereas, like, math physics people, they would never, like, use something unless they. They had proved it first. So, like, I definitely, I definitely get that. But also I, I love how in this book in particular, I've looked at some kind of statistical learning, machine learning, AI sorts of books that kind of start from the beginning and explain everything. And it is kind of all very. It's very mathy. Right. Which is good. And it's good to learn those things as well. But then it's, like, somehow disconnected from the coding side of things. And I think one of the reasons why I latched on to a lot of your content was the code is right there. And you can go ahead and see. Oh, yeah, there's this abstract thing, but this is how it's directly connected to Python code. And I can see that it's not that hard to implement this thing in a small number of lines of code, and it kind of makes sense. Was that kind of something you saw as a gap and part of the reason why you wrote this book or why was it important for you, you to kind of take this code first approach?

So here's what I'll say. I think that when you put a lot of math equations in a technical book like that, people will zone out and not follow it. And for good reasons, you don't want to be reading a ton of math equations in a tech book. But the other thing is that a lot of times those equations are Actually not the best way to explain something if you're using it in kind of an applied context. Right. So, you know, basically what I tried to do was, okay, here's a mathematical concept I want to teach, but instead of teaching it by giving you, you know, some latex up equations, I want to teach it by writing code that embodies it. And so that way hopefully you understand the math that's going on, but also you can see from a Python point of view exactly what's happening. And so it's sort of, it's sort of this middle ground where I want the math to be part of the text and part of the pedagogy, but I also want the math to be framed in terms of Python code, not in terms of equations.

So I guess in the book, when you're building from scratch, what do you start with? And what is that order upon which you're building layer after layer so that the reader is kind of grokking it all in their head? How do you approach that?

You know, it took me a long time to figure out a good order for doing this, but in essence the structure is, you know, a brief introduction to Python. And then now we've got Python. Let's talk about linear algebra, which is mostly just working with vectors. And like, I went serious from scratch, where we're going to represent vectors as lists of floats and we're going to do arithmetic on list of floats. So starting from pure Python and just what is a list? Now we've got linear algebra. You know, I can take vectors, I can multiply them, I can do their dot product, and now that I'm equipped to do those things, I can say, let's start talking about doing statistics. So, you know, take the mean of these vectors and I can start talking about probability. What does it mean to have a distribution over things? And I can start talking about hypothesis testing and drawing inferences and just building step by step. And now once, once I've spoken about all these things, there's a slight detour. And let's talk about how do I get data, how do I work with data, how do I clean data, how do I process data? And because of the machinery built, we can talk about things like, okay, now I'm going to do principal component analysis. What does that mean? And now once I've done sort of linear algebra and statistics and, you know, probability fundamentals, and once I've done working with data fundamentals, now I can start to push into machine learning. What is machine learning? Like, what do I need to understand about machine learning independent of what model I'm using, and then just start building up, okay, what is simple linear regression? What is multiple regression? What is logistic regression? What is a decision tree? What are neural networks? And then once you build up all those things, then you start talking about even further applications. What does it mean to do natural language processing? What does it mean to do recommender systems, things like that? That's pretty much the progression of the book.

So why the second edition? I guess were there things that you felt, you know, we're left out? I know that there's. Of course, now we're kind of in. I don't know if we're past the data science hype, but we're definitely into the AI hype. Right. And I think, if I'm not mistaken, you have a little bit more content about neural networks and that sort of thing. Was that part of the reason for the second edition?

There was one overriding reason for the second edition, and that's that the first edition was written in Python 2.7 and basically told people to use Python 2.7. And every day I woke up and felt guilty as hell that there was a book out there with my name on it telling people to use Python 2.7. And so I said, I'm going to do a second edition, that's Python 3. And then once I was like, so that was like 99% of the reason I did it. And then once I was doing that, I thought, okay, well, now maybe I'll add a chapter on deep learning, because that's pretty important thing to know and let me flesh out some of the other chapters and add some new stuff and I'll take the time to clean up the code and, and so on and so forth. But really, the Python 2, Python 3 thing was 99% of my motivation.

I like, I like your answer, by the way. I like the mea culpa.

It's.

That's funny, I don't even want to.

Call it a mea culpa because I think at the time the first edition came out, probably Python 2.7 was the right choice. I, what I should have done is I should have written the code in the first edition in a way that was compatible with either Python 2 or 3 or Python 3, which wouldn't have been a ton of work. But I didn't do it. And so that was, that was a bad call on my part. But the reality is, is that even if I had written, you know, a Python 3 version, it would have been like Python 34 maybe like @ the time I wrote that book or 3:3. And that's not like a great long term version to be on anyway.

So. The Data Engineering podcast is a weekly deep dive on modern data management with the engineers and entrepreneurs who are shaping the industry. Go behind the scenes on the tools, techniques and difficulties of data engineering so you can learn and keep up with the knowledge to make you and your business.

Successful. Can you give a bit of.

An outline about the motivation for choosing.

Jupyter Nopix in particular as the core.

Interface for your data.

Teams? Yeah, and actually when I first.

Joined Netflix, it was sort of tossed at me and I was definitely like, why are.

We? And the answer was like, we.

Might be a little.

Crazy. Go to dataengineeringpodcast.com to listen, subscribe and share it with your friends and.

Colleagues. So Joel, we've talked a little bit about data science from scratch and what you put into the book and like the code first approach. I've also noticed like, with, with some of the other content that you've put out, like both in the, in the book but also in like, you know, talks that have gotten like a good bit of attention. Like I don't like notebooks and I saw another one writing code for NLP research. It seems like you put a lot of focus on sort of bringing good software engineering practices into like data science and AI. Is there a particular reason why that's kind of become a, a passion of yours and like a driving.

Force? Yes, one, yes, that's. That is kind of like my, my big pet cost. So the selfish reason is that I've become sort of, over my best judgment, sort of a software design geek. And what I mean is I spend a lot of time thinking about what is the difference between good code and bad code, even if both work. And in particular, you know, as someone who uses code to teach, I spend a lot of time thinking about what is the clearest way I can express the idea I'm trying to express in code. And what I found is that going through that exercise over and over again has really drilled into the importance of doing that. And it's spilled over into, you know, my actual work. So even when I'm not teaching data science, but I'm writing code for my job, I'm still always thinking what is the clearest way I can express what I'm trying to do? Because, you know, in a month I'm going to come back and look at it or someone else is going to come back and look at it and, you know, if it's not clear, they want people to understand it. So that's kind of the selfish reason. The practical reason is there's a lot of bad data science code out there. And I think some data scientists even sort of take pride in. You know, I'm a data scientist, I write data science code. And I think that's unfortunate because for a couple reasons. One, I think that by not understanding software engineering best practices or how to write good code, you're basically putting yourself in a position where you cannot contribute to, where you should not be contributing to production code, say at your job. And you get this artificial division where I'm the data scientist, I'll write a model and then I'll hand it off to someone who's not a data scientist to put it in production and make it work in code. And I think that's unfortunate for a couple of reasons. One, I think data scientists can have a lot more impact if they can cross that divide. And I think they also become better data scientists if they can cross that divide. Because the idea is from good software engineering kind of flow back, you know, a second reason why, or maybe third, another reason why it's really important, I think, is that a lot of the things that are software engineering best practices are intended to help you, one, write correct code and two, get better at writing code. And those are both things that are equally important for data scientists. If you're a data scientist, even if you're writing code, you know, to run some experiment or something, you want that code to be correct and you should be writing unit tests for that code, because unit tests, I'll make sure your code is correct and you should be getting code reviews on that code because code reviews are a second set of eyes and they'll find bugs and they'll point out things that maybe they're not wrong, but you could have done better or hated. You know, that scikit Learn has this thing that would have done that for you. And so all of these best practices will make you a better data.

Scientist. You know, it's funny, I'm sorry, I keep.

Going. No, no, that's.

It. I was just going to say I actually sympathize tremendously with that take on it because I came into this from a software development, software engineering background. And as I moved fully into the AI space and started working with data scientists in a lot of cases straight out of school and into the job, and they were weak on the programming side, that was definitely created a tension that I noticed. You Know where you had people that had different skill sets kind of working and bringing different things to bear. That certainly was the case for us. I know that, like, as we were thinking about the product of what we were doing, having to be productized and getting into, you know, the data science pipelines that had been created at the company, I was at the time with that tension. Have you experienced that where, you know, people are like, well, I'm a data scientist, I just don't care. It sounds sort of like you may have. What kind of. How did you get through those dynamics in terms of dealing with those different perspectives.

Together? I think you really have to do a sales job. Right. And so that's one thing I've been trying to do with some of my talks and also in the second edition of my book to some degree, is to say, look, writing unit tests sounds like a hassle. Doing code review sounds like a hassle. But I'm going to show you how your life will be better if you adopt these practices. And I think it's really hard to convince people to do it without having that sales pitch. This is going to make your life better. And that's not like me talking bs. I do think as a data scientist, it will make your life better to do those.

Things. Yeah, I definitely, like, I remember when I was in my first data science position and I realized, like, you know, people started talking to me about like, you know, not cutting over to master before you, like, have a review and a past test and all these things. And I was like, I was very confused by a lot of that because I did not come from a software engineering background. And so I was like, terrified when I first went into code reviews. But over time I realized that, yeah, I could skip those things and then the whole system crashes down and it's my fault. And then it's like a big situation where I caused this big thing, or I could go through a code review and have some assurance about how this thing, have some knowledge sharing, but also have some assurance that my logic was sound on this or that. And now I feel like I don't feel good pushing anything into a place where it's going to be used without having someone review it. So I think it is like a shift of perspectives. I'm glad that there's kind of this effort going on to educate people on these.

Things. That's one angle. Another angle is that by doing code reviews you learn a lot, both in terms of having a second set of eyes on your code that can tell you, here's another way you could have done this or by reviewing someone else's code and saying, wow, I never would have thought to do it that way. But that's a cool way to do it. Doing code reviews is a great way to keep learning and building skills and getting better at what you do. Hopefully if someone's a data scientist, that's something that they're interested.

In. Yeah, I think hopefully a lot of people in this field have that passion for learning. And it's definitely true that as you have a more diverse set of, of people and ideas working together, you can definitely learn a lot of things that you wouldn't have learned otherwise. So in particular with your Writing Code for NLP Research talk, I was kind of just looking through that as we were preparing for this interview. And you talk about how to write code in a way that facilitates good science and reproducible experiments. Given that this is like practical AI, that is a lot of our focus is giving people like practical discussions on this topic. I was wondering if you could just give us a few, like maybe the, the, some of the most important points from, from that, from that talk or maybe other talks in terms of what are some like, immediate wins that you can have in terms of writing code that is reproducible or writing code that facilitates good.

Science. Yeah. So that talk is actually a tutorial from EM NLP last year that it wasn't just me, I presented it and wrote it with two of my colleagues here at AI2. But since then I've sort of gone on the circuit of academic AI conferences sort of beating this. Software Engineering Practices for AI Researchers drum with reproducibility in Mind So the first thing is just what I said before. Write unit tests. And that sounds weird to say write unit tests for your research code, but if your model is not doing what you think it's doing, I mean, that's bad science out of the gate, right? Like, if your model is like accidentally ignoring one of its inputs, then it's not going to be doing the right thing. So if you write tests, and the way we tend to do it is take a data set like the one you want to use, get a very small version, maybe it has two or three instances in it. And now write a unit test that takes that data, reads it in, puts it through your model, checks that checks that output looks right, that the model can run on it without crashing, checks that the model can learn to predict it perfectly. Because if you have three instances, your model should be able to learn it perfectly. Things like that and you make these actual unit tests that run along with your code. Then that gives you this really simple but basic guarantee that code is doing at least sort of what you think it's doing. So that's one, I would say, really important aspect. Another one is that typically you're going to run a lot of experiments, maybe with minor variations. I'll vary some hyperparameters, or I'll try different kinds of word embeddings, or I'll try different number of hidden units. And so what you want to do is you want to structure your code that makes it as easy as possible to run these experiments. And so what that speaks to then is making a real clean division between what I'll call library code and experiment code. So I want one code that's like, this is the details of how my model works and this is the details of how data gets read off disk. And I want this code in a module that's like well tested and it's not going to change. And now I want to run an experiment and maybe that experiment needs to point into specific data path and it needs to use some other parameters and various other parameters. And I want the experiment code to basically lean very heavily on that module. And just all it's really doing is taking the parameters that I fed in and calling that model. And so when you separate things out this way, it makes it really easy to, you know, run an experiment. You don't have to be saying, oh gosh, when I did this one, what version of the code was I using? When I did this one, which version of the code was I using? In all cases you're using the same code, you're just feeding in different parameters to it. And so this was a little bit, you know, this was one of my gripes against notebooks is that notebooks don't really lend themselves to this kind of division very easily either. You have to pull all your library code out into a module, in which case you're only half working in the notebook and half not working in the notebook, or you have to do something really elaborate like the system that Netflix uses where you parameterize your notebooks and have a 50 piece system that runs them and instantiates them and saves them and so on and so forth. So I think making that clear division is another one, it's.

Python. So depending, not letting the notebook be a crutch in essence to where it's inhibiting you in terms of good.

Practice. Yeah, I mean, I could talk about that in various other contexts at great length, but yes, that is certainly one aspect of it. The other one that's sort of obvious, but a couple more one is write good instructions for how to use your code obviously because someone's going to want to rerun it and reproduce it and if it's not obvious how to do that, then that's no good. And finally the other one that's important is Python. Dependency management, like in most other languages, is a real pain in the ass. And whatever experiment you're running is going to depend in very intricate ways on oh, you know, I use this version of Pytorch and I use this version of Spacy and being as clear as possible about here's how to set up the environment and you know, that can be your requirements. Txt, that could be a Docker container, it could be anything like that. But if you go out and look on GitHub at like data science projects, you can find a ton of them where there's literally no indication of how did I set up the environment to do this. It just starts with import Pytorch and it goes from there. And your chances of being able to reproduce that are very.

Small. Yeah, I definitely have felt that pain. I was curious on another point too. Like you've mentioned tests a couple times when I'm teaching workshops or something. A question that comes up a lot of times when we start talking about testing and this sort of thing is oh well, people will tell me, oh well, machine learning and AI is kind of different, you see, because there's these non deterministic pieces and there's randomness, so you're not always going to get the same answer. So how do you test that? Or they give the example. In software engineering you could have a bunch of table driven tests and you expect all of those things to pass, but when you're testing on a test set in, in machine learning, you don't expect all of them to pass and that sort of thing. And I kind of take issue with that because if there was no way to test randomness and non deterministic sort of processes, then physics wouldn't exist basically. But I was curious to hear your perspective on that. There's definitely cases where you have some randomness and other things in your code, but it seems to be a point that people get hung up on when they start talking about.

Testing. So there's a, there's a couple of answers to that. One is that there are a number of things you can test that should be there are invariants of your code that don't depend on the randomness. So, for instance, given an input, it should run without crashing, like, that's not a function of randomness. Given a model and an input that has the right shape, the output should have the right shape and it should have the right fields or whatever. Again, that's not a function of randomness. That should be an invariant of your code. And so those things are easy and they don't depend on randomness. As I mentioned earlier, there are other things that depend on randomness, but you can test them in ways that are sort of randomness invariant. So, you know, I trained my model for on a small data set for 10 epochs, and I check that the loss goes down each epoch. So is it possible that there's some weird random setup that would make that not be true? Possibly. But that test is basically always going to pass. And if it's going to fail one time out of a million, you mark it with a Lakey decorator and that's fine. Similarly, it's true that I have a big neural network. It has random weights. Who knows what it's going to do? But again, if I take a data set that has three examples in it and I train that Data set for 100 epochs with this big model, I can be sure that model is going to learn to predict perfectly on those three data points. It just has to. Again, you can come up with these tests in a way that give you some confidence that the model is doing what it's supposed to do and are robust enough that they don't care, you know, what the random initialization of the weights happened. They should work regardless of.

That. Well, hello there, listeners of practically, how are you? This is Adam Stakoviak. If you haven't heard yet, we're launching a new show called Brain Science. It's a podcast for the curious. Are you curious? Because if so, we're exploring the inner workings of the human brain to understand things like behavior change, habit formation, mental health, and what it means to be human. It's brain science applied not just how does the brain work, but how do we apply what we know about the brain that can transform our lives? Learn more about the show and subscribe@changelog.com Brain Science until then, here's a preview of episode one, where we talk about the fundamentals of being.

Human. We're also all designed to be in relationship. We are fundamentally hardwired to have social groups and this sense of attachment. And because I'm sort of a geek when it comes to research, what researchers have found is that attachment, which that's what we label how we relate and connect with others. Attachment is 100% learned which means our genetics don't actually contribute to how we learn to stay in proximity with other people and with that, that we all develop ways to manage the threat of the loss of a relationship. But nobody gets to opt out of going I need to be in relationship with.

Others. It's almost like we need to have that echo from another human being to let us know that we, yeah, we're there or we're alive or just some sort of feedback loop. I'm not really sure how to describe.

That. Well, it really is this sense of being with. Right. Like I can't fight battles on my friends behalf or on my kids.

Behalfs.

Right. But the simple fact that I know of what's going on makes a difference because I would contend it sort of like I help them hold that weight emotionally. And so that actually leads me into the third thing. And the third thing that I would say in regards to the fundamentals of being human is that we all struggle. Oh yes.

Right. Big.

Time. And that, you know, we don't always get to pick the way in which we struggle, but we all.

Struggle. Well, if you like what you hear, you should go to changelog.com brainscience the show is not out yet so, so don't get too excited. But you can subscribe and be notified as soon as this show launches. Once again, changelog.com.

BrainScience. So turning toward Allen AI, could you tell us a little bit about the purpose of the organization, what its mission is and the types of problems it's oriented.

On? Yeah, so Allen Institute is a non profit research institute. It was founded by Paul Allen and it's funded by him and I guess now his.

State. Yeah, you may want to note who Paul Allen is just in case there's anyone out there who's not familiar with.

Him. Paul Allen was one of the founders of Microsoft, so he got pretty wealthy that way. And then he, he left. He started a company called Vulcan which does like a million different things. And you know, he was a big Seattle person. He's a big everywhere person. And so our mission, our mission at least if you look at like the sign on the website, it says AI for the common good. What that means is a little bit vague. I once interviewed someone who was very disappointed to find out that it didn't mean that we used like we were working on like the homeless problem or anything like that. But what we do is fundamental research in various areas of AI. So as I mentioned, the team that I work on is called Allen nlp and we work on sort of fundamental research and nlp. So not super different from what people who are NLP researchers in a computer science department would be working on. In fact, we collaborate with them, we go to the same conferences they do, we present with them things like that. There's another team called Prior which works on problems in computer visioning and visual reasoning. We have a project called Semantic Scholar which is an AI enabled search engine for academic papers where it indexes the academic papers, but it also tries to figure out what papers were influenced by this one and what information can I extract from the paper programmatically and things like that. And we have a project called Aristo which I mentioned earlier, which does science question answering. So can we get an AI to answer science questions? And we have a project called Mosaic which does, tries to get an AI to do common sense reasoning so you can ask questions about science. Question is like, you know, when a plant makes, you know, food, what is that called? A photosynthesis. B. Whatever. But the common sense reasoning is more around understanding, you know, folk physics and folk psychology. You know, Billy did xyz. Why is it likely that he would have done this? And so those are kind of the different areas that you focus.

On. Yeah. So LNAI is there like there's obviously a lot of sort of academic like research going on. But there's also like you've mentioned projects like Allen NLP and there's like, I guess products like you know, Semantics Scholar that are people are using and out there. So there's, there's obviously like some mix of like researchers and engineers. I was just kind of curious like how the teams are structured there like and how that might be different from like maybe regular academic research where things, maybe it's that things are more geared towards open source or like I don't know, I was curious what that sort of looked.

Like. So I would say one, it varies a lot by team. So like my team, Alan NLP is actually very researcher heavy. It's mostly researchers and only a handful of engineers. Whereas Semantics Scholar, they're running a website that has to be up so they have a lot more engineers, they have to be on call and they have a lot more front end people and things like that that we don't really have. So it really varies by team. In terms of comparison with the academic department. I say the engineering support that the researchers get here is I would say mostly not comparable to what people get in an Academic research department. I mean, the engineers here are, you know, I'm ex Google, other people are ex Google X, Amazon, ex Microsoft, and so on. And my understanding, I mean, I've never been in a computer science department, but my understanding is that if you're working, you know, as a professor in a computer science department, typically you don't have these kind of engineers who are there to collaborate with you and support you. So I think that's a real deep difference between the way we do things and the way that an academic department does things. Now if you go to somewhere like, you know, Google Research or Facebook, AI Research they do, or Microsoft Research they do, those places do have this kind of engineering.

Support. So I noticed, I'm looking at the, at the webpage for Allen nlp and you were talking about that being your own project and it describes that it's an open source NLP research library built on Pytorch. I'm wondering who is that geared for? What is the audience for using it? What are the main use cases that you might apply it to? Any thoughts.

There? Yeah, so I mean, officially the target customer is someone doing NLP research. So originally, originally our target customers were the researchers here at AI2, but it was always an open source project from day one. And so then we quickly started taking on as customers researchers over researchers at other universities, people at more corporate research institutes. And then, you know, the line between research and work is really narrowing a lot, in a lot of ways. So there's a lot of companies that you can go to them startups or big companies or whatever, and they're doing things that are, say, at the cutting edge of NLP for their own problems. And so their workflows for solving these problems look a lot like the workflow of an academic researcher trying to solve an NLP problem. And so we do have some customers who, you know, work in companies but still use it anyway. And it's always, it's sort of an ongoing source of debate to what extent we should be actively soliciting supporting more corporate customers versus academic customers. I'm giving a tutorial on Allen NLP at NLP Market more in general at the O'Reilly AI conference in September, which is more of a practitioner focused conference, not an academic conference. And when this was mentioned, someone asked me, are we focusing on those practitioners now as customers for the library? And I'm like, well, you know, we've always sort of wink, winked that those are our customers, but they're not officially who we're focused on. But we Want them using it.

Too. Yeah. And what are. So I'm just trying to think of like the workflow wise, like if I'm an NLP researcher, like where Alan NLP might, might fit in. So I'm thinking of like a comparable. People might be familiar with something like Spacey or something like that where Spacy has like, you know, pre trained things that are available. Right. Like I can do some types of NER and other things and it's all pre trained, but I can also train custom models. But I'm not really like messing necessarily with a lot of the like architecture that's the architectures of the neural nets or something like that. Is, is Allen NLP kind of like a level down from that to where like you have more of the ability to mess with your models and all of that sort of stuff or I'm just trying to kind of like gauge where it.

Sits. Yeah, that's exactly right. So Allen NLP is not, for the most part, it would not be a substitute for spacy. Instead it would be a substitute for someone who's doing NLP stuff in pytorch or in TensorFlow and says, you know what, there are a lot of problems that are caused by my working with text. And so I would like to use a library that has in it abstractions that are really intended at people tackling NLP problems and allow me to not worry about some of the nitpicky details around masking and padding and recurrent layers and things like that. But working in LLP is much more similar to, I mean when you work at lmp, you basically are working in Pytorch, you're just working in a Pytorch that has a bunch of higher level abstractions in.

It. Gotcha. So you're. So it's built on Pytorch essentially. Right. So if you were coming at it, I'm Just curious from TensorFlow, do you need to kind of go learn Pytorch first and then move into it as an intermediate thing or you know, what would be with all the people out there that are also doing TensorFlow in addition to Pytorch? What. What's kind of the migration path that you would see into.

That? That's an interesting question. You would want to learn some Pytorch because, Allen NLP is not really insulating you from understanding how PYT works. The way I would describe it is, and this is going to be a terrible analogy and I'm going to regret saying it as soon as I say It. But imagine you have like a big bag of Legos, right? And just like only the squares and the rectangles.

Right?

Okay. And then you're building houses or whatever, cars or whatever, and you build everything using the little square blocks. And then someone comes in and says, hey, you know what? I notice you're making like a lot of walls. So here's a piece that's just a wall and now you can use this one piece to make a wall instead of having to build each wall out of like little blocks. Right? And so that's kind of my analogy for what we're doing. And so that allows you to, you know, now I don't have to worry about walls or now I don't have to worry about doors, but you still have to know how Legos work and you still have to know how to build with Legos and you still, you know what I mean? Does that make.

Sense? I do, and it's a good analogy. I have a 7 year old, so we have Legos all over the place. That was perfect for.

Me. So I'm kind of curious like being, you know, I've been involved in certain open source projects over, over time, but always geared towards kind of like the, I don't know what you want to call like the regular software engineering community. What have you found to be kind of like unique challenges of creating an open source project for the AI community or more for the research side side of things. Have there been unique challenges with.

That? So this is like the biggest open source project I've ever really worked on. In fact, it's probably the only like multi person open source project I've ever worked on. So it's hard for me to say whether the challenges we face are unique. I can tell you some of the challenges that we face are people who submit issues without enough information about what they're doing or how to reproduce the problem they're running into. But I imagine most open source projects probably face that issue. The one that we struggle with a lot I think, is that we have extremely high coding standards. So we have just a ton of continuous integration checks before you can merge your code. So we use Python 3 type annotations and we actually run MYPY in the code base. And so if you submit a PR that doesn't type check per mypy, the CI system will spit it out. And we use Pylent with Pylent is sometimes the band of my existence. But again, if you don't format your code right, Pylent's going to choke on it and we have some code to make sure that you've actually written Sphinx documentation for every class you've added. So basically, in order to maintain this very high bar for code, what that means is that contributing to library is sort of a pain in the ass because most people who want to contribute, they might not know about mypy, they might not know about Pylent, especially if they're researchers. And so getting them over that hurdle can be pretty difficult sometimes. So that's one aspect. The other aspect that I think we always struggle with is when you own an open source project and you know it's an open source project, but it's also our project, you have to support people and you have to maintain it. So every time someone wants to contribute something to the library, someone who's not on our team, say, like, on one hand, that's awesome, I love that you want to contribute to my library. The other hand, like, you're putting this piece of code in the library and now if it breaks, I'm the one who's got to fix it in a month. And maybe I didn't even understand it when you wrote it. And when people have questions with it, they're not going to go to you, they're going to go to me. And so it's this constant tension between, yes, I love that you want to contribute to the library, but at the same time, do I want your contribution to be something that I'm going to sign up to support and maintain forever, basically. And that's another ongoing struggle that I don't think we found the right answer to.

Exactly. It's funny, that last answer. I had a question for you, but I'm actually going to ask you two because since you're working on a team that is, you know, it's more than just yourself and you have to juggle multiple people and their perspectives and all. What I was going to ask you, which I'd like to address, but I want to tag on to, is what is the team working on right now? You know, what are some of the cool things? But also as an individual on the team, what are you excited about in terms of, you know, the future for AI and natural language and the types of things you guys are working on. So, you know, kind of team perspective and also your own. I'm.

Curious. So our team, like I said, is mostly researchers and they're doing research around things like paragraph understanding, semantic parsing, question answering, language modeling, sort of wide variety of NLP topical research kind of things. We have a handful of Like I said, we have a lot more researchers than we do engineers. What I'm working on right now is mostly, I don't know if you got a chance to check out the Allen NLP demo, but we have this, you know, demo.llp.org, we have these interactive demos for a lot of our models. And that's one of the value propositions of the library, is that once you've trained a model, we make it sort of easy to get an interactive demo. And so one of the things I'm working on right now is basically once you've trained a model, basically for free, you can get a really ugly demo, which is like text in and JSON out. How can I make it so that you can get, for almost free, like a beautiful interactive demo that's like text in and widgets out and make it so that someone who doesn't know, you know, a ton of JavaScript or React can get it working very easily. So that's. That's kind of one of my main areas of focus right now. We have other people who are working on things like how can I train more efficiently, you know, how can I speed up these models, how can I use less compute, as well as just adding various other.

Stuff. Well, I for one, would love to see this, this demo functionality that you're. You're talking about. I would be super excited to use that. It would, like, make me seem way cooler than I actually am. So you're cool, buddy. I would always appreciate things like that. So to kind of wrap up, first off, thank you so much for joining us. As I already mentioned, like, I've been a fan for quite some time, so it's been a real honor to get to chat with you a little bit and get to know you a little bit. Just as we wrap up, where can people find your book, first of all? And then maybe also, what's the best place where people could start learning about Allen NLP? I know there's. It's probably on GitHub, but are there other places? Do you have a website? Are there slack groups or anything like that? What's the best way to get.

Engaged? Yeah, so you can get my book. Any of our books are sold. I went in the Barnes and Noble a couple weeks ago and they had a copy there. So I took it out from where it was hiding and put it right in the front of the rack very prominently. Of course, you can get it from Amazon. O'Reilly shut down their bookstore, so you can't buy it directly from them. But if you want to buy a PDF version, apparently those are sold through something called ebooks.com which is where you can buy PDFs of O'Reilly, but really any. Any online bookstore, you should be able to get it. If you want to find out about Allen nlp, the best places to start, just at our website, Allen NLP.org that will take you to demos and tutorials and so on. If you want to look at the actual code itself, that's on GitHub, AllenAI slash Allen NLP. You can go there and see our enormous backlog of issues and pull request and laugh at us for having such a big one. Can I pitch all my other.

Presences? Yes, please.

Do. So I have a website. It's just my name, Joel Grus.com I blog very infrequently, but when I do it's good. I am on Twitter all the time. That's oelgruce and you can find me there. And it so happens that I have my own podcast with Andrew Musselman called Adversarial Learning, which we rarely record, but sometimes we record it and that's@adversariallearning.com.

Awesome. Yeah. And I must say it's both on the blogs and the podcast. When they do come out, I highly recommend them. So thanks for. Thanks for sharing.

Those. Oh, and one more. I like to make live coding videos and those are all up on YouTube and I think that's just YouTube.com Joel Gru so you want to watch me live coding frequently. Problems that I haven't looked at before I started life coding them. Go there and check those.

Out.

Cool. Awesome. Definitely will. Thanks again for joining us. Hope we can meet up at a conference sometime. But look forward to to more great content online and thank you so much for the.

Discussion. Yeah, thanks for having.

Me. All right, thank you for tuning into this episode of Practical AI. If you enjoyed this show, do us a favor. Go on itunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you got to do. Share the show with a friend if you enjoyed it. And bandwidth for Changelog is provided by Fastly Live. Learn more@faster.com and we catch our errors before our users do here at Changelog because of rollbar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers. Head to linode.com changelog check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@changelog.com when you go there. Pop in your email address us get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next.