#31 Chatbots, Conversational Software & Data Science (with Alan Nichol)

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Alan Nicoll about chatbots, conversational software and data science. Alan is co founder and CTO of Ruza, who build open source machine learning tools for developers and product teams to expand bots beyond answering simple questions. Which verticals are conversational software currently having the biggest impact on? What are the biggest challenges facing the fields of chatbots and conversational software? What misapprehensions do we as a society have about these technologies that experts such as Alan would like to correct? And how can we all build chatbots and conversational software ourselves? That's enough questions. It's now time for some answers. I'm Hugo Bown Andersen, a Data Scientist at DataCamp and this is DataFrame. Welcome to DataFrame, a weekly DataCamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter Hugo Bownt and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com community podcast. Hi there Alan, and welcome to Data Framed.

Hey, it's great to be here.

It's great to have you on the show. I want to jump in and discuss your OG Medium post from April 2016, which really got the ball rolling for everything you're working in now. You open this post with the following statement. We don't know how to build conversational software yet. I feel like you meant we as a society and community of tool builders. Conversational software includes chatbots. I'm wondering now, two odd years later, is it still the case that we don't know how to build these types of things yet?

I would say broadly, yes, I still agree with that statement, but we've definitely made some progress. But it's still very much early days for building great natural language interfaces for computers.

Tell me a bit about the progress you've seen in the past two years.

Yeah, I mean, maybe a bit of context as well where that blog post came from. So we built a few Slack bots because Slack was one of the first platforms that opened for this and Alex, my co founder and I had actually we had a couple of Slack bots that companies were paying for and they were both around actually making data science more accessible, so turning natural language queries into SQL and then running those queries on a database. Anyway, so we had some experience with building these things and just looked around and thought, wow, there really aren't any great developer tools for it how to build conversational software. And then we saw that the Facebook messenger platform was about to open up and that was right around this time. And we thought this is just. This is not going to go well.

And what you mean by Facebook messenger platform opening up is opening up in order to have conversational software in it.

Exactly. So letting people, letting developers build chatbots that anyone with a Facebook account could then chat to. And we thought, well, we have now quite a bit of experience. We know that it's really hard to do this well. And this is just going to. These are not going to be great experiences for people. And I think that is actually pretty much how it panned out. There's a lot of hard work involved in getting things working well even in a narrow domain. And doing something which is kind of open ended conversation is definitely nowhere in sight. But I think we've made some good progress in terms of libraries and tools that people can use that make it easier to build something that does work, even though we've definitely not cracked this problem by any means.

And what are the most pressing things that we haven't cracked with respect to building conversational software?

So there are a number of things. One is the kind of two things you need for building conversational software. So the first is understanding the messages that people are sending to you. Right. And that's called nlu, or natural language understanding. And what that usually means is like classifying a short piece of text as belonging to one of n intents. So those are kind of class labels for things like a greeting or saying goodbye or asking for some specific things that your chatbot can do.

Like looking for a hotel or something like that.

Exactly. Like I'm looking for a hotel. And then the other part is pulling out some structured data. So it's usually called entities. So you know, I'm looking for a hotel in San Francisco and then knowing that San Francisco is what you want to use in your query and that's nlu. And I think it's fine to call it that so long as you remember it's absolutely not true. The computer doesn't understand anything. Right. It's just able to classify text into one of these buckets and then pull out some entities.

Yeah. And just this is a large concern in general with terms like artificial intelligence, the kind of anthropomorphism involved in nomenclature and the way we name stuff. Right. It's dangerous, especially when it permeates common language.

Definitely, definitely. And I think there are lots of complications and limitations to that. Even that simple model. Right. Of the intention of what a short message means being a universal single label. Right. So this always means this. And you can only represent its meaning by a single sentence of one hot vector, let's say. That's obviously a very limited model of understanding what people can express and it's a good starting point and it lets you build on top of that. But of course, that's not kind of when we think about the future, I think that's not how we're going to represent the meaning of short messages.

Yeah. So we have NLU as being a huge challenge. What was the other one you were going to speak to?

Yeah, so the other one is then dialogue management. So if somebody says yes in the middle of a conversation, the way you respond, of course, depends on the context, depends on what happened before. So it's not enough to just map each of these intents or each of the kind of outputs of your NLU system to the same action. Right. You need to always build up some kind of statefulness. And then the question is, how do you handle that in a way that doesn't break and is actually maintainable? And that's a big part of what we've been working on for the last couple of years. And that's actually, I think the part that the biggest part that was really missing back in 2016 was a reasonable way of dealing with that complexity because the way that people were doing it basically manually, writing a lot of rules just, you know, doesn't work and it doesn't scale and it causes a lot of headaches.

Right. And I like this idea of scaling because as you say, writing a bunch of rules is, I suppose, the most bare bones, naive way to think about writing conversational software. I mean, I imagine you can have a set of like nested if else's to try and deal with every everything or a subset of everything. And then as soon as a new use case comes up, you then need to nest them even further. And this is something which definitively does not scale.

Right, Exactly. I mean, maybe if you nest them deep enough, then it counts as deep learning. But that's hilarious in general. Right? You have. So, I mean, I've tried this plenty of times. Right. And even building like a relatively simple bot that just does. It was a banking chatbot, right, that we just built for fun to see, okay, how badly does this work? If we really just do it with rules and, you know, it could just do a couple of things like, you know, check your balance and transfer some cash to people and everything and I think it came out at over 500 rules just for doing this kind of simple stuff. And of course then when you want to update something or something goes wrong and you add a new rule, it clashes with the old ones, right? And then you go, oh no. And then you have to go and try and reason about all these rules and figure out why it is that something broke or something clashed. And there's an asymmetry there, which is really interesting because in the middle of a conversation, right? I mean, the kind of cliche example by now is, you know, what do we want? Chatbots, when do we want them? Sorry, I didn't understand your request.

Okay, that's provided a great teaser into a lot of the through lines that we're going to talk about with respect to NLU scaling a lot of different use cases of this, these types of chatbots and conversational software. But before all of that, I'd like to find out a bit about, about you and Rasa, so maybe you can tell me a bit about yourself, what you do and what Rasa does and what Rasa is.

Yeah, so Rasa is two different things. So Rasa is a company, it's a startup, and it's also a pair of open source libraries. There's Rasa nlu, which does language understanding, so parsing short messages, and then there's Rasa Core, which does dialogue management. And I'm a maintainer of both of those libraries and the aim of them is really to expand chatbots and conversational software beyond just the kind of answering simple questions, FAQ style, one input, one output, kind of turning it into a real conversation and building that in a way that scales. And so where I see us as a company and also where we come from was around 2016 when we said, okay, nobody knows how to do this. Actually there's a lot of research on how to use machine learning to overcome some of these problems. And there's a lot of great papers written on it, but there weren't any libraries that developers could use to actually implement those ideas. And so where we see our role is really in that big gap between archive and GitHub, let's say. So something that's actually well maintained, has lots of tests, has people responding to issues, has support, gets updated regularly. So we do a lot of applied research in this field and we publish papers and we work together with, with universities, but it's always very strictly applied. And then we always, the primary output is always to put out some new code that people can do that does something better. That they couldn't do before. Right. So one recent example was we completely changed how we do intent classification and we shipped a new model which threw all the old assumptions out the window and just said, okay, now we're going to learn word embeddings in a supervised way for this task. And that lets us do things like building up hierarchical representations of meaning, understanding that a message can contain multiple meanings, because sometimes people just say multiple things. And so we do a lot of that. And then the primary output is a piece of code that people can use. And then if we write up a paper, that's a kind of nice bonus.

So this speaks to the open source side of Rasa, but you said that Rasa is, is two things.

Yeah. So the other side is a company, and the first year that we operated, we basically did a bunch of consulting work on top of the open source. And that was really great because building stuff with it yourself, it keeps you really honest about its limitations, helps you understand your customers. And then after we did that for a year, that was obviously very nice, and we kind of bootstrapped the company, then we said, okay, actually we think there's a scalable product that we can build here for an enterprise version. We talked to a lot of these big companies that we've been consulting for, and there was a clear need for an enterprise package with more features and a different product. And so we thought, okay, that's something we want to take a bet on. And so for the last sort of six months now, we kind of stopped all the consulting work. We raised some venture capital and really just went full on, on building out the enterprise version. But then still, all the machine learning stuff goes into the open source. Right. So a big part of what we believe in is that, you know, you can't build up a competitive advantage by having kind of secret implementations of algorithms lurking around. The machine learning stuff needs to be open, it needs to be tweakable, people need to be able to play with it. And there's just so much nonsense around in the AI space that it's better to just say, okay, no fairy dust, you know, there's no magic, it's just stats. Go look at the code. All the machine learning is open source and you can see exactly what it does and you can tweak it for your own purposes. Yeah.

And I'm sure all the different interplays between your company and the open source development are really exciting. So, for example, correct me if I'm wrong, but you've recently hired two machine.

Learning researchers, yeah, so we're only 10 full time people, but two of those are full time on ML research. And the measure that we really care about is how quickly can we take a new idea like, oh, this know little trick actually works and then put it in production. And the great thing about having the open source community is that whenever we have a new idea or we have something that kind of works, there are just thousands of people who are just ready to check out the master branch and see what it does on their data sets. And so that feedback loop is really awesome. And you can't do that. If you build things in a closed source product, you don't get that kind of insight.

That's really awesome. And I suppose we should say that also it's a Python library, right?

It is, yeah, yeah, it's written in Python. But also we obviously work a lot with large companies, right? So our main enterprise customers are the Fortune 500 and they're, let's say, not mostly on Python, right. So they mostly write in. Net or Java or C. And then there's a large sort of chatbot developer community which uses JavaScript. And so what we did was we made sure that the libraries, even though they're written in Python, you can use them without writing any Python and you can consume everything over a HTTP API. And that means that you don't have to be running a Python stack yourself to use these libraries. You can kind of just spin them up in Docker containers and just use them and deploy them to production without having to actually write any Python yourself.

That's really cool. And I may be putting the cart before the horse or the chatbot before the company, to extend the analogy into absurdity, but there is a fantastic data camp course that, that you've created and that I facilitated last year on building conversational software with Python and using Rasa. And there's also a lovely interplay between, for those people who know a bit about the Python data science landscape, lovely interplay between Rasa scikit Learn and Spacy, for example.

Yeah, exactly. And you know there's no point in reinventing the wheel, right? So there's a lot of really great libraries out there for doing different things. For example, in Rasa Core, which is the dialogue manager, you can plug in different backends to implement your model in and actually do the machine learning part. So one way you can think about Rasa Core is it does all the hard work to get that conversation into the kind of XY pair format that you think about when you think about Machine learning and then you can plug in whatever classifier you like. Of course we have some good ones implemented already, but then you can implement your stuff in TensorFlow or Keras. And actually the Keras API was a big inspiration for that. Just saying, okay, can we just abstract over all the things that aren't important for understanding the problem and really just present the API that makes sense to you? And then we don't need to build our own autodiff library, because why should we? You can use TensorFlow for that and that also runs on GPU, etc. And similarly, for doing things like part of speech tagging, there are lots of things that you can do with spacy and it makes more sense to build on top of libraries like that. So you can kind of choose different backends that you want to use and implement some, let's say, lower level functionality for both Rasa NLU and Rasa Core.

That's fantastic. So I want to get in in a second to think about what type of use cases, which verticals and industries you see most interested in conversational software. But before that I just want to step back a bit and what's a good working definition of a chatbot or conversational software? Are they the same thing? That's a relatively ill formed question, but maybe you can speak to that a bit.

Yeah, so I'd say the things that we're interested in are not just, let's say strictly chatbots. Some people would call some of these things virtual assistants. I actually think that voice is also extremely important. So anything where you interact with the computer through natural language is something that we're interested in. And I would call all of that conversational software.

And so that's different to just a chatbot. Right.

I think chatbots mean different things. So in, in some groups of people, a chatbot strictly means actually just something that does chit chat. So, you know, can't actually do anything for you. It's just there for having an interesting conversation that's actually less what we're interested in. Small talk is not something that we really focus on. We're more interested in purposeful conversations that actually do something. But then there's another kind of definition which says that a chatbot is on these mobile messaging apps, so on Facebook, messenger or Slack or WhatsApp or something, or Telegram, and that any kind of application that lives inside one of those apps, even whether it's chat or button based or it shows you a little web view that all of those are also chatbots. So it's yeah, ill defined, as most terms are.

We'll jump right back into our interview with Alan Nicol after a short segment. It's now time for a segment called Freelance Data Science with Susan Sun, a freelancer who has worked with the New York Times, Cooney, Google and General Assembly. Susan, we're here today to talk about the legal side of freelance data science, right?

Yep, we sure are.

There seems to be a riddle. As a freelancer you'll need to build a portfolio, but as a data scientist working for private enterprise, I'm sure you need to be constantly signing non disclosure agreements, otherwise known as NDAs. How does this trade off play out for you in practice?

I prefer to err on the side of caution. Even if the client doesn't specifically ask me to sign an NDA, I rather treat every client and situation as though I've already signed an NDA. NDAs actually mean a lot more than simply promising not to take a code I wrote for one client and using it for another. Ideas, concepts, strategies. They are all considered intellectual property and can and should be restricted by NDAs. On top of that, some clients don't want the public to know that they are contracting work out to freelancers, and that is their right. So if you sign an NDA like that, you can't mention that you're working for certain clients. And finally consider this, if the client you're working for is international, you also have to think about the data privacy laws that differ by country. Most relevant one that's on top of everyone's mind right now, are the GDPR compliance rules.

Absolutely. So what implications does this then have for portfolio building? As a freelance data scientist, it definitely.

Puts a restriction on showcasing some of your best work. It's pretty difficult if you're just starting to freelance and you don't have the history of past clients to help vouch for your work, the only thing you really have is your GitHub repo to prove yourself. I see this problem a lot in my students, whether they just finished a data science bootcamp or they just graduated from university with that master's in data science. My advice to them is usually 1 to participate in some data science hackathons and 2 to contribute to some open source projects. So both of these are perfectly legal ways to showcase your work publicly.

Great. And as a freelancer, do you have to form an actual company such as an llc or can you just operate independently?

Absolutely, yes. There aren't that many rules that would say are absolutes in freelance, but this is definitely one of them. You must protect yourself under llc. Not only does it make you look more professional to your clients, but also from a legal and accounting perspective, you want to separate your business expense from your personal expense. And finally, being incorporated under LLC gives you an extra layer of protection in case of any legal disputes.

And can you do this yourself or do you need to hire some expert help, like a lawyer?

For me personally, it was much easier for me to work with a lawyer, and my lawyer specializes in business law. So he helped me write my LLC charter and he filed appropriate paperwork for me as well. I still consult with him from time to time every time when I pick up a new client, just in case there are any clauses in the contracts I'm signing that have any conflicts of interest.

Thanks, Susan. That was an informative and practical introduction to the legal aspects of freelance data science. I'm looking forward to speaking again.

Soon.

After that interlude, it's time to jump back into our chat with Alan Nicholl. This may be a relatively naive question, but I think naive questions can give us a lot of insight. Why should we care about chatbots or conversational software?

I think it's a great question. I think one of the main reasons I'm really excited about it is that it makes computers usable by people who aren't experts. And that can be on a very sort of banal level, or it can be on a more sophisticated level. So, for example, my grandfather never used a computer in his life. He knew what the word Internet was, but he didn't know what it meant and he never had any experience of it. And then I think about my parents, who are to some degree computer literate, but also certainly not know the same as my generation. And then I think of how quickly technology is progressing, right? So if we all believe that tech goes sort of exponentially fast, then if my grandfather was behind, imagine how far behind we're going to be when we're old. So I think a big step that we need to take is, okay, how do we, rather than have the need for computer literacy, how does the computer adapt to deal with how we already think about things? And it's sometimes for really simple stuff, right? Just asking a simple question and getting an answer rather than having to, you know, open up a document and find where it is or something. And then sometimes it's for more sophisticated things, right? I always like to think, how many Google searches do you think there are every year for how do I do X in software Y? And then imagine that you didn't have to Google that You just said to the software, how do I do X? Right. So Photoshop is something I know is very powerful. I have no idea how to use it, but I could express some of the simple things I'd like to get done right, you know, add a blue border around this image or something. You know, why can't I just engage with computers by, by saying what I want and just having it happen? So I think it's, it's powerful in a lot of different ways. And especially for making tech more inclusive, it's really important.

That's great because that also speaks to a very personal individual power. Not the type of power that you know when you're working with corporations or businesses achieving, you know, business goals, but really on, on the ground empowering.

Yeah, yeah, absolutely.

Which verticals and industries are currently most interested in using conversational software?

So my sample is definitely biased because, you know, I work on Rasa. So the companies that we know about and that approach us are the ones that find this important enough that they want to invest their own engineers into building this out and they realize it's strategically important and they want to build up these capabilities. And I'm sure there are other industries that, where it's really relevant, but know the considerations are different and they, you know, they don't want to necessarily run the tech or own the tech in house. But what we see the kind of strongest pull from industries in financial services, so insurance and banking, and then another one which is automotive. And I think that makes a lot of sense. I remember when I had, when I got the Amazon Echo, I had it for about six months and you know, I thought it was, it was cool, it was kind of fun, but wasn't a game changer. And then I went to conference in the south of Spain and I rented a car and I had to drive like three hours from the airport. And I remember just driving on the motorway in a foreign country and then I had my phone clipped to the dashboard and I was kind of tapping on it to change the map and like to change the song on Spotify. And I just thought, this is so dangerous. Why can't I just talk to the car and say play this song right? The same way I was used to doing with the Echo at home. So I think automotive is also a really valid use case for, especially for voice interactions for sure.

And that's very clear where the value can be delivered. As the example you just gave was a paradigm, I think. How about in insurance and banking? What type of value can conversational software deliver?

There, I think it's on different levels. If you think about what is your main interface to your bank, to the data that you have with the bank and how do you engage with it. You know, some banks have really nice mobile apps and you can just do everything there. But one other interface that a lot of people fall back to is just going to the branch, right. And talking to a human. And why do you do that? Because you have questions. Right. And you can't ask a question of an app generally. Right. You can do the things that were, that were suggested to you that the developers decided to implement, but you can't say the things that you don't know. Right. Or ask for the things that haven't been implemented. And so kind of asking for advice, understanding things in a bit more depth. Those are all things where there's a lot of value to be added, especially on insurance, where the, you know, the actual product that you buy, the policy can be very complicated and hard to understand. And, you know, talking to a human is also kind of tedious. You have to make time, take time out of your day to engage with them. And so, you know, if you have a kind of 24, 7 agent that you can chat with and can answer, you know, most of your questions, it's actually really powerful.

And having some sort of intelligent conversational software for banking in that sense, for, you know, questions and FAQs and that, that type of stuff would stop me becoming infuriated when I call the bank I'm on, you know, you end up in one of those graphs that directs you downwards and you just keep shouting representative like 15 times in order, like press zero for whatever, because it actually makes no sense. I also have the added challenge, and maybe you can speak to this, that I'm an Australian living in the United States most of the year round, so that I, I, I have to like put on a fake American accent to be understood by the automated telephone system of bank of America.

I can definitely understand that and I've seen a lot of that also. My girlfriend's from Scotland and, you know, her accent can be kind of misunderstood both by humans and by software. Also non native English speakers, right? Understanding language, so definitely, you know, understanding speech. There's been some great successes, but it's certainly not a solved problem or a commodity by any means. That's not something where we try and compete. You know, you gotta focus on something and that's not something that we do. But it's certainly a really interesting problem.

Yeah. Hey, I remember you telling me some time ago about an insurance chatbot. But it wasn't, it wasn't voice, it was an SMS chatbot. But maybe you could tell us a bit about, you know, these, these types of use cases.

Yeah, and that actually was a really interesting one. So it's a, it's a large insurance company in Switzerland called Helvetia, really big company, it's like 160 years old and they wanted to do something to engage people whose house insurance policies were about to expire. So these are five year policies and they run out and they have a large customer base. But because employing people in Switzerland is very expensive, it actually literally wasn't worth their time to have an agent call up all these people and ask if they want to renew their policy. And so what they did was they built a chatbot actually with Raza and it went out over SMS and engaged with these people and it said, hey, your policy is due to expire and is your living situation still the same? We'd ask some questions, has anything changed? Maybe you've moved, maybe you got a dog or you got a more expensive car or whatever it is. And then if everything, you know, if the person wanted to renew and they collected all the data they needed, they'd say, okay, you know, here's your quote, is that cool? If they agreed to it, it would just actually finalize the policy and the policy would be in the post with them within a couple of days. So it's fully kind of end to end, automated. Right. So that's also what I mean by conversations that really do something. It's nice to answer questions, but it's a lot more powerful. You can say, okay, it's now done right, and it's on the way. And I think one of the interesting things is it challenged a lot of assumptions around what chatbots are and what they're useful for. So people think about it firstly as a customer service thing and kind of saving costs, which is certainly relevant, but if you can actually increase revenue. Right, that's really compelling. So this was actively reaching out to these people over SMS and it was a 30% conversion rate, which is really astonishing in terms of getting people to buy a new five year policy. And then, you know, the other thing was that people think about chatbots and they think about, you know, Generation Z and messaging apps, when actually the first person to buy an insurance policy of this chatbot was a 55 year old Swiss lady. And so that's all, yeah, really interesting and I think also speaks to the fact that it does make Tech more accessible for a larger group of people. Right. You can just speak to these systems and you let your customers speak to you how they think about the problem rather than forcing them into your paradigm, which is the software that you've built for them to engage with your company.

Exactly. So you've spoken to some really interesting use cases in insurance, banking, the automotive industry. Are there industries that aren't as interested in conversational software as you think they should be?

That's a really good question. I am not sure I have a good answer to it. I mean I have personal frustrations with things like telecom companies getting your Internet set up in your flat and all that kind of stuff where it's certainly be a lot more useful if you could have a kind of 24, 7 automated support that you could chat to and get things done. But I'm not sure if I can think of off the shelf examples of industries that are really neglecting this. We see a really big, actually a really big variety of companies reaching out, telling us that they're using browser, sometimes asking about the enterprise or sometimes just have some questions. It's actually much more diverse than the use cases I listed but definitely the dominant ones are things like financial services and automotive.

Okay, great. And as you mentioned expressed your frustrations with you know, calling out telcos for example. I actually remembered, you know, government I think is probably a great example. The ir, the IRS and the tax system. I actually ended up, I mentioned my frustration with those graph, those telephone trees that you get sent down like press one for whatever and I ended up somehow when I called the IRS several years ago on, on a loop I was somehow I thought they'd be. You wouldn't be allowed loops in these grass. But I ended up on a. I was about to swear I ended up on a loop in one of these, these conversations and I ended up having hanging up. So there's definitely room for a lot more aspects of conversational software in these types of places.

Yeah, I think when you're the experience as someone who's in a phone tree is how do I most quickly get to speak to a human? And it's not how do I most quickly get the thing done Just because there's such low expectations of these things actually because they're mostly just pre qualifying what your problem is right to send you to the correct person or that when the person talks to you they already know half the information that they need and they don't have to collect that. So it's really optimization on their part and it's really not optimization on the point of you as a, as a, as a customer, you know, getting your problem solved quickly for sure.

So I think this has provided a lot of insight for our listeners into conversational software where it's being used, the ins and outs. And I'm sure a lot of them are kind of eager to about where they can get started. So I think both technical and non technical listeners alike would be interested in hearing about how they could get started with conversational software. So maybe you can give a few, a few pointers.

Yeah, definitely. So also partly because there was this big boom when Facebook messenger opened up. There are actually some really nice online tools that you can just point and click and build a little prototype of a chatbot. So dialogflow is one example, it's owned by Google, chatfuel is another. And you don't have to really write any code. You can just kind of design a prototype of your chatbot. You don't have to set up a server or anything. And you can very quickly get something that you can try out and just kind of get a feel for how something like that would work. And then if you want to go beyond that, there's lots of good resources on understanding the tech behind it, building something that you can really maintain and scale. There's of course the datacount course that we created, which really covers the fundamentals. Right. What are you really doing? What are you really going on? Kind of demystifying this concept, which if you've never worked in NLP or anything, it seems really kind of bizarre and almost impossible. Right. How do you build computer programs that understand language? It seems so sort of inconceivably hard, but we go into a lot of fundamentals there. And then if you want to build more advanced things, you can either then build everything from the ground up, which is always an interesting, especially for a learning experience, interesting approach. Or you can use something like Rasa, you know, an open source library where a lot of the heavy lifting is done for you and you can then kind of get started, build something out and then, you know, really tweak parameters to get better performance on your data set, give it to real people and it's right. But I'd say the best way, if you just think, if you want to tinker, then okay, build a chatbot first and give it to some people. But if you want to actually solve a problem and you actually want to understand what people do, then actually the best way is not to have a chatbot, but just to pretend to Be a chatbot yourself and just say that it's a bot and ask people to talk to it and then answer them yourself. If people don't like the experience with your brain behind it, then they're definitely not going to like the experience with a sort of artificial machine learning system behind it. So that's a great way to kind of validate if what you're doing actually makes sense and really get some inspiration for all the things that you didn't think about. And so one mantra that we have, and it's really one of the principles that we use to build our products, is that real conversations are more important than hypothetical ones. So I'm less interested in giving people tools to design hypothetical situations and think about all possible conversations that people could have. It's much more important to look at real conversations that people do have and learning from them. And that's what Rasacor is all about and is learning how to have conversations from real data.

Absolutely. There's so much in there. And I just want to kind of recap a couple of the takeaways. The first takeaway for me, of course, is take this Data Camp course. I do think it provides a wonderful introduction. I mean, in the first chapter, you get to build a chatbot, which is based around one of the early chatbots, the Eliza chatbot, which is. Which is incredible. You do a bunch of natural language understanding, and then you get to build a virtual assistant that it's a personal assistant that helps you plan trips, which is. Which is really cool. I think you get a lot of insight in this course, course into everything, everything we've been talking about. The second takeaway, I think, is that you need to think about what the purpose of your chatbot is. For example, a lot of people, I think, have a. Have a misconception that, you know, chatbots need to sound like humans, for example. But the question is, if you're. If you've got a virtual assistant who's going to help you build a trip, do you care whether it sounds like a human at all or do you want it to do the job you want it to do? Right?

Yeah. I have to agree with you. I do think there's a big, important topic as well, around design and writing good copy and being empathetic and doing things like active listening, and that's a whole kind of orthogonal set of problems to what we're really tackling. It is important. But of course, the question is, do you actually solve a problem? Do you actually do something for people? You know, I end up using at least a couple of times a month, some product that I just completely hate, but I have to use it because it's the only thing that gets the job done that I need. Right? And so I think also in startups, you know, if you think about product market fit, you almost want to put barriers in people's way. You almost want to have your product, your first version of your product, be really crappy, maybe even intentionally so, just to see people persevere because it really solves a problem for them, right?

We'll jump right back into our interview with Alan Nicol after a short segment. Now it's time for a segment called Data Science Best Practices. In this segment, Justin Boyce from Caltech is back to tell us about when we should use logarithmic scales in plotting. Justin, you seem to be talking a lot about plotting in these segments.

Yes, I have been clear. Informative plots are really one of the most important products a data scientist can produce. So I guess I talk about it a lot because it's important.

Before we hear what you have today, I'll remind our listeners that they can check out the plots you're describing here by clicking the links in the notes for this podcast. That said, what have you got for.

Us, Justin, Today I want to talk about using logarithmic scales, also known as log scales, when constructing plots. When we first make a plot, we usually choose a linear scale. To be clear on what this means, imagine we have three evenly spaced ticks on a linearly scaled axis, say at 5, 10 and 15. The difference between the value of the second tick and the first tick is the same as the difference between the value at the third and at the second. The difference is 5 in both cases. Now, conversely, if we have three evenly spaced ticks on an axis with a log scale, they could be at 5, 10, and 20. The ratio of the value of the second tick to that as the first is the same as the ratio of the value of the third tick to that of the second. In this case, that ratio is 2. So a log scale is used when the data you are plotting vary over orders of magnitude.

Great. So can you give us an example of where we might use a log scale instead of a linear one?

We often use them in scatter plots. Say we suspect that more populous places have higher home prices. We want to plot the county level median home price per square foot versus the county populations. County populations in the US vary over several orders of magnitude from under 100 people in Kalauao county in Hawaii to over 10 million people in Los Angeles county and everything in between. So we should choose a logarithmic scale for the population axis of this scatter plot.

But what about the home prices? Don't those vary over orders of magnitude too? I live in New York, and it sure feels like people are paying a lot less in other parts of the country.

That's true, they are. You could also plot the price axis on a logarithmic scale, but here you might consider what you want to learn from the plot. Choosing a logarithmic scale will nicely stretch the data at the lower prices to visualize them better. And we should probably perform a regression using a log log scale if we wanted to quantify the relationship between the two variables. However, plotting things on a log scale visually dulls the magnitude of outliers. For example, the median price per square foot of a home in Manhattan is over $1,600, while that in San Francisco is just over 1,000. This is a big difference, and it is not as striking when plotted on a log scale.

So it is in many ways about what you're looking to discover in the data?

Yes, I would say so. But it is important that you don't choose log or linear scales to help with some sort of confirmation bias. When in doubt, just show it plotted both ways.

Are there situations where you use a log scale that's not in scatter plots, say with empirical cumulative distribution function plots, which I know you love?

Yeah. It's often a good idea to have your X axis on a log scale if you are plotting an ECDF of a variable that varies over several orders of magnitude. Interestingly, if you are trying to see if a variable is power law distributed, you can plot the empirical complementary cumulative distribution function, the eccdf, with both axes on log scales. If you see a strong linear trend on the log log plot, you very likely have yourself a power law. This is fun to do with the population of metropolitan areas in the US which follow a power law known as Zip's law.

Thanks, Justin.

No problem. You know, I wonder, if you plot the number of weekly listeners of data framed with the y axis having a log scale, does it look linear with positive slope? If so, you've got exponential growth, my friend.

Ha. I'll go do that now. See you next time.

Justin, always a pleasure.

Hugo.

Time to get straight back into our chat with Alan. So, what are the most common misconceptions surrounding conversational software that that. That you want to correct?

So I think the first one is that it's kind of really artificial intelligence, maybe even artificial general intelligence. And that you need to wait for, you know, DeepMind or OpenAI to solve these grand problems and then you'll be able to download a brain from the Internet and, and that's going to magically solve all your problems. You can build stuff now with the techniques that exist and you just got to put in a bit of work, right? And you don't need a degree in statistics or computer science or anything like that to build these things. You can definitely self teach and you can build something that's really useful and adds a lot of value with what's out there now. And you shouldn't even try, I think, to, to build a kind of do everything kind of system. And then I think another one is that only Facebook, Google, Amazon, Apple have the tech to build conversational AI. It's just not true and we see that a lot. There's also some academic papers published where they compare Rasa to some of these closed source tools from Google and Microsoft. And actually it does very well because there's no fairy dose there. I mean this is, I don't want to pitch Razor too much, but the way Microsoft and IBM think about this problem is we have magical algorithms in the cloud. Upload your data here and we will turn it into gold. And I think that's just nonsense. You can build a lot of great things, even better things with open source tools because you don't have to, you have full control, right? You can tweak things, you can customize things for your use case. And so it's definitely not something that's only in the domain of the big tech companies. And then another one that we already spoke to a little bit is that this kind of generational bias that it's only something kind of fun and light and for millennials, it's actually something especially for, you know, maybe the boomers and older generations as well. It's a really important piece of tech now.

So one thing you mentioned was this idea of product market fit. And I kind of want to zoom in on where Rasa core came from, how it emerged and I want to approach it from, I suppose the idea that we discussed at the start of this conversation about needing to have convers software that scales. So maybe you could tell us a bit about the landscape before Rasicor and then how Rasicore emerged in relation to the idea of scaling this software.

Yeah, definitely. So before Rasacore came out, there were some nice sort of cloud APIs for doing the NLU part. Right. So you Send it a sentence and it sends you back its interpretation of what that means. What's the intent, what entities are in there? And then the question is, what do you do in response to that? Right? So you have things like, you know, yes and no. And of course the response to that depends on what happened before. And so what do you do? Well, you write out some rules, right? So you write a rule for, okay, if we currently, if the last thing we asked was this and the person says yes, then proceed down this direction. If the last thing the person said was no, then go down this path, right? And that's great. And then you say, okay, well we now need to maintain that state over multiple turns in a conversation. So you don't just build kind of point A to point B, you build a state machine and you say, okay, this person is currently in this state. Now they've said this, I've moved into this state where they're now kind of maybe ready to make a purchase or they're asking about this topic or whatever. And that of course works to an extent. But then you deploy this and you give it to people and you ask them a yes or no question, right? Like one example is, should we just send that to your home address? And then you've maybe built the branch for yes and you build the branch for no. And then the person responds, oh, what home address do you have on record for me? There's always an edge case that you haven't thought of, which is fine. And of course all software has edge cases. But then the question is, how do you deal with that? If that's an important thing that lots of people say, you need to answer that to them. You can either then add another nested if statement in your logic, or you can add another state to your state machine, which then is kind of explaining some deeper information, which then probably you can get into that state from multiple different other states. So you have kind of one new state, but then you have order N squared ways of getting in and out, right? Because you have all the other states and that very, very quickly becomes unmanageable. So you have this kind of bag of rules about how people navigate this state space. And then whenever you want to change something, it clashes with old rules or breaks something else. And the thing is that, you know, mid conversation, it's absolutely trivial to know if the chatbot said the wrong thing, right? You can give that to a four year old and say, no, that's nonsense, that doesn't make sense. But it's really hard to Then figure out, like from that big state machine, why, why did that happen? Right? Why did that go wrong, kind of reversing the logic. And so we said, okay, why don't we do it completely differently? We don't build up that same machine at all, and we just say, okay, here's a conversation that went wrong, right? Say what should have happened instead. Add that to your training data and then train a machine learning model that learns how to have these conversations. And so rather than having a fixed solid representation of this state machine, you learn a kind of continuous fuzzy representation, like a continuous vector that represents that state state. And you learn that such that you can do these conversations that you've had, you've seen, and then you can measure how well it can generalize some of these patterns. So there's a recent paper that we just finished, I'll make public very soon, where we actually study how well you can take these general patterns. Like, you know, answering a clarification question like which address do you have on file for me? And then reusing that in different contexts and even in different domains. And that's kind of what we've always wanted to do with Rasa Core is say, okay, throw away your state machine. Don't try and anticipate and write rules for every possible conversation. Because it's basically impossible to build a flowchart and reason about every conversation that somebody could have because it's just a combinatorially big space. We said, just don't do that. Just have real conversations and learn from them. That's kind of where Rosin Core came from. And there was lots of research on doing machine learning based dialogue management. We certainly didn't invent that, but we had to do things a little differently from how this sort of academic world was doing it. They were doing a lot of work on reinforcement learning. There were sort of technical reasons why that didn't make sense for people getting started with browser core. I mean, the short version is basically you have to implement a reward function that's not a trivial thing to do. So we said, okay, we don't go for reinforcement learning. We do everything with supervised learning, and we let people build up these conversations interactively by talking to the system. And that's kind of what we've been doing with browser core is taking some of the ideas from the literature, the ones that we think are most applicable, and then kind of straying from the literature where we think it makes sense to, and then building a library that lets people who don't have a PhD in statistical dialogue systems actually build machine learning based dialogue and not have to have to build these kind of unmaintainable state machines.

That's great. And this movement from state machines to the importance of real conversations and machine learning, implementations of machine learning algorithms which learn from more and more conversations and more training data, real data is incredible. So what happens next? What does the future of conversational software look like? Thinking about it in these terms, that's really interesting.

I think for developers it's a really cool time to be in this and we engage a lot with our community. That's probably my favorite thing about open source is we have literally thousands and thousands of developers who use our software. We have over 100 contributors to the code base who are just people who are using it for their own purposes and contribute back. And there's this real sense of excitement. People are building and inventing new things. And I kind of think of it like being a web developer in the 90s. If you build websites now or you do front end development, it's well understood what you have to do building a CRUD app. But none of that's really been invented yet for conversational AI. I mean, some of the kind of early versions of it are there, but as a developer it's like, okay, this is kind of greenfield. I get to invent a lot of stuff. So that's really fun. But there's still lots of challenges. So it's not by any means a solved problem. And so that's why we invest so heavily in research, in kind of shipping, breaking through the limitations of what we see currently in our own libraries and in what people do in academia and kind of pushing beyond that and shipping that into the libraries. And I think on the consumer side, the future is lots of little magical moments where something just works. And so I had a great one. Literally yesterday I was on Google Analytics and it's not something I look at very often, maybe once every few months, just because I was working on our documentation and I wanted to know how many people view our documentation on a mobile device. And I'm not an expert in Google Analytics, so I would have had to go and build some filter, but they have this cool feature where you can just ask a question and I literally just typed it into the box. How many of our users@nlu.rasa.com our documentation are on a mobile client. And it just answered, you know, 6%. I thought, amazing. I didn't have to do anything. And it worked the first time, right? And those kind of magical moments I think are really cool, especially fun because that was one of the first use cases that we worked on back in early 2016 that we had some, we actually had some paying customers for. And then to see that kind of deployed at scale in the wild is, was really exciting. So I think lots of amazing, a lot of really magical moments that we'll all experience over the following years and what we won't have anytime soon is the kind of do everything magical assistant that replaces a human butler or something. Yeah.

And it's great to hear your experience as a user as well as someone who develops a lot of these things also. So you spoke to the future challenge or current and future challenge of scaling to multiple use cases. Are there any other big challenges facing conversational software development?

Yes, I do think the biggest one is, okay, you have something which works in a narrow domain and then how do you extend it to more domains? And I think, you know, we recently saw this demo from Google Duplex where they showed, I mean, primarily very, very impressive text to speech, you know, speech synthesis, but also like a very nice functioning dialogue system that can handle quite a bit of complexity. And they're very open in their post about the software that it only works because it's very limited and it works on restaurants and hairdresser appointments. And then the question is, okay, how do you build the next hundred use cases? And if you look at Amazon, I think there's 6,000 engineers working on Alexa. So that's a big effort. It's not something you can just do without a lot of hard graft. And then the other part is really not just around the technical challenge of building it, but it's really also the conceptual challenge for programmers to build software where the kind of core logic is learned from data. Right. So that's very different from calling an image recognition API and you send it a picture and then it tells you that there's an Apple in the picture. And then you have a statement that says if Apple, say Apple or something. The way that conversations go is learned from real conversations that people have had that you've checked and annotated and fixed and learned from. And so managing that training data becomes a way of programming and a lot of product management needs to be invented there in terms of how do you, how do you actually do that? And that's really interesting and that's obviously something that we spend a lot of time thinking about. And I think we have some cool things in the pipeline as well. So we'll have some great things to show there in the future.

Great. And I look forward to hearing about those future developments. So Alan, my last question for you is, do you have a final call to action for all our listeners out there?

Yeah, I mean all of Rasa is open source, so go check it out. Rasa.com is the website. If you search Rasa core or Rasa NLU, Google should show you the documentation, the GitHub repository and build something. Try it out, let us know how it goes. I'm really curious. People always come up with infinitely creative use cases and yeah, if you have any problems, you know, let us know.

Fantastic. Alan, it's been an absolute pleasure having you on the show. Thanks for joining Our conversation with Alan about conversational software and chatbots we saw that two major verticals that are currently being disrupted by these technologies are the automotive and finance industries. We also saw a wonderful illustration of how chatbots aren't merely for millennials. In the example of an insurance chatbot helping a 55 year old to buy insurance, we saw that the biggest challenge facing the conversational software developer community is scaling the software both in terms of natural language understanding, otherwise known as nlu, and dialogue management and the impact that moving from state machines to machine learning can have on this. Alan also made clear, and I cannot stress this enough, that we need to dissolve the misconception of artificial general intelligence being the key to solving these problems. We've seen this before and we'll see it again, that many of the algorithms to solve many of these challenges are and will be specialized. Also, make sure to check out our next episode, a conversation with Derek Johnson, an epidemiologist with Doctors Without Borders who leverages data science to investigate the barriers impeding people from accessing health in Lahe Township, Myanmar. If you thought data science was all SQL databases, A B tests and product management, this is going to be a real trip. As to get the data for their baseline health assessments, Derek and his team ride motorcycles into villages in northern Myanmar for weeks on end to perform in person surveys equipped with translators and pens and paper because they can't be guaranteed of electricity. Derek also uses data scientific techniques to research the factors associated with the transmission of hepatitis C between family members and has helped to conduct studies in Uganda, Nepal and India. All this and more in next week's episode. I'm your host Hugo Bown Anderson. You can follow me on Twitter Hugo Bowne and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com community podcast and see.

RA.