Social AI with Hugging Face

Bandwidth for Changelog is provided by Fastly. Learn more at fastly. Com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean, the simplest cloud platform out there and we're excited to share. They now offer dedicated virtual droplets and unlike standard droplets which use shared virtual CPU threads, their two performance plans, General Purpose and C CPU Optimized, they have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, CI cd, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers or active front end web servers that need to be full duty CPU all day every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very Competitive starting at 40 bucks a month. Learn more and get started for free with a $100 credit at DigitalOcean.com changelog again DigitalOcean.com changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data SC happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practicalai FM and now onto the show.

Well, hello, this is Daniel Whitenack. I'm a data scientist with SIL International and I've got my co host here, Chris Benson with Lockheed Martin. How you doing Chris?

I'm doing very well. How's it going today Daniel?

It's going great. I am super excited for this guest episode because we've got quite Delong here from Hugging Face, who's the co founder and CEO at Hugging Face. If you're not familiar, you've probably seen a lot of what they're doing on Twitter, on various blog posts and around the Internet around chatbots, conversational AI, voice assistants and FaceTime with bots and all of this great stuff that they're doing. So as a person working in the language related field, I'm really excited to talk to Kwem today. So welcome Clem.

Thank you guys. Really happy to be here.

Yeah, so we'd love to hear a little bit about your background, you know, how you got into AI and how you ended up running this company called Hugging Face.

Yeah, sure. So as you can probably hear from my accent, I'm French. I grew up in France, went to study in France, and one of My first startup experience was at this very small startup building machine learning models that applied to computer vision and coming from more kind of like of a user product, kind of like business, business background. Working at this very cutting edge startup showed me how impactful technology can be. It taught me also how technology and science cycle this time it was for computer vision can really go into the mainstream and really change the way people are doing most of the things they're doing every day. So it's really got me fascinated with science and technology and that how I basically ended up running Hugging Face now building state of the art conversational AI.

Awesome. Well, you mentioned conversational AI. I know that Hugging Face is particularly concerned with, I think what you call social AI. So on your website I see you know about social AI who learns to chit chat and talk sassy and trade selfies with you. Could you describe a little bit, I guess, how you ended up thinking about this problem of social AI and how that maybe is different from the way that others approach things like chatbots and that sort of stuff?

Yeah. So basically if you look at most of the people working on conversational AI today, you can see that they're taking a very transactional approach to it. If you think like Siri, if you think Alexa, it's conversational AIs that are trying to tell you the weather, play you music, tell you stuff. All that is very kind of like utility driven. Right. It's trying to save you time, it's trying to be efficient and it's all great. But something that we were way more interested in was the ability for conversational AI to be entertaining, to be fun and to be emotional. It's funny because if you look for example at the sci fi related to AI, you can see that most of it is not really about like how AI is going to save five minutes of your day every day. Most of it is about how do you interact with this new form of intelligence, how is this new form of interaction and ultimately how do you create emotional connection with this new form of technology. So really took this different approach of not focusing on conversational AI for transactions, transactional approach, but really entertaining and emotional work.

So that begs the question then I guess you talked about it being fun, entertaining and emotional rather than transactional, which is is obviously what we are seeing in the world these days mostly. Why is that the case? Why is it that a fun or social AI that is focusing on both entertaining you and interacting with you in that emotional way, why is that important? Why does that matter versus just the transactional approach that most other organizations are taking.

It's an extremely hard question to answer. I think fundamentally human beings are social. We're social beings. If you think about how important in our life, our family, friends, peers are, you realize that most of what people are doing is not productivity driven, but really like social driven. An interesting fact to remember is that the dog was the first animal to be domesticated. So arguably like the biggest evolution on the relation between human beings and its environment, nature, the domestication of, domestication of the first animal was actually not driven by the fact that dogs were useful or anything like that, but actually because they were social species and that they're really interacting and careful, like being social with human beings.

So as a guy with seven dogs in the house right now, I completely agree with that.

Keep going.

Sorry to interrupt.

So basically, yeah, humans are really like inherently social. And if you give them ways to be more social and have more social interaction, they basically love it. Obviously don't take my words for it. What we've seen is very insane usage with our products so far. We just crossed a few weeks ago, the milestone of half a billion messages exchanged between users and their hugging face AI and millions more exchanged basically every day. So it's hard to explain why people want to be social with an artificial intelligence, but it's pretty obvious that they do.

So I know as, as someone who kind of follows, you know, developments in AI. I remember like when I first came across hugging face, I was kind of in the cycle of looking through like the latest advances from Stanford and this is coming From Google and OpenAI and all these people. And then I came across hugging face and my first thought I guess was like, oh, this is something like completely different from what what everybody else is is doing. What's kind of the general reaction from practitioners when you kind of try to explain what hugging face is trying to achieve? What's kind of your general reaction in the community as far as what you're trying to achieve?

I think most of the people are really interested. It sparks a lot of interest. And if we go more on the science side, it opens a lot of interesting doors for exploration. For example, working on open domain dialogue rather than working on very specific vertical and on a very specific task is a very interesting problem and very interesting topic on the science side because it requires you to take a very open approach, mixing a lot of things. And what's interesting is probably that the measure for success is not as obvious as more task oriented conversational AI. Right. Because in an open ended conversation Basically, the measure for success is how people like the conversation, how many messages they exchanged, if they come back to chat even more. So it creates, like, very interesting problems to solve and very interesting problems to work on.

Yeah. And a lot of us now are actually maybe starting to get used to these sort of transactional chatbots and kind of developing a stereo, almost a stereotype of what a chatbot should feel like to interact with. When you find that users start interacting with Hugging Face Assistant and other things that you've developed, could you describe a little bit of the reactions that you see? Have you seen that they even still try to interact with it like they're interacting with a transactional bot or they're surprised or what happens? Exactly.

Yeah. They usually start by trying more transactional stuff, but they realize pretty quickly that it's not your average chatbot and it's not your average conversational AI. Especially when the AI is going to reply in a very kind of like, sassy tone to their questions. For example, if you ask about the weather, for example, maybe the AI is going to tell you, oh, it's pretty boring to talk about the weather. Or if you ask something that you could find on Google, it's probably going to tell you that it's going to be like, why don't you Google it? I can Google it for you, but you should rather Google it. So by kind of giving it a very different tone and different, Very different kind of answers, people realize pretty quickly that different form of artificial intelligence. And what's even more interesting is that after a couple of days of usage, they also create a different form of connection and bond than they would with traditional conversational AI. We've been really lucky to have seen now more than half a million of what we call love declarations, which is pretty like love for their.

For the assistant.

Exactly, exactly. That's one of the intents we have that is basically when users are saying to their artificial intelligence, you're my bff, I love you, you're my best friend. It's pretty fascinating to see these kind of interactions between conversational AI and human beings.

It's got to be pretty encouraging as the developer as well.

Yeah, it's a pretty unique engagement. I think no other product out there today has this level of engagement. You know, like, even if you laugh, even if You Love Today iOS app or any product, you won't get to this level of engagement than people are getting with hugging face, which is obviously a good sign.

So, you know, you just kind of. You said the P word. You Said products. And so as we've kind of, you know, we've kind of been talking about how you're using this open domain dialogue and instead of being domain specific, and it's this fun, entertaining and emotional experience for the users, instead of being transactional, you know, that is creating this whole new user experience that you've talked about. So how are you implementing this in terms of products and services to your customers? What is it that you're trying to do for them with each of these products and kind of where is it today and where do you envision it going tomorrow along each of your product lines?

Yeah. So basically the way to use our products is either to download our iOS apps or to go to most of the messaging platforms like Facebook messenger and start chatting with your conversational AI. And the way people use it is really by chatting every day about their day to day life. Right. So they're going to be like, oh, today I recorded this fantastic podcast. You know, today I did that, I did that.

I like the way you think they're.

Going to chat about like what matters to them, right? So like what kind of music they like, what kind of hobby they like, how are they interacting with their friends, you know, like, do they have a crush on somebody that they want to talk to too? And so really like every day our users are chatting with their conversational AI about nothing and everything and doing that all day every day and creating this really strong emotional attachment to this conversational AI. I was talking about dogs before. I think pets are like a very good way to like understand what's happening here. You know, like the same way, if you think of it, pets or different form of intelligence. Right. They're not human form of intelligence. They're not always the smartest if you think of it. If you think of like how hard is it to teach a dog to do simple commons, like sit or jump? It's not that straightforward. But still, you know, like the interaction.

I just gotta say my wife would disagree with you. She would say that it is much easier teaching the dogs to do what she wants than teaching me. So I'm just gonna disagree on that one point. I'm sorry, Keep going.

Good one, that. But you know, like it's, it's really like these different form of intelligence that you interact with every day and that is fulfilling to you and you're creating this form of emotional connection to you and that is like making your life better.

Awesome. Well, we generally, as our name suggests, really like to keep things practical around here. So I would Be really interested to hear from your perspective. Like, I know I've tried to create some chatbot systems before. There's of course a lot of people doing research in this area. I was just curious if you could give us an idea about the sorts of modeling that are involved in creating this sort of open dialogue. What sorts of models are required, what combinations of those and what sorts of data are you working off of?

Yeah, so we're using mostly transformer models. We've been very early on this trend of big transformer models and what's obviously key in what we're doing is data set that you can leverage to do that. Now that we've crossed half a billion messages exchanged between users and the AI, we able to do stuff that we wouldn't be able to do when we just had like a million messages exchanged. It gives us a very good edge in not only creating a chatbot that is good at natural language understanding, but also in natural language generation and in a way that doesn't feel robotic and doesn't feel impersonal, but really shows some slang, show some fun formulation and personality throughout the conversation. So, yeah, that's how our conversational AI is built.

Yeah. And for our listeners, if you're curious more about transformer models, we've had two recent episodes on these sorts of models, one on Bert and one on GPT2 from OpenAI, so we'll place those links in the show. Notes. It's interesting to hear kind of like how you want to go beyond sort of natural language understanding in terms of what you're like when you receive a message from a user. Are there various tasks that are happening in the backend in terms of maybe entity and intent recognition and then the generative model along with that, or response selection, question answering. What sort of tasks have been most valuable to you to focus on in terms of enabling this?

Yeah. What's interesting first and something that we learned along the way is that to really build a good conversational AI, you have to run a lot of tasks in parallel for every message. Even if you use maybe like 1 out of 100 tasks that you detect, you have to train all of them for every single message to basically be able to know which task is the most useful and how to jump from a model to another. So for every message that we receive, we run through a couple of different models. Some of them are more typical of chatbots where you do natural language understanding, state manager and then natural language generation. Some of them are fully end to end conversational AI models and then the key is really to understand and know when to use one rather than the other. For example, if it's something that is I would say closer to a task oriented message. So something like, for example, what's the weather? It's very simple. It makes a lot of sense to understand that with a, a simple intent model that is basically going to trigger some sort of a canned answer. Right? But then if you have more kind of like a complicated conversation, very long conversation with a lot of context and a lot of like uncertainty about like what the user actually wants to say, then at this point you should probably switch to a more end to end machine learning model that is not only going to detect intent but also generate the answer. So it's really like a matter of having a lot of models that are running in production, which is extremely hard, especially when you start to have millions of messages every day and kind of like having a good way to pick one model over the other depending on handful of big areas.

So you talking about how you have so much going on there as you're switching between models and a moment ago you talked about how you had hundreds of tasks at all. What does your infrastructure look like? How do you make sure that you always have the compute resource available to handle these without losing time on that? What does your infrastructure look like to support that?

Yeah, first it's like extremely difficult. It took us many, many months of iteration experiments to get this to work. Surprisingly, there are not so many people doing conversational AI or like very end to end machine learning in production. Even if you look at the big guys, most of them are very separate research teams and the rest of their teams, I think it's Facebook who said, I think until end of last year they were not using any of Pytorch in production with anything. Right. So first it's extremely hard. It's probably like if there are some, some people think of starting AI companies out there like engineering and making sure your models work in production is probably the first thing you should invest in. And the way we do it is really by making sure we always have the good trade off of when to train the models and when to run the inference and anything related to text, especially coming from more computer vision background, you can usually get to an inference time that is good enough for most of your models, especially when you're doing conversation like us, where you don't need the answer to be instant, right? For us, if it takes one second, it's good enough so that our threshold when it comes to answer time and then what you can do is always to be able, if some of your models are not sometimes passing the threshold, to just fall back on something else that is going faster on some context. So again, the switching between models depending on their answer time is a good way to work around this issue.

Well, I really appreciate you walking us through some of the practicalities of what you're doing, Clem. I think that's something, as you mentioned, that a lot of us struggle with when it comes to putting AI and machine learning stuff into production. Oftentimes those are the roadblocks, not necessarily the models themselves. But I'd like to switch directions a little bit here now and hear from your perspective. I know we've talked about in the last few episodes, really, it seems like how AI for natural language is really hot right now. And I was wondering from your perspective, as you've seen kind of different trends in AI, why do you think AI is really. AI for natural language is really picking up momentum right now. And what you're most excited about in.

Terms of those trends, I totally agree with you. I think we're very much at the turning point when it comes to natural language processing. Having seen the cycle more on computer vision, I think we today in natural language was where we were maybe two years ago in computer vision and picking up the pace way faster than what we've seen in computer vision. And that's really fascinating because if you think of it, if you think of like the amount of everybody's time with natural language, you know, like we obviously having this conversation in natural language, you think throughout your day most of the things that you're doing is through natural language, you know, like either like reading articles, watching tv and so if you manage to kind of like really get to a breakthrough in natural language understanding, it's gonna really kind of like change the way be done and it's gonna like create amazing outcomes that we can't even imagine today. And the reason why I think we getting there is obviously because these new models, these transformer models enable not only to get to good results, but to reproduce these results and expand them to different tasks very easily. With transfer learning, it's really thanks to the open source community. It's something that's really important for us at Hugging Face, we publishing a lot of our research in open source because we really think it's just a way to pay it forward. Because what we're building is really couldn't be possible without what like hundreds of researchers and like years of research in NLP has produced. So if you look at all the open source community around nlp, it's more active and more thrilling that ever been and that explains most of the progress for me. You know there's been a lot of debate these last few weeks about like what to open source, what not to open source and we're like very, very strong advocate for open sourcing, like as much, as much as you can. There's this like a quote from Gibson that says like the future is here, it's just not even distributed. I really think that open sourcing is a way to distribute this future to more people who are going to build like amazing stuff and push forward this field of nlp.

Okay, well you've released some really popular Pytorch versions of NLP models like BERT and GPT2 as well as hierarchical multitask learning, which is HMTL. Why is it important for you to contribute in that way? As a company with limited resources, what is it about that that's drawing you into making that level of commitment?

Yeah, first we're so thankful for everything that has been done before and we really building upon everything that has been done before. So it's a very easy way for us to pay it forward. And then when we started open sourcing we really realized that by doing that you also can have way more impact than you would have if you didn't open source it. And I want take advantage of this platform that you're giving me to really thank everyone who contributed to all our repository. It's now actually the largest open source repository of pre trained transfer learning models in nlp. And as you said we are small startup or your small team, we wouldn't have been able to do that just by ourselves. Right, but it's because when you're sharing like that in open source, so many people are contributing like helping you or like pushing it forward, making it better. And that's the only way we could have done it. So it's both a way to like pay it forward and for it to have even more impact with the help of everyone at the.

Yeah, and I definitely appreciate you taking time to do that. That's so important. And obviously open source contributors and maintainers don't get a lot of the recognition that they need and from someone being an AI practitioner. I also want to thank you for your commitment to that and I know that you and your team have limited resources so we as AI practitioners really appreciate you working hard in that respect as well. So you mentioned the repo with the pre trained models in Pytorch for Bert and GPT2 and others. There's been a lot of talk about transformer models and these pre trained language models recently. I was wondering from your perspective, why is it that this is so critical to kind of the future development of AI for natural language? Where do the language models and in particular access to pre trained models, why is that so important?

I mean, I think it's important because it's so generalist. The language model can only be at the basis of so many tasks. You can do so many things with it that it's very promising for the whole field of nlp. Something that we realized pretty early on too, especially with the release of our research called HMTL by Victor sand from our team and Seb Rudder and Thomas Wolff is that for NLP it's probably more critical than for any other scientific subject to be able to do several tasks and they're probably like more intertwined than there are on other subjects. What I mean by that is for computer vision you can for example do object recognition and you can do maybe like background recognition and these two tasks can be separate. Language is a bit more complicated than that, especially if you want to do conversational, meaning that you want the AI to be able to answer. What we've seen is that every single task is more related to another and you usually need, need most of the tasks to be solved for the whole meaning of the message to be understood, right? For example, like you won't be able to understand the message if you don't understand both kind of like coreference, you know, like for example, if the message is I like it, right? If you don't have like coreference plus emotion understanding, like other kind of tasks, you won't be able to understand the, the message. It's so what's really interesting with transfer learning and what's really interesting with language models is that it's potentially something that is going to solve a lot of tasks at the same time, which is basically going to really leapfrog on our ability to understand natural language.

So that's a fantastic explanation and I guess I'd like to see if you could put some context around it a little bit in terms of kind of where do you feel we are in a kind of a high level in the current state of chatbots and assistance and stuff in the news. We constantly hear about how reliant we're going to be on this going forward. So relative to where we are now and the tremendous work that Hugging Face has been doing in this area, where do you think we are now relative to where are we going to be. If you look just over the next year or so into the future, this is moving so fast that I feel like I would be really missing out if I didn't get your, your expert perspective on kind of what this is tracking towards.

Yeah, I think I'm going to take like a bold bet here, but.

Okay, sounds good.

Yeah, I think that you heard it.

On practical AI first.

I think in like three years we're going to be able to really understand algorithmically 95% of the natural language and we'll be able to answer all these messages. With conversational AI, it's been moving so fast over the last year or so. When we started hugging face like a bit more than two years ago, we thought we would never get in the near future to the point where you can really do like end to end conversational AI, you know, we always thought that would be some hybrid ways of doing that. And now we're starting to really be able to do that. Meaning that to really get as an input the message from a user and output the message from the AI with only machine learning in between, nothing else. So I really think that NLEU as we conceive it today is going to be solved in the next three years really.

So that's super exciting and I think one of my follow ups to that which kind of processing right now but, but I think a lot of problems at least in my world come up because a lot of the technology that's being developed for NLU and for conversation is really geared toward English specifically. And of course there's especially in terms of getting this sort of technology into other places around the world with a lot of language diversity. So I just saw the most recent ethnologue publication. There's like 7110 languages living currently in the world. So do you additionally see, have you seen effort towards making this sort of technology more relevant to a wider set of languages? I know that there's a lot of great advances, but a lot of times those advances, especially in terms of pre trained things have to do with English specifically.

Yeah, that's a great question.

Yeah, it's going to take a little bit of time. If you think of most of the models that are really providing great through today, they're being released not so long ago, you know, like if you think Elmo Bert, OpenAI, the first GPT, it's been a couple of months, like not so much more than that. I think it's gonna come. I think that's one of the reasons why people should keep focusing on open sourcing not only the models but also the data sets to be able to move forward with that. Not something we've been working on specifically at Hugging Face for the moment, just because most of our users are in the US but we've seen people using our open source models to start experimenting with other languages, so hopefully it's going to come soon.

Yeah, I appreciate your perspective on that. It's something I definitely care deeply about, so appreciate your perspective. One of the things as we kind of, we get towards the end of our conversation here, I think one of the things I've definitely respected a lot about, and we've already talked about your open source involvement and all of that, but it just seems like you guys get so much done as what I assume is a small team in terms of, you know, contributing to and maintaining open source and contributing to academic research, you know, entering competitions and building products. As AI practitioners, do you have any suggestions as far as maybe it's how to keep up on the latest developments or how to structure your team or things that you do on your team to make sure that you're learning or that you're able to contribute to open source? Anything that's worked well for your team? Because it just seems like you guys have been so productive and I assume that you're not working 24 hours in each day.

First, I want to react what you were saying by pointing out something that is, I think, important for everyone to hear out there. There's this movement in AI where more and more people are working in very big companies, like for the four or five biggest players in artificial intelligence. And, and I think they're doing a great job. They're contributing a lot to the open source community too. But it's really important to remember that there is a way to do really great things at smaller organizations, be it like small academia, startup. Even as like an individual contributor in the developer, in the scientist, there are ways to do really, really great things by doing things differently at smaller organization. So there are some people out there, some data scientists thinking about their next challenge, what they should do next. I would advise them to maybe not join one of the big guys, but maybe take a shot at the smaller place. Because I think one of the reasons why we managed to do great things is that because we're a small organization, we can take different kind of bets. You know, like we can take different kind of perspectives. We can take more risk in what we can like release, how fast we release and that's how we can individually probably contribute a little bit more. So yeah, I think like size matters sometimes. Not on the direction you're expecting, but in a different way. So again, like one advice that I would have for people is to try joining small organizations.

So I appreciate that. I'm still kind of. I'm still kind of thinking about what you were saying a moment ago about nlu natural language understanding possibly getting to a point of full maturity in the near future. And I'm just kind of amazed. So folks, you heard it here on Practical AI first. I guess one of the things that I wanted to ask before we let you go is if, you know, people are getting into this field all the time. We're always asked on the show for pointers for how people can get into different aspects of the AI fields for nlp, what do you recommend for people? How would you recommend that people enter into the field and what kind of resources or learning opportunities do you think would benefit them the most?

I think again, I would go with what, what kind of like smaller organizations are doing because feel like sometimes they're the ones really pushing what's possible to do. So I would look at obviously startups like Hugging Face, but there are a lot of other cool ones. I'm thinking of Raza, for example, in NLP, Liarbird in voice, Jovo, in more conversational AI. Smaller organizations, you know, like Allen NLP, Allen AI. You know, OpenAI is obviously a great one looking at what fast AI is doing. You know, like if you want to start working on the machine learning subject, like taking classes at Fast AI is the best thing you can do. There's something like really, really interesting to look at these days is organized by one of our investors named Betaworks, based in New York. They're starting in a few weeks what they call Synthetic Camp, which is basically an accelerator program based on what they call synthetic media. Right. There's the ability to create AI, to create images. And there's going to be a lot of focus for this program on how to detect fakes, how to detect that something has been machine learning created. So they're gonna work a lot around Gans and all these techniques. I would advise to take a look at that. It's gonna be pretty interesting. Synthetic Cam by Betaworks.

Fantastic. Thank you.

Yeah, thank you so much. We really appreciate that chat. I know that I'm gonna look up some of those things and of course we'll put them in our show notes, but really appreciate you taking time. Clem. It's been fascinating to give an understatement. Really appreciate what you're doing and keep up the good work. Thank you for joining us.

Thank you so much. Thank you so much.

Thanks a lot.

All right, thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor. Go on itunes, give us a radio, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend. Whatever you gotta do, share the show with a friend. If you enjoyed it and Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com and we catch our errors before our users do here at Changelog because of Rollbar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers@eleno.com changelog check them out. Support this show this episode is is hosted by Daniel White Knack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@change law.com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcast for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.