deeplearning.ai's Heroes of Deep Learning: Yuanqing Lin

Welcome. Yan Ting. I'm really glad you could join us today.

Sure.

So, you know, today you're the head of Baidu Research, and when the Chinese government, the government of China was looking for someone to start up and build a national deep learning research lab, they tapped you to help start this thing. So, you know, arguably, I think maybe are the number one deep learning person in the entire country of China. So I'd love to ask you a lot of questions about your work, but before that, I want to hear about your personal story. So how do you end up getting to do this work that you do?

Yeah, so actually before my PhD program, my major was in kind of go for something else, because back to early 2000, I think Nandong technology was really, really hot. But I was thinking that probably I should look at something even more exciting. And there's a good chance that I was taking some classes at UPenn and I got to know Denli, so later he became my PhD advisor. And I was thinking like, machine learning was a great thing to do, and I got really excited and I kind of changed my major. So I did my PhD at UPenn. Like a majoring in machine learning. Yeah. And so that kind of. Actually I was there for five years and that was kind of really exciting. And I think I learned lots of things from scratch. Lots of algorithms, even like a pcas. I didn't know those before. Yeah, it was kind of. I feel like I was learning new things every day. So it was very, very exciting experience for me.

It was one of those teams with a lot of stars, although, you know, just did a lot of work and it was underappreciated for his time.

Right, right. That's right. Yeah. So I think ENG was an exciting place and I was there at the beginning as a researcher again. I also like to feel like, wow, I learned lots of things and I actually later kind of started working on computer vision. I actually started working on computer vision very late. I mean, relatively late. And the first thing I did earlier was I participated in the ImageNet Challenge. That's the first year of ImageNet Challenge. I was kind of managing a team to work on the project. It was lucky. We were quite lucky that we were quite strong. And at the end, we actually got the number one praise. Overwhelming number one praise in a context.

So you were the first ever in the world imagenet competition winner.

Right, right, right. Yeah. And I was the person that did the presentation at the workshop. Yeah. So I was. It was really nice experience for me and that actually get me into these really large scale computer vision tasks. Yeah. And so I have been working on this really large scale problem since then. Like and so when Lisa your cat Pavement, New York Dunn cat came out and also later on like this Alexander come out, it really blow my mind. I thought like wow, deep learning is so powerful. And I think since then I invested lots of effort, I think to work on those.

So as a head of China's National Lab, National Research Lab on Deep Learning, there must be a lot of exciting activities going on there. So for the global audience watching this, what should they know about what's happening with this national lab?

The mission of National Engineering Lab is to build a very large deep learning platform and probably hope to be the biggest one, at least the biggest one in China. And on this platform we would offer people like a deep learning framework like pedopadle and we offer people really large scale computing resource. And we also offer people very large, really big kind of data. And if people are able to develop our research or develop kind of good technology on this platform, we also offer them big applications. So for example, the technology can be plugged into some big applications in Baidu so that the technology could get iterate and improved. So we believe that combining those, kind of, combining those results all together, I think this is going to be a really powerful platform. Let me also give you kind of one example on this. For example, like right now if we publish a paper, say let's say illustration, publish a paper, if someone want to reproduce it and they need to kind of the best thing to do, probably the person will provide the code somewhere and you could download code to your computer and you also try to find the data set somewhere. And you probably also need to get a good configuration of your kind of computing resources to run smoothly. Right? So this easily kind of take some efforts at least National Lab things will become much easier. So if someone using these platforms to write a paper to do the work and then write the paper and they will have the code on this platform and the computing structure is already set up for this code and the data is there too. So basically you just need a command line to reproduce the result. So this is a huge, I think it's a big, big kind of relief of lots of reproducibility issues in computer science. So you easily just kind of, just a few seconds, you start learning something that you see in the paper. So this is extremely powerful. So this is just one example we are working on to make sure that we are providing a really powerful platform to the community and to the industry that's amazing.

That really speeds up deep learning research. Can you give a sense of how much resources the Chinese government is putting to back you for this deep learning national labor?

So I think for LISA National Engineering Lab, I think the government definitely going to invest some funding into here to build up the infrastructure. But I think more important would be LISA going to be a flagship in China that going to lead lots of kind of deep learning efforts including a national project and a lot of policies and things. So LISI actually is extremely powerful and I think for Baidu we are really honored to get LISI Lab.

You're somewhat at the heart of deep learning in China. So there's a lot of activity in China that the global audience isn't aware of yet and hasn't seen yet. So what should people outside China know about deep learning in China?

Yeah, I think in China, especially in the past few years, I think deep learning powered product and service is really booming like coming like ranging from like search engines to like a phrase recognition to like to surveillance to like E commerce like lots of place. I think they are investing big effort in the deep learning and also like they are really making use of those technology to make the business much more powerful. And this actually is very important for developing AI technology in general. I think for myself and also lots of people share this. We believe that actually it's really important to have this which often call the passive loop. For example, when we start out to think of building some technology then we will have some initial data and we try to develop some initial algorithm, we launch some initial product or service then after that we'll get the data from our users and after we get more data and we will develop a better algorithms because we see more data, we know what will be the better algorithm. So we have more data and better algorithm, we're able to have better technology for a product, service and then definitely we hope we'll be able to attract more users because of the product, the technology is better and we'll get more data. So this is a very good positive loop and it's very special actually for AI related technologies. I mean for traditional technology like Laser, like laser that I was working on before. So it's more like the growth of the technology can be more linear. But for this AI technology actually because of this positive loop you can imagine that definitely at certain point you will come with very fast growth of technology and the lease actually is super important. And when we designing the research endeavor, when we design our handi but we work on the direction that we're able to get to this quick kind of improving period, right? If we are not able to, if our business we are not able to, the whole kind of, our whole business is not able to form this positive loop, or if we are not able to form these kind of strong positive loop, then this direction probably should not work out because someone else who held a strong business to find a strong loop and they will get to this a kind of a phrase much more quicker than you are. So this actually is a really important logic for us when we're looking at, like I say, hey, in this company what direction we should work on and what directions we should not work on. This definitely is a really important factor we should look at.

Today, both in China and in the US and globally, there are a lot of people wanting to enter deep learning and wanting to enter AI. What advice would you have for someone, someone that wants to get into this field?

So nowadays definitely I think people who start with open source frameworks, I think that's extremely powerful to many starters. I think when I was starting my deep learning study, it was not much like open source resource. I think nowadays in AI or especially in deep learning, there's a really good community outlet, right? And there are multiple like a really good deep learning frameworks. It start with like a TensorFlow Cafe. Like now it's called Cafe 2, right? And in China we have really good pedal, pedal and even like for most of those online they have lots of like courses to kind of teach you how to use those. And also nowadays also there are many like publicly available benchmark and people could see, hey, very kind of skillful, well experienced people like how well they could do on those benchmarks. So basically to get familiar with deep learning, I think those are really good starting point.

How did you gain that understanding?

So actually I do it in an opposite way. I kind of learn, I learned pca, I learned like LDA and all those like before I learned deep learning actually. Is a good way. I feel like we kind of lay down lots of foundations. We learn graphical model, so these are all kind of important. Although right now deep learning is very dominant. But knowing laws actually give you very good intuition about how deep learning works. And one day probably there's a connection of deep learning to laws like a framework approach. I think there's already lots of connections and the loads actually make deep learning richer. I mean there are richer ways of doing deeper learning. So I feel like it's good to start with loads of open source preference loads are definitely extremely powerful resource. But meanwhile, I would suggest you also do learn those basic things about machine learning.

So thank you. That was fascinating. Even though I've known you for a long time, there are a lot of details you're thinking that I didn't realize until now. So thank you very much.

Thank you so much for having me.