EP04 - Interview with Jeff Herbst, NVIDIA

Hello everyone, and welcome to the Tech Journal. My name is Marc Vermenhom and I am the digital speaker in this series. I project my digital twin into cyberspace to bring you the latest and greatest from the digital world. I cover all the digital news, from blockchain and crypto through to quantum computing and AI. I always try to take it a step further and look into what these digital technologies mean for our personal and professional lives. That is why today we are taking a different approach. And I will be interviewing Jeff Herbst, Nvidia Vice President of Business Development and head of Inception, GPU Ventures. Welcome, Jeff. As Head of Inception at GPU Ventures, you come across many startups. What are some of the trends that you see? How are startups applying GPUs, and what are some of the most promising novel applications that you see?

I love startups. Nvidia loves startups. I think about startups all day long and I have been doing that for the last 19 years. And so we created this program called Inception, and basically it's a virtual acceleration platform for companies building AI applications. At this point, we've got about 6,000 or more than 6,000 companies in the program around the world. We created it to basically enable an ecosystem of companies to develop AI applications. So what we do for the companies is we give them developer support, we give them access to our tools, we give them credits in some of the GPU clouds, aws, Microsoft Azure, we give them classes, we teach them how to do AI. It's called doi. And then sometimes we even invest in the companies. And that's where our inception GPU Ventures or program comes in. As part of the program, we also work with all the VCs around the world to help them teach their companies about AI and how the Nvidia and GPU platform can be helpful to them. I'd say within Inception, out of those 6,000 plus companies, the largest single group actually right now is healthcare. Healthcare is an industry just ripe for disruption. And the things that have stopped it in the past is obviously technology barriers, but the workflow just doesn't work. And so there's an amazing number of companies that are using AI to recognize diseases, whether it be cancer, to do drug discovery. We're even working with companies that are making hospitals and doctors offices more efficiency by tracking the workflows and making recommendations to how to make it better. Retail is another area. I mean, if you think about what's gone on in this unfortunate period, we're in with this coronavirus pandemic. More and more things in retail have been brought online and getting those systems to work and analyzing the data and the trends is really difficult. And AI is a perfect solution for that. And as more and more of it goes online, it creates a huge opportunity for companies to go do that as, you know, self driving cars, autonomous vehicles, huge opportunity for AI, smart cities, and eventually even virtual reality. So what you're involved in is going to benefit greatly from AI avatars. Creating reality from virtual situations takes a lot of computing horsepower. And AI is a perfect solution for that because it can learn from the patterns of the past and create future instances and future conversations that actually not just mimic, but are created out of those past experiences. Because what AI allows you to do is it allows you to collect knowledge and collect patterns and innovate from there. So really, really exciting stuff.

Recently, Nvidia and Mercedes announced a partnership. Can you tell us a bit about this partnership? Why is it important for Nvidia and how can it change transportation?

Yeah, super important and I know it did get a lot of attention, but quite frankly, I don't think people realize the groundbreaking nature of the partnership. And it's great for Mercedes, it's great for us, it's great for Europe, it's great for the world, basically. Starting in 2014, our Nvidia Drive AGX Orin architecture will be rolled out across the entire fleet of next generation Mercedes Benz V vehicles. And it's basically gonna, it's basically an affirmation by Mercedes that you need to design the car around the computer. So, you know, I've spent a fair amount of time with Our Automotive team. 19 years in Nvidia. I've watched us build the automotive business. And what I learned about cars and how they've been built in the past is they're a combination of like hundreds of different systems that get connected over a can bust to some kind of central readout which tells you you got error code 53 and you got to go figure out that means your muffler is bad or something else is bad. They're not built around centralized computers. I think that the Mercedes announcement with Nvidia is turning this on its head. And it's basically saying your car starts with a computer, it's set up to do autonomous driving. And by the way, the software that we're going to continue to build keeps getting better and better. And so we're going to be able to upgrade your car over the air, just like we upgrade your computer over the air. We're going to add more functionality over time. And so these Cars can be customized as the user wants. If you don't want any autonomous driving functionality, you don't need it. If you want the most advanced, you can have it. And there'll be all kinds of different business models for Mercedes around doing that. So it's just a fundamental, it's kind of the iPhone moment for the automotive industry. And, you know, the ultimate result is cars will become better, cars will become safer and efficiency will be improved and everybody wins. And so it works in all situations and all circumstances. Very, very flexible architecture. And by Mercedes working with us, they future proof the cars moving forward. And they're going to benefit from all the computing advancements and all the software advancements we make over time. I think they're extremely insightful to work with us because they recognize they need a very, very strong partner and we're probably the best partner to work with.

In this series, we discuss how digital technologies are changing societies and businesses. How do you see digital technologies changing our world in the coming decade?

Well, in fact, the world has been changing due to digital technologies for many, many years. And you know, you can kind of look at this as kind of the latest wave, but I believe it's going to be the biggest wave and it is the biggest wave. So without dating myself too much, I have the benefit of watching the industry grow from I think I started, I just caught the end of the mainframe period when computers moved from mainframes to PCs. I actually had the opportunity to work at IBM's PC division in Boca Raton, Florida in the very early days of the PC. Then I saw things move from PCs to client server architectures. Then we moved into the Internet revolution where not only did you have a client server architecture, but all the computers in the world were connected with one big network. Then we moved to mobile, where you no longer needed a fixed line connection and mobile enabled a whole bunch of other applications to work better or new applications. Where we are now is in the biggest wave I think that we've ever seen. And the reason why this one is big is we're changing the way that you program a computer in the past, the way you made a computer do what you wanted it to do. And for those of us who coded, I mean, it seems like a mystery to many what all you're basically doing is you're creating instructions and you're telling the computer to do this, this, this, this, and add these two numbers and move this data over here. The computer cannot do anything that the human can't do. It can just do it. Billions of times faster. So you create a code loop. It works, and you've got an application that requires a lot of work. And it's basically coding driven. The revolution that we're in right now is the AI revolution. And the biggest difference is that computers now basically program themselves by recognizing patterns from large amounts of data that are fed into the system. So it's a completely different programming model. It starts with data, and from that data we recognize, or the system recognizes patterns that evolve from that data. You know, if you feed them a thousand pictures of me and you say, this is Jeff Hurst, when they. When they get the thousandth in one picture, they can go back through the other thousand. They've trained a model. They know that's me, and they know that's not you, Mark. So this is a fundamentally new technology. And the systems that program computers, the systems that create the technology, the systems that answer the questions, have been flipped on their head. And what's amazing is you no longer have to rely on the bottleneck of coding to create code and to create algorithms. The algorithms can be created by data, and there's a lot more data out there and a lot more pieces of data than there are coders. And so this. This is already creating an exponential increase in the amount of programming and code that's being created. And so it's the most fundamental change that I think we will see during our lifetimes until the next big one comes. But this one's pretty big. I think it overwhelms all the other paradigm shifts that I talked about before by a large margin, because it's exponential. You no longer are in control that the programmers no longer, you know, controlling the system. The data controls the system. And so we can, as long as we can feed data, we can create algorithms. And so that's why data becomes so important, and that's why moving data and storing data becomes so important. And that's why Nvidia made a big investment by buying a company called Mellanax last year, or actually closed this year, that essentially are experts in the movement of data between systems and processors within a system.

For companies that want to digitally transform their business, they have to stop doing digital and become digital. What would be your advice and how can you help organizations in becoming a digital business?

Well, this is what I actually think about every day. So I think a lot of companies are having what I might call the Kodak moment. So if you remember when digital photography came, the Kodaks of the world initially. And by the way, I was a photographer as A kid. I did photography in college. I was the photographer for the Brown Daily Herald at Brown University. I had my own dark rooms. I loved doing this. But as hard as that was to give up, it was pretty obvious that everything was going to move digital. And the problem is it's very, very difficult for an established company who does things one way to, to basically start again and build from the ground up. But I would say that companies really need to take this into account and basically they'll have to destroy their. Many will have to destroy their existing businesses to build the new businesses. The good news though is AI is more of a horizontal technology than a vertical technology. And so this is what we do all day, is we create, we help developers build technologies based on AI. You know, startups are a great example and the reason why we're working with more than 6,000 startups is because when you have a new technology like this, startups are often way ahead of the curve. But if the big companies are smart, what they'll do is they'll learn to collaborate with the startups, they'll learn from them, maybe buy some of the startups and you know, it doesn't have to happen overnight, but in the background they should be disrupting their business models. The good news though is if you are a big company and you have vertical expertise, whether you be a healthcare company, a retail company, a transportation company, you know, and you have access to a lot of client or customer data and information, if you're smart, you should be able to remove your existing legacy platform with an AI based platform and be okay. But they definitely need to move quickly because if they wait too long, you've got a whole bunch of startups and other companies who make. By the way, we see tons of startups who get into healthcare who know nothing about healthcare. They just think they're good programmers. You know, I've seen retail companies, they know nothing about retail industrial companies. So I think the legacy companies do have an advantage if they move quickly because ultimately domain expertise is going to be super important.

In May this year, you introduced the Jarvis framework for conversational AI. Could you tell us a bit about this framework? Why is it important and how do you think conversational AI will change businesses and society?

So I share those thoughts with you. I'm extremely interested in conversational AI and I think it's the next going to usher in the next big wave of AI development. And the reason it's so important is the things that most people do all day as part of our daily business is we Interact with people, we ask questions, we look for data and insights, we talk to customers. So I think we spend most of our time, you know, interacting, whether it be through email or through conversations. And it's an amazingly challenging task to actually recognize voice and do natural language processing and take action based on the information that's coming in. There's really three parts of it. You know, first, I've got to recognize what you're saying. You and I are speaking English. Like, you know, I have to take what you're saying and actually take those words and bring them into some kind of digital forum where I actually know what you said. So that's number one. Secondly, once I know what you said, I need to take action on it. So I have to run that data through some kind of natural language processing engine which gives a response or an answer. And that's an AI model that needs to be trained. And the third piece is I have to re encode it back if I'm a computer talking to you and basically tell you the answer and encode it back. And so what JARVIS is, is JARVIS is essentially Nvidia's GPU accelerated application framework for building AI services using an end to end deep learning pipeline. So we have created tools and APIs that basically do all the things that I just mentioned much, much faster than anyone else can do it. And I think if you look at kind of the difference between CPU based systems and GPU based systems, we're thousands or millions of times faster. I mean, just infinitely faster. And so the reason we created JARVIS is we want to spread the word and we want to create the tools for companies, businesses, developers to build these conversational AI applications. And we know that GPUs make them go faster. And so we've made this available to the world and we encourage people to develop on it. And it's no different than our kind of drive platform. It's no different than our embedded in IBA platforms. It's basically a stack of technology that developers can use to make the world better. So we're extremely excited about it. And the last thing I'll add is a lot of the early activity on deep learning and AI was based on image recognition. This all started with researchers at Stanford competing to figure out whether something was a dog or a cat and who could do it better. And once the deep learning pipelines could do it much better than any human, we knew we were there. The focus on voice really just has started in the last couple of years. And in many ways it's a much Much harder problem than image recognition and it requires a lot more computing horsepower. So I'm super excited. We've made, not only is Nvidia investing in the era, but through inception we've actually made financial investments. I mean, one company that's really interesting is called Deepgram. It's doing GPU accelerated speech recognition. And they can do it much, much faster than anyone else. Like, think about call centers that have, you know, hundreds of agents talking at the same time. What if I want to know while those things are happening, if a customer needs help or if I can help one of my salespeople close a deal? These are all things that are made possible by conversational AI and all the analytics around it.

The GPUs developed by Nvidia have significant progressed AI development. What is your opinion on the opportunities of quantum computing for AI and how is Nvidia involved in quantum computing?

Yeah, in fact, we are involved in quantum. And I'll tell you my views on quantum and Nvidia's views, which are basically the same. I mean, at a very, very high level, computing has basically been done through what we call classical techniques, which have essentially been binary logic that are computed on semiconductors, which is essentially strips of metal layers, wires embedded into silicon. And the concern by people in the industry is the laws of physics are starting to take over and at some point you just can no longer squeeze more performance out of semiconductor, current semiconductor technology. The interesting thing is that Nvidia has kind of turned that on its head and we've been able to escape the gravity of Moore's law that has, that has held back many companies through basically better algorithms, special purpose accelerators, you know, GPUs. Are those a better software, better system design, a better movement of data. That being said, at some point, maybe in our lifetimes, we'll see some type of new technology for computing that's different than the classical methods. And that's what quantum represents. And so it's great to see so many people working on this and without boring you on all the technology and all the physics. It's a fundamentally different way of representing the data and algorithms and using these things called qubits. The problem you have right now is that these systems are really expensive, they're really large, they have to be cooled to like ridiculous thermal. I mean, they basically, you know, there's smoke coming, you know, dry ice kind of smoke coming out of the systems. So they're very limited right now in the number of qubits that they can, they can, they can utilize and the quality of the qubits, they're orders of magnitude less than they need to be to do any kind of useful computation. But they're working on it and there's some large companies doing it and I think they'll eventually get, get there probably at least a decade is our view of the world to where you're going to have a real useful, commercially available quantum computer. What we can do in the meantime though is we can, we can provide the picks and shovels, so we're not out there trying to go get the gold. But what we can do is we can provide the tools. And it turns out that a GPU is an excellent tool for simulating quantum computing. And so we are able to help all these companies doing that development simulate when they actually have a system that's viable, what it's going to look like. And at the same time, when quantum computing does finally make it to commercial viability, it's still going to need a big complement in the form of classical computing. So these systems will likely be hybrids. So not only will we help them develop the quantum computers, but we'll be sitting side by side with them as part of the classical systems in the data center. So it's really a win win for us and we're watching it closely, but probably not yet ready for prime time and not for several years.

Nvidia is also heavily involved in deep learning. What is the future of deep learning and AI right now?

If you look at deep learning, it's just one small subset of AI and it's basically built, as we discussed, on neural networks that are taking huge amounts of data and are trained via pattern recognition techniques. And the interesting thing about what most people are doing with deep learning right now is it's heavily supervised. So for me to send data into my training model, I actually have to supervising that I have to label the data. So we're seeing a lot of companies out there who are solving that problem is, you know, labeling, storing, cleaning, sorting data. You know, so the cloud, the current techniques are just the start in my view. It's just the iceberg because ultimately we're going to see unsupervised deep learning techniques that hit the marketplace which make this go even faster because you won't have to basically identify all the things that are in the data in advance. We're going to see federated learning techniques where basically all the collective knowledge of the world can be shared amongst one system. Right now there's issues relating to data privacy. For example, do I really want my competitors system to be trained and using my data that I spent years and years collecting and my customer relationships. So the deep learning we see today is just one instance of AI, but there's many, many areas of AI, including machine learning and other algorithmic techniques that are going to hit the marketplace. And we're just now moving to the edge right now and we're going to see a lot more computing on the edge moving forward. And video is working on this pretty intently. And there's just so much, I mean, like I said, we are really just scratching the surface of where we can go with this. And again, it appears to be our second intractable problem. Computer graphics was the first one, so we got a lot of room to run.

Very interesting. Thank you. And on that note, that's all we have time for today. I have been your host, Marc Verrimen, the digital speaker, and this has been the Tech Journal. Do not forget to hit the like and subscribe buttons and I will see you next time. For your information, download Stay Digital.