Answering recent AI questions from Quora

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode servers. Head to linode.com changelog this episode is brought to you by DigitalOcean. They now have CPU optimized droplets with dedicated hyper threads from Best in class Intel CPUs for all your machine learning and batch processing needs. You can easily spin up their one click machine learning and AI application. This gives you immediate access to Python 3R, Jupyter Notebook, TensorFlow, SciKit and PyTorch. Use our special link to get a 100 credit for DigitalOcean and try today for free. Head to.co changelog once again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practical aifm. And now onto the show.

All right, I'm really excited to be here at O'Reilly AI with some of the intel innovators in AI and ML. Really excited to ask these guys some questions today and hear from them. But before we do that, let's kind of go around and introduce who's in the room. So I'll start with you, Vinay.

Thank you Daniel. I'm a huge fan of Practically AI, your podcast. I've listened to every one of your episodes. My name is Vinay RAO. I'm the CEO of RocketML. We make a distributed machine learning platform to help data scientists do experimentation much faster.

Awesome. And maybe everyone could share something that they're excited about in the world of AI too. What about you Vinay?

I think the market opportunity for AI, whether it's doing machine learning or putting out products, AI products is very, very large. Everybody's underestimating the market size. My own prediction is that I'm going to go on a limb here. It's going to be a trillion dollar market size.

Awesome. So let's go on there, Peter.

So thanks for having me Daniel. My name is Peter Mahn. I'm working on a project called Clean Water AI. It uses computer vision artificial intelligence to detect the bacterias and the harmful particles in water.

Awesome. That's really exciting.

So what I'm really excited about AI is what it can do and how it can apply into the current world. Right now, from Self driving down to pretty much repetitive healthcare tasks. And I think in the very near future we're pretty much going to have using it to reduce all the costs on the healthcare that a lot of doctors pretty much they're just for simple screening. At least in the United States. That's going to be very helpful.

Yeah, yeah. AI is going to make a large difference for good as well as all the, you know, malicious things that we see kind of advertised in industry. Yeah, it's great to get that perspective. What about you, Dave?

Hey, thanks for having me. So I'm Dave. This is actually my first time on practical AI. So it's very exciting to be here. So I'm a research assistant or research affiliate. It's a new position I just got at the University of Florida. I recently just finished my PhD PhDs. Thank you. So on this side I'm also looking at other projects. I'm one of the intel innovators. We had a demo at intel booth today on applied AI in retail. What am I excited about? I'm going to say FPGA. So FPGAs are essentially like GPUs for acceleration. And I do know Vinay is heavily into GPUs. Right. With Rocket ML, I'm more into FPGAs for inferencing. So I do see a very large market for inferencing. My personal thing three years ago was when scientists were really like really going hard to develop ML models. And we're in the era right now. We're trying to take those ML models and put them into production. So there's a much larger market space in that area for sure.

Awesome. Yeah.

Ali.

Yeah, this is Ali with Netrolux. It's my first time with practical AI Netrolics. We're excited to use AI in the network network field. We control the traffic engineering of our core utilizing deep learning and I agree gpu. And both because we're pushing FPGA also to the edge of the network. We're working on pushing the AI to the edge of the network.

So thanks for having me, Dan. I'm also with Netrolix and I asked Ali to join because he's much smarter than I am and the more I learn about this, the more I realize I don't know. And that being said, I like what we're doing because we're kind of the underlying technology to support all of the AI initiatives, which is the network and optimizing the network and creating a neural behavior and self correcting behavior in the network which is 90% Internet and even more so as we grow into IoT and other things like that. What excites me about AI is actually pretty simple. The adoption of it. How quickly people are going to start trusting it and actually not being scared of a Skynet or something ridiculous, but actually putting faith in it like we do here, the group of us. So thanks.

Awesome.

Yeah.

And Ali and Wes, just to kind of clarify, because we do talk about networks a lot, but you guys are talking about networks in the sense of the Internet and interconnected infrastructure, right? Not necessarily neural networks, although you're using neural networks on the network, right?

That's right. So it's funny, I say kind of tongue in cheek to people that we've harnessed the Internet. Right. But what that really means is that we're leveraging about 20,000 sensors plus a footprint of about 68 data centers globally, only 22 of which are in North America. And essentially we just collect data performance, data latency, jitter, packet loss, throughput availability, and hundreds of other metrics that we as humans will not even think to account for that impact network performance over the Internet. We take that same intelligence and apply that into security as well. So our security posture is equally as strong, powered by AI as well, all over the Internet as an OTT software. So a little bit disruptive to the service provider equipment space, but it's fun.

Yeah, that's awesome. So I have something maybe a little bit special for you guys today. Normally we just kind of do a standard interview, but since this is a panel and you guys are all experts in the field, I thought it would be fun to just pull some of the recent questions from CORA about artificial intelligence and machine learning. Get your guys perspective to hear from some of the experts about how they would answer some of these questions that people are really asking as they're diving into the field. So does that sound okay for you guys for the challenge?

Yeah.

Awesome. So the first of these questions is what do I do next when I've achieved a machine learning program at 97% accuracy and good fit? And I think really, you guys, you guys are laughing a little bit.

Stop.

This is the point that I was alluding to. Right. So few years ago, researchers were working so hard to develop models that would meet 97%, 98%, but what's next after that? Right. And so we're in the area right now, we're trying to productionize machine learning models. And it's very exciting to see startups that are coming up almost every day with a workflow to let you take machine learning models, put it into production in a more scalable fashion. So I really think that's the next phase of ML and AI. It's how do you put into production at the edge. Right. Or some IoT device.

Yeah. So what's different? Maybe to follow up on that, what's different about production or utilizing a model after you kind of trained it to get a particular evaluation metric that you might be after?

I think about your original question a little bit differently. 97% might be at particular level of problem statement. But as humans, we know how to split the problem into minute pieces. So something underneath that may not be 97%. As we collect more data, as companies collect more data, the new problems will start to emerge again. Accuracy within that would be like 70%. So you'd have to increase it. For example, if you take translation as a problem, 96% accuracy, whatever. But maybe some languages it's not. So the problem will shift as AI practitioners will be chasing unsolved problems all the time.

I think we just need to get out of the way. I think we, we learned so much as soon as we got out of the way and let that machine learning and those algorithms do what they do and what we designed them to do. And we keep getting the way. And we put a number like 97%. Who am I to define that in my own mind? I can't comprehend or, or factualize what the 90% looks like. Or 97%. Right. And you said other languages. We can't assume that it will be that it has. If you get to a certain level, like we say what 85%, we get to 85%, we're super excited and, and then the rest, we kind of manage that 15%, that variable.

Right.

But you have to always view it as a variable, not a static. I reached 97.

You know, just to add on that, for example, self driving car. If you take self driving car, the accuracy of whatever the little functionality within self driving car could be 90 plus in daytime, for example, but in nighttime it may not be. So there will always be challenges in the area.

Yeah, I think something I always tell people as well is to think about these accuracy or other evaluation metrics. It's really on a case by case basis because like you said, Wes, it's not always 97%. If I'm in a actual real world scenario and all we need is 80%, I would get fired if I spent, you know, six months trying to get anywhere past that. Right. So you definitely have to take it case by case, just to add a.

Little bit to that too. I think it's also left to us, the community. AI community is sort of define what a standard is. So there are efforts like MLPUF where folks from Google, Facebook, Amazon, IBM actually come together to define what a set of metric is like, what is the accuracy, what is the throughput and latency. So I think such efforts actually could help propel a standard benchmarking in ML.

Awesome. Well, let's go on to the next one of these. The next one is if I can't afford to buy a GPU like me, maybe for deep learning. What does it mean? Does that mean that I can't do any serious neural network training? What do you guys think? Peter?

So I started the entire AI without a gpu. I actually just got my entire GPU recently. There is a lot of resources online, including Intel's AI Academy where it gives you using CPU Xeon servers that you can train. And also there's a lot of schools that you can leverage their GPU servers, but you have to wait a little bit online that they are generally available, the resources available for you to go train. And for the most part when you're starting with the AI, I don't even recommend you go training. There's thousands and thousands of pre trained models that you can actually start practicing. And a lot of times those models are like some of the models I used right in the beginning. I still have not beaten their benchmark, you know, so. So that's the way I see if you want to get into AI deep learning. Just use pre existing models first and then once you crush that threshold, you have a really, you know, you can go through AI academy and then one Intel's AI academy. Once you go through that, you can, you can get funding. I mean if you're already an AI.

Expert by then, yeah, I agree. And the new development in FPGA and in neural network core, including that core support inside the new CPUs making it a lot different. You'll be able to see a lot of neural network applications existing today on your smartphone. Android and iOS already support SDK is already in there and you're actually training on a small set, of course, small data set. You're training on the CPU doing inverse quickly without worrying about the CPU power. So probably 50, $60 raspberry PI like product will help a lot. And there's a lot out there in the market. Intel Movidius is another good example. It's a neural computer stick that you can actually get for under $70 and using the resources on the AI Academy. Intel AI Academy. You can actually start with a pre trained model as you basically mentioned. But you still don't have to go GPU or gpu unless you already figured out the model size and what you're doing. You're working on a larger scale of data. That's when you need gpu. So it's not ready to start with. It's very advanced stage with AI.

I'd like to add a point there. So we specialize in distributing machine learning load for training. That's what we specialize in. And we're working with intel to team to show actually we have benchmark data that it's already there. But I'm letting the cat out of the bag here a little bit prematurely. We're co writing a paper with intel that one can do training faster than GPU, cheaper than GPU for sure. Just with commodity CPU, that's RocketML intel combination paper is combined coming out. So far there is inferencing and there is training where you consume a lot of the hardware compute resources. So far CPUs outperform GPUs in the inferencing. But that wasn't the case in the training space. But the reason for that was that most of the software people use on top of the hardware is bloated. For example, Apache Spark is great software for some other purpose for training. It's a bloated software comes with a lot of baggage that's not required for machine learning training. So we have built a system that can overcome that barrier and now we are hitting a benchmark below the GPU speeds. So it's much faster than GPU speed. So that's an interesting thing that we found out.

Awesome. Yeah, that's exciting to hear about. I'll look forward to seeing that paper.

Thank you.

Yeah, I appreciate all of your guys perspective on that. I think that a lot of people see that as a barrier when it doesn't have to be. And I think you know, I would emphasize as well that you know, training isn't the only bit of the AI workflow in general. Right. Even if you, you're thinking of training only, there's a lot, there's a lot to learn about the fundamentals about pre processing, about inference like you said Peter. So there's a lot you can learn and we'll post some of those resources in the show notes. Okay, so this next one's kind of interesting. Hear you guys perspective. This question is why is there a sudden craze of programmers with little math background jumping onto machine learning which requires a much different skill set than traditional programming. What do you guys think? I mean, have you seen SDK?

SDK tools? SDK tools that we see see out today made it a lot easier to consume and build and utilize training models compared to start doing everything on your own using Python for example. I mean this. Why? Yeah, I mean if there is a ready to use machine learning model tools that I can go to AWS for example and just pull it and immediately start using it and have prediction ready to go. Yeah, you will see a lot of people without mathematical background trying to get into that. But you will face issues, you'll fail. You'll need that mathematical background.

Okay, so as a hacker, you know I basically build prototypes a lot. So that's one of the trend is that you follow where the entrepreneurs generally follow what the trend is. And the latest trend is basically, you know, blockchain and AI. Blockchain is still really difficult to program but people gets into it because one or two really good applications like Bitcoin is, you know, they see Bitcoin as one of the things that's going to change the world. All the programmers get in to study blockchain and they see self driving car, they start seeing this as being deployed in the real world. They're like, okay, this is a time where you have emerging technology that's being applied in the real world now. And this is a time for me to invest my time to understand and to execute and to find opportunities within this space. This happened back in mobile. When mobile start getting to people's hand, all of a sudden all the programmers start jumping on mobile. Because I jumped in before that and I remember just being one of the independent Android developers, I was invited to give a talk at TED Global that was really rare because people didn't get into it. But as soon as that tipping point has been passed through, we have current applications being deployed in the real world now. That's when the programmers all want to jump in. Then this is where opportunity is.

That's such a great point because if you think about it, you know, if they started from square one when they found out what was hot on the market right now, like blockchain or AI, they would never catch up the ideas or where they think the application would be, would be two years out by the time they caught up in calculus, right? And then understanding derivatives or, you know, that's, that's messy. So what they do is, you're right, they go in and advance, they say, hey, I'm just going to. And then I'll backtrack and then I'll, I'll try to figure out that, that actual knowledge down the road. And to Ali's point, you know, that can be dangerous. Absolutely. So I think to the, I would caution those, don't, don't not innovate and don't not jump into it. Absolutely. Because I will say something that will spawn the rest of your minds here much better than anything I could come up with on my own, regardless of what my mathematical history is. Right. And I think that's what we need to spawn as innovation. But be careful. I totally agree with that. They're, they're following a trend.

Yeah, I think there, there's definitely a balance to, to be had there. I think, you know, we've had guests previously on the show that don't have a math background but are still, you know, innovating in a really great way in AI. And so I think that, yeah, I would love to encourage people to, to get in, get in the field, get their hands dirty like Peter said, get some pre trained models, try out some things. This is a new layer in the software stack that people can experiment with. But then, you know, once you start getting into it more, you're going to naturally be drawn into these ideas around the theory behind neural networks and other things. And you know, you don't have to have a ma. I don't know, do any of you guys have a math PhD or something.

Like this from it?

So I definitely don't. And so I think the math is important like you guys have said and people will get into it. But you don't have to feel like if you don't have a math PhD then you can't get into AI and start innovating.

One quick point I want to make is also a question you have to ask yourself as a developer, where in the software stack do you see yourself? Right. The machine learning software stack is pretty complex and deep. You can look at the analogy of say an assembly language developer or a Java developer. Like where in that stack do you want to be, do you want to be writing drivers? You want to be writing assembly language programming? Because very soon AI is really going to be a box. Essentially you're just going to be talking to APIs. Right. So do you want to be writing applications for this box or you want to actually build the box? Right?

Yeah.

So that's one question the developer has to answer himself.

Great point.

That is so true. Very good point.

Awesome. So this next one is probably somewhat controversial. So I talked actually to Wojciech Zaremba yesterday about general intelligence. What is your guys take on how intelligent is AI now? So the question is how intelligent is AI now? This was in July, but you know, now, now we're a little bit later so maybe it's more intelligent now, but what do you guys think?

Very dumb.

So AI right now as we. Because everything's called AI now because of the, you know, the buzz as explained earlier. But right now as mostly consists of inferencing. That's what I consider deep learning. That's what, you know, that's what spawned the entire industry. It becomes a sensor rather than just, rather than intelligence itself. Right? Because a computer doesn't know what a cat is before 2012. And the whole point that right now computers that this is a cat and this is a dog, that's what intelligence. And this is all it can do. But you know, with all the other programming of human intelligence that we have combined with, you know, what's so called deep learning artificial intelligence, this is where the creating the real intelligences and that can go into some danger field, right? Like if you think about not just what, how it can benefit a human being, but what technology can do, it gets into very, very dark sides of humans as well as dark sides of machine itself. For example, you can easily target people using dropping bombs. That's just something. So as we develop the AI, I think this is the thing we have to consider is that right now it is everything's dictated by human intelligence. So we have to decide whether we want to like build something that can just drop bomb because they sees human. Because at the end of the day, if you train that model just detecting humans, you have to figure, you have to have, you know, you have to have something to back in. So you cannot use this to drop bombs.

Can I summarize your answer? The answer is yes and no. Good and bad. Smart and not so smart, right? And as an example, if I apply AI in a retail architecture and I want to do RFID tracking to push advertisement or just to track foot traffic generally in a store, or motion tracking for you know, for my, my ac, I'm saving power, simple things like that, right? AI is super smart, AI is brilliant and I see an ROI immediately. But if you take it to a more complex model like what you're saying, where it can get really scary, you think about simple things like an autonomous vehicle. Everybody talks about autonomous vehicles, but let's say I'm in a car and let's say I'm in an autonomous vehicle that tracks vitals now because it's self driving, maybe it's tracking vitals. Maybe I have a heart attack. Maybe the car needs to pull over. Does the car understand, which we just taught it, what a road is? Does the car even understand the density of the gravel on the side of the road, how far to pull over, how to look over your shoulder and see how much traffic you're going to cause by pulling over here versus 10ft ahead? It's that consciousness. And so it requires so much more. And so in that instance it's not very smart, not yet. Instead, I would argue that it's not autonomous at all in vehicles, but rather automated, programmed based on a set of parameters, not autonomous at all. I don't think we're there yet.

There are two ways to think about it. One is it's very simple statement is easy, is hard. Hard is easy. What is hard for humans is easy for machines. Like if you want ask someone, data scientists or whatever, take Larry Page also and say that hey, what does the recommendation engine look like? He won't be able to answer that question. But whereas machine learning can do that much better than humans, on the other hand, what's easy for humans, that is cat is a cat, whatever it is, that's a lot easier for humans. But for machines it's harder. But it's getting better and better. Andrew Ng, one of the key guys in the AI segment, we almost worship him, he says that anything, I think he said one second for humans to do an inferencing, this is a badge or whatever, this is a human being, this is a mic. If it takes one second, machine can do it at the moment. But I think that threshold is increasing very rapidly. We increase the compute capacity, data quality and of course there is algorithms, the math behind it. As people invent more new math ways of representing the real world, it's all getting faster, rapidly at an exponential rate. And I think there is some threat because of that. And also at the same time you could solve significant problems, for example in cardiology or in radiology, medical spaces, you can literally make a huge difference to the human beings.

Awesome. Yeah, thanks for your perspective on that. So this one probably builds on that one. And maybe this is a point of confusion for a lot of people. So the question is, is a neural network actually a good model of how the brain works?

So we, I mean if you look at a perceptron as an example, I mean the way your brain actually get signal from sensor in your fingers and the way you react to it. Just building that same model in a neural network, it may look easy and it may work for a lot of application. As we said, we can actually have machine learning as models. But putting these models together, the sensors together, to actually make an intelligence decision based on all these. I don't think this is how the brain work. I mean, the way you may sometimes think about like a smell or you hear a music remind you of something, you don't actually know how that worked. You don't have any idea how it worked with the machine. The consciousness is not there. You don't expect it to be able to do that or come up with new ideas on its own. It's hard.

Just to add on that. I think there is a symbiotic relationship between neuroscience and neural machine neural network. People who are trying to hack Biohack, you know what they think the way the brain works into machines. I've heard neuroscientists say that by reading the papers of mathematicians writing about neural network, they learn a lot about the brain. So there's still a large areas how the brain works we don't understand, even though the neuroscience is improving quite a bit. So this is an interesting, definitely very intriguing area to keep an eye on.

So I just want to add something. I think the neural network is just one part of the brain, right? As a part of the brain, we have a lot of things. We have memory, which computer have, you know, built a hard drive that's specifically just simulating our memory. A neural network is basically just classification object detection, and that is part of that. And as we do more computer science and more AI in the future, we're going to uncover different parts of the brain that's, you know, some of them is going to react to smell that currently where is hardwired, but is human hardwired or is that learned? These are things that we're going to come like is going to be very interesting next few years.

So kind of building on what we were talking about, about what a neural network is, I'll jump into this next question. The question is, in a neural network, each neuron is a hidden layer. In a hidden layer is said to focus on a certain feature. Take an eye for an example. But how does it deduce that an eye is an eye or since it can contain a combination of things like an eyebrow and, you know, other parts of the face and the pupil and all of those things. So any one of you guys want to take a stab at Describing that.

I think it depends on how much data we provide the machines. With enough data, machine is able to parse out those similarities. Everything that machine does is just trying to match a pattern. And every layer helps match what one little pattern at a time. So that's how the neural nets deep neural net keep adding layers and layers. And each layer has quite interesting set of information that you can make use of for some other purpose too.

Yeah, so maybe a follow up on that. I mean there's a lot of talk about deep neural networks. Now is the purpose of all of these hidden layers in the deep neural networks to detect these more complex patterns or why, why have the deep neural networks kind of advanced so far?

It's basically like in your brain, the way your brain have a receptor field. Like when, when you think about the way your brain react to caffeine, for example, or to nicotine, I mean it bind with certain perceptron and neurons in your brain to actually feel that. And now the same thing. I mean with these layers you, you got a lot of filtering, you got a lot of control that happening in there. And part of that is basically having multiple receptor fields. So you're actually going faster because if the receptor field doesn't match, you're skipping, you're actually not going in that neural network completely going through it. And remember, you're doing that thousands and thousands and thousands of times during the training level. So basically adding the receptor fields in there, it changed the way we see it, it changed the way we control it. But it's still, it's the way neural network will work is going to go more about distribution than just as a single node network that work together. You're going to have multiple nodes making decision together. That's just what makes sense basically. I mean distributing the decision making. Instead of deciding on one type of neurons, for example, with certain sigmoid control, you're going to have multiple based on the decision of the first one, which one to trigger, which one to attack. That's how the eye work. Actually. There is multiple layers of filtration that happen and sensitivity that change with the lights around you. We're far away from that. But cameras will have these technologies. Slowly, as we understand more, we'll build more into the neural network.

You know, there's that constant debate if machine learning is actually science or arts, you know, So I think it's really hard for data scientists or non machine learning experts to kind of figure out what layer should I put where, how many neurons should a single layer have? Right But I think efforts like Auto ML are actually trying to solve that problem. So you can install AutoML or such tool and you're ready to go, but time will tell, really. See how far we we get close to getting accurate models based on such tools.

Yeah. And just describe AutoML a little bit because people might not be familiar.

I think Wes was going to say something about it.

I was actually just going to say these questions are so presumptuous. Right. They're again, they're human questions about a non human platform and they're presumptuous. Right. Peter.

Right.

You mentioned about how neurons are ali perceptrons and how those impacts the whole model. Right. And now we're only focusing on maybe two elements of our brain storage. Right. And potentially some reasoning. Right? Yes. And decision making. But the fact is maybe AI will evolve even past that to where it will define that maybe our brain isn't connected the way it's most optimal in a compute environment. And so the questions are presumptuous. So when you answer some of these questions we have to answer based on unknowns. The easiest analogy I like to use is when you say, oh, that planet can't support life. Who are you to define what life is and what another life form requires to exist? Right. How would you even proceed to know that? Right. And so some of these questions, they have to be open ended and you have to keep your mind open to what is a much larger picture than what we can even think can exist.

I'm glad that we've covered the full gamut between neural networks and aliens. So I'm happy that we made it there.

Did that for you.

So yeah, just briefly before we go on, I'll circle back to Dave and let him mention what automl is so people can.

Yeah. So typically when you train a machine learning model, you're tuning a bunch of parameters or hyper parameters like batch size, learning rate, iteration and whatnot. So automl lets you sort of tune those knobs here and there and then gives you a way to accelerate the process. Essentially the other tools to like AutoML. But that's one of my favorite awesome.

Just to add on that great models are built through iterations. Iterations are very tedious and taxing for data scientists to do it. AutoML reduces that extra effort quite a bit. It's a machine learning on machine learning makes sense.

And we'll put some links in the show notes of course to some of those resources. So the next question, if deep learning is the future, which maybe everyone says it is, what is the need for machine learning methods like svm, decision trees, Bayesian methods, Markov chains, et cetera?

I mean, each accelerate in a different field. I mean, deep learning. Deep learning will not work for a lot of mathematical equations that we deal with in the network workfield today because the data type, the way it behaves, it changed with a lot of factors that is not fitted in a table. I cannot just fit it in a table and have parameters define it. So there is a lot of fields in AI where deep learning may be good to actually predict a model based on tree size, information that you can fit easily in tables. And also like pictures. If you look at pictures, it will take in pictures and actually turning that into a model that's basically looking at the pixels inside. It's the same idea. So yeah, the other machine learning models still exist and it actually a lot faster in performance and in behavior, especially supervised behavior when you control that. Deep learning doesn't have all of that built into it, but again it accelerate into pattern recognition a lot, a lot faster.

Yeah, not everything is a nail. You need a hammer for a nail, but if you have a screw, you need a screwdriver. All methods have value and a good fit in different areas. Deep learning turns out to be quite versatile, but it has its own weaknesses too.

Yeah, on that note, for people kind of getting into the field, they probably hear a lot about deep learning, but there are so many methods to learn about. What would you guys recommend as far as a person getting into the field? What sorts of things might they want to learn about before deep learning? And how quickly should they kind of make that leap to learning about these deep neural networks?

I mean, you start with machine learning basics before you go to deep learning. What is machine learning? What are you trying to achieve? And you understand basically how you're taking parameters, what training mean. You actually get familiar with the framework. So there's a lot of frameworks you will get introduced to. Each of them got a powerful side that you can use in certain systems. Yet before all of this, in my opinion, distributed systems, you need to be able to, you know, run around, know your way around as a distributed system engineer, because AI need a lot of that. You're going to have more models sitting in the cloud, you're going to have inverse happening on the edge. You're going to have training maybe happening on the edge in some cases.

So to get into this, I would recommend somebody just first, this is how I got into it. Just basically first figure out what is a cat and what is Dog. I'm sure all of us have went through that once you go through that because it's the goal that's pushing you to keep learning. First, how do I, I determine elephant from dog and cat. And from there I was like, okay, I need photos of elephant. So I have to basically google a bunch of photos of elephant. I was like, okay. From there I basically said, okay, what framework to use? And I actually select a framework. I select a couple of them. I'm not going to say which one. I basically choose the easiest route that I was able to train my model. And this is. But this is. It's a limitation of a person's imagination is what will keep their curiosity going. We'll keep them googling for like, you know, stack overflow for informations and when they get stuck. And then each step you learn a little more and eventually pretty much how your own way would work. That's a good way for learning. Get into deep learning.

That's a very good advice. I agree with them that it's a complex topic. Math involved, there is domain knowledge involved and there is ability to define a good problem is very critical. And then you have the tools like TensorFlow1day, PyTorch the other day. There is so much I would agree with Peter in saying that. Just get in there. Start with a simple problem. You don't have to worry about defining a problem and you're trying to make the machine identify cat versus dog. So you kind of minimize the buttons you have to push and start small. And as you gain confidence and curiosity kicks in and your goals change and you learn new methods, new complicated approaches, that's the best way to do it.

The other quick thing I want to add is it also depends on how deep you want to go. So if you're in the academia, then you probably want to do statistics and probability, or if you're not, if you're in the industry, you probably want to build prototypes as fast as possible. Then you want to use frameworks like TensorFlow, Keras et cetera especially is one of my favorites. It's very easy to go four lines of code, you have your model trained and then it's doing inference. So it really depends on the user, the developer, where you see yourself being in that machine learning stack.

Awesome. Yeah. One of the things that we've emphasized before is kind of in line with what you guys are saying. Find a problem that you're passionate about, find some data that you're interested in and just try to start answering those questions and that will Lead you to the, the right frameworks and the right methods and the right things to learn about.

It's a great point. Like, just start where you want to start, find interest in it. Because if you're trying to do static tasks A, B and C to get to the end result, I think you're following a set of steps. I think you have to have a passion about what it is, what do you want to solve for, what do you want to do, and then the excitement and the motivation behind that pushes you further. But I think, yeah, as soon as, if you're just trying to give someone a guideline, do this, do that, do this, do that, that's boring. That sends me back to school. And you just had to get your homework done. That's not fun.

Right.

But I think everybody at this table is super excited about what we do and we follow our own path. That's why we're all doing different things.

Right.

So awesome. So I don't know if all of you guys primarily work in Python. I think we could extend this next question to whatever language you guys work in, but apparently the Internet wants to know what is the coolest thing that you've done with Python? Slash, whatever language you're you're interested in. So maybe if you guys just want to highlight one of the cool things that you've done to give a little bit of inspiration to the audience.

So my kind of projects, just going back again to what excites me, it's FPGA. So field programmable gate arrays. So FPGAs are pretty low level and they don't really operate at a level of Python. But as you know, the frameworks operate at a level of Python. So what I typically would do is create wrappers around the FPGA low level APIs. Right. So I would want work with C C and then create wrappers for Python so you can easily plug in the device to such frameworks.

Awesome. Are any of those kind of wrappers and that tooling, is that available to the public?

Typically, no, because those are very low level driver API stuff. Right, Exactly. I don't even think the users would be interested in knowing what goes on at that level. Right. So Python is the way to go, I would say.

Awesome.

Yeah.

And if is any of that stuff that you've done public to where people.

Could take a look or currently not public.

Awesome. Where I know that intel, for example, is putting out a lot around FPGAs. Are there resources out there or webinars or anything like that?

So yes. Intel has done quite a lot in terms of enabling FPGAs for data scientists and machine learning. There's actually a tool called dbsdk. It was recently released not too long ago. So cbsdk would allow you to take your trained model and run it through what is called a model optimizer. It's actually a Python script and then from that you would get an intermediate representative which you can then use to port against pretty much any hardware that intel makes today. Movidias fpga CPUs. So OpenVino used to be OpenVino is it OpenCV, SDK, OpenSK.

It's the same thing, it's integrated together.

Right, Right. So cbsdk is what it's mostly called today. That's the. A tool that you can use if you're using Intel's hardware. It's pretty easy to use. It's Python based.

Awesome. Yeah, I appreciate that. I definitely want to dive in more myself on that.

Yeah. And the most exciting thing, I mean FPGA is a very exciting thing for me right now. But I believe Python control of SDR and FPGA together, that's what's going to change the way we push AI to the edge. Because imagine, I mean you don't have the limit of a certain technology that already sit on this device just because the technology changed in a few weeks or we found a silicon level issue with the design itself. Since fpga it's easy to rearrange. It's just a firmware that you can actually push make it easier and make it a lot easier to control with SDR also pushing SDR which is software defined radio basically that going to make 5G technology, LTE technology and IoT technologies without being stuck in a limit. And imagine if you actually let the machine decide the control of the waves and the control of which which band to actually use to make sure like a swarm of drones for example, stay always connected. So AI going to get pushed to that. And I believe fpga, I mean as an industry, we've been in the network network industry for a while. It started with FPGA especially on the, on the network side because it's easier than building a complete ASIC go to the market. But with FPGA pricing going extremely down in the last five, six years until introduction of a new type of FPGAs that actually built for intermediate devices. It's not just basic input and output features. You have more cores from the technology that's available to you. I believe for me, seeing Python being able to actually utilize to control SDR and fpga and we did some research on that on the edge of the network, it's extremely amazing. It can push AI capabilities a lot more to the edge and use more sensors capabilities. Utilizing that FPGA and sensor fusion, you don't need a lot of sensors. You can use less channels by fusing your data to get together and having like, we refer to it as the edge of the network, which is, we refer to it as Mac today, multi access, edge, compute. But we also believe there's another layer. So it's not everything going to be in the cloud, it's in the edge. But also furthermore on the device itself there will be some decision making happening there to eliminate the amount of bandwidth we use. Otherwise these all connected devices will send so much data, no matter what technology we have, it's not going to be enough to process it.

Awesome. Well, any other thoughts around interesting things you've done with Python or other languages that you want to highlight?

We work at the infrastructure level, just bare metal up, silicon up level. So we kind of try to be hardware agnostic, be it CPUs or FPGAs, whatever it is. So we work C level, not so much the Python.

Yeah, well, what have you been doing recently at the C layer? And also maybe I'd love to hear you speak towards the role of C and C in ML and AI. A lot of people see Python as kind of the only player, but I think that's kind of a facade in some ways. So maybe you could speak to that.

At the infrastructure level you need compiler language like C. And then at the user level we put a wrapper and allow people to use our product with Python interface, the Jupyter notebook, they can call our libraries and make use of it. So all languages have value there. But I think a lot of the models will get built in Python only, that's the fact. But when it comes to distributing, to consume compute, you need something else underneath.

Gotcha. So kind of on that, on that note, you know, maybe the last thing that we can end with is there's a lot of people kind of getting into, like you said, training with Python and maybe they're hitting some blockers as they're trying to scale things, or they're trying to build up an AI team and maybe not knowing how to productionize models and that sort of thing. What general recommendations would you guys have around kind of the team that you build up around that you're trying to put together to build AI? Models and kind of some of the tooling that you might need to consider, the methodologies that you might need to consider as you're actually trying to scale AI.

I would add by saying it depends on if you want to be a machine learning developer or an infrastructure engineer or you want to be both, which is great because I'm sort of somewhere in between a little bit of this, a little bit of that. Tools that I would recommend are Dockers, maybe Kubernetes to then manage docker containers that are running these ML models across multiple nodes. But again, like I said, it really depends on what you really want to do.

On that note, do you think like as far as teams that people are trying to build up, maybe you're trying to start an AI effort in your company, do you need to kind of have both of these components or you do, do you think there's people out there that can fulfill both of the roles themselves?

Right. So it turns out they're actually really nice devop tools you could use. I would say maybe just a team of one person should be able to manage the infrastructure that runs these ML models. And really you can deploy Docker containers and Kubernetes mesos, for example, to kind of manage these models. It's a different thing. If you're running it at scale, then you may need a lot large engineering team to kind of make sure that business is running up and going. Yeah.

The most time consuming part about building an AI great AI product isn't necessarily the deployment piece. It's thinking right about the domain, about the problem you're trying to solve. Especially if the team is being built in a bigger company. The problem definition itself requires a lot of multiple levels of agreements and consensus. That is a challenging thing. So you need people who can build consensus, you need people who can do critical thinking, and then you need great data scientists who can translate those problem statements into some kind of a model. Okay, it's a classification problem or a clustering problem, whatever, somebody who is familiar with that kind of semantics, be able to translate that and hack that together and experiment, put out a prototype, you should be able to get to a prototype really fast. That should be the goal. And that requires a multidisciplinary team.

I think that's a good point. I like to say you need, you need translators, you need people, because the folks that are doing the dirty work and getting their hands dirty and the programmers and developers, those aren't the people with the cash a lot of times. And those aren't the people making the business decisions. And so you kind of need someone in between that's savvy enough, enough to get the idea, be able to communicate with those that are really putting the rubbers of the road, but also communicate what the why is. Right, Vinay? What the end result is going to be and even bigger, what's, what's the roi? What's the return on what, what, how many sales am I going to get? I mean, you almost have to dumb it down and it's almost insulting. That's why, you know, you hear that the joking tagline, don't let the engineer make the product. Right? Because you'll keep fixing it, you'll keep getting better and better. And that doesn't scale for, for enterprises or for a product to go to market.

Right.

And so while we constantly want to do better things and we can never deliver something because it's never done, it will always be better. I think you got to have that buffer, someone in the middle that can translate from the tech to the business side. Because if you don't have that, there's a huge gap between the business guy and the people like folks at the same. Like you guys are actually doing it, right?

Awesome. Well, just to kind of end things here, I want to thank you guys for taking, taking time out of the conference. I know there's a lot of great talks going on, so thank you guys for taking time to talk. It was really some great perspective and, and thank you to Intel AI for, for helping arrange, arrange this panel. I really appreciate it and thank you guys so much.

Thanks.

Thank you, Daniel. Thank you for having us.

All right, thank you for tuning into this episode of Practically AI. If you enjoyed this show, do us a favor. Go on itunes, give us a rating. Go in your podcast app and find favorite it. If you are on Twitter or social network, share a link with a friend. Whatever you gotta do. Share the show with a friend if you enjoyed it. And bandwidth for Changelog is provided by fastly. Learn more@fastly.com and we catch our airs before our users do here at Changelog because of rollbar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers. Head to linode.com changelog check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder and you can find more shows just like this@changelog.com when you go there, pop in your email address. Get our weekly email keeping you up to date. With the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.