#115 Inside the Generative AI Revolution

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI Transformation Journey on datacamp.com.

You're listening to Data Framed, a podcast by Data Camp. In this show, you'll hear all the latest trends and insights in data science. Whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization. Join us for in depth discussion with data and analytics leaders at the forefront of the data revolution. Let's dive right in.

Hello everyone, this is Dal, Data Science educator and Evangelist at datacamp. If you've been following the data space, I think we can all agree that generative AI has been absolutely exploding over the past two years. From the arrival of text generation models like GPT3 to the proliferation of AI image generation tools and the research space showing us what's possible with modalities like audio and video, we seem to be on the precipice of something truly special. There is arguably no better person to talk to about this than Martin Musial. Martin is a Data Science Manager at IBM and an instructor@generativeai.net where he shares a wide range of learning material related to generative AI. Throughout the episode we talked about where we are today in generative AI. Some of the main use cases we've seen emerge today, what successful business models may look like for generative AI, privacy and copyright concerns with generative AI and what generative AI will look like in the future, and much more. If you enjoyed this episode, make sure to rate and subscribe to the show. And now onto today's episode.

Martin, it's great to have you on the show.

Thank you for having me.

I'm very excited to chat with you about generative AI, your work and courses around it. The type of use case we can expect to see in the near and long term future with this type of technology. But maybe before give us a bit of a background about yourself.

I am from Munich, Germany. Originally from Northern Germany, I moved to Munich because of the University Technical University of Munich. The steps leading me to where I am right now. So after I studied computational science and engineering at the Technical University of Munich, I went to Airbus, the airplane manufacturer, researched on predictive maintenance there. Next was a joint frog design as a data scientist and ultimately then I joined to IBM first as a data scientist, took slowly over some projects and now I'm a data science manager leading various kinds of projects with clients from different sectors and also leading a team of data scientists. The topic that we are talking about started to be very relevant for me around 1st 2014 when Ian Goodfella came up with a vanilla gun structure. And a year later I then started talking about it on conferences Europe wide. And I saw sort of like blurry, but also in a clear vision the impact of generative AI. This led me in doing and creating an online course, having a newsletter, giving various kinds of impulse sessions, some kind of workshops in other companies and here we are.

That's really great. And definitely you mentioned here you hit the nail on the head with generative AI. This is a space that has been very exciting to see especially over the past two years. You know this space I've been following quite a bit over the past few years and this space is very exciting. You know, for image data. You mentioned here it's been eight or nine years almost since the concept of GANs or generative adversarial networks were introduced by Ian Goodfellow. For Dex data, it's been like almost five years since the transformer was introduced. And if you start counting from 2014, which is when GANS were first introduced, I think in less than a decade we've gone from relatively brittle tools that create relatively low quality outputs that need to be trained on specific data sets, to universal models like Dall E2 and GPT3 that can really do awe inspiring image and text generation tasks. Now we're seeing the same being applied to video and audio as well, which is going to be something that's going to be very exciting in the next two years. So I think this really begs me to ask where are we today in the generative AI revolution and where will we be in the next eight or nine years?

That's a good question. So looking at where we are today, I think also that the evolution of generative AI is quite strong, but I think we are still very much in the beginning. For instance, if you look at image generation created a lot of attention and many very jaw dropping results have been produced. I still think even there is a lot to improve so we can still observe some kind of artifacts Looking at the whole canvas, not always it is being executed like good enough. I think there is like still quite some potential, especially like also looking at these top models like from Dall e, midjourney party AI etc. There are plenty of them. And also looking at NLP applications now OpenAI has launched whisper which is really really good translating languages or understanding languages what has been said. But even there I see I quite quickly could identify errors occurring when it's for instance come to a switch off of a language etc. And I think this will iteratively improving. So looking at the next eight to nine years, first off, I would like to say that just recently, just before our interview, I read that Gartner has estimated that by 2030 that synthetic data will completely overshadow the amount of real data by AI generated data. And so further applications that I'm seeing in the future is text to video which already Meta tried it out and I think Google also gave it a chance. But of course there is like lots of more potential, longer videos, higher quality, maybe more interactive. Other applications I see is Improved versions of 3D object generation for various kinds of product development. Could be architecture, could be interior design, etc. And ultimately also virtual worlds. Yeah like the Metaverse are big steps. And all of these virtual worlds we don't want to make everything imagine manually, we want to generate them and or have a generative AI supported. On the application side I think also there are two more aspects actually I think so. The one aspect I see is more on the services side for companies because the one things are encapsulated applications, but then also having it a bit more productionalized in the industry. And I think at the moment, generally speaking, I don't think that companies have integrated generative AI much into their existing services or have created much new services. And many companies are frankly not even aware of generative AI. And looking at all of these applications I see like in law, healthcare, banking, marketing, education, I see an innumerable amount of possible applications going to like simplifying contracts or image generation for maybe some customized product packaging, etc. On the service side. And then also I think data sets can be improved a lot. Enhancing data sets in general as a next step.

That's very exciting and I'm very excited to deep dive with you on the use cases and kind of flesh them out even more. But maybe let's start with the fundamentals first, just for the audience. Walk us through the definitions of generative AI and how it differs maybe from traditional machine Learning.

So generative AI, I think I'd say its main task is to generate data, all kinds of data. Yeah, it could be image data, it could be video data, it could be text data, time series, 3D objects, etc. So literally whenever we have some kind of data, the generation of it is I would say in the realm of generative AI, I would then also say there are like three main tasks. So the main task is generating data, then transforming data from one style into another, maybe, or domain transfer. And then thirdly, I say also data enhancement is an important task and it differs to traditional machine learning. It generates. And traditional machine learning is more about discriminating data, classifying or some kind of regressions, dimensionality reductions, reinforcement learning, these are all. Yeah, it's more about discriminating or choosing, deciding on the problem at hand versus generating a data here.

Okay, that's really great. And kind of, you know, looking back at GPT3 and Dall E2, I know that these are completely different solutions or like two different models completely. But looking at the technological trends that have shaped the explosion of generative AI in the past few years, what have been these technological trends and what has been underpinning them?

Okay, that is a great question. So you mentioned just now, Ian Goodfeller, 2014. But before that there were already a couple of technological trends leading up to it. From my point of view, it started with autoencoders where you can insert data, for instance an image, it gets compressed into a low dimensional representation called the hidden, the latent space and then decoded again into a reconstruction of the input image. And this has to be learned as well. So this is actually more like a reconstruction and not true generative capabilities. It has a generative nature. Yeah, but it doesn't truly generate data because what is missing is in between this interpretation, between the seen data before and here is where then variational autoencoders. This by the way also I use in my research at Airbus, with predictive maintenance. We were aiming to generate data points because we didn't have enough, especially when it came to turbulences, etc. We use a variational autoencoder that has this model first time has these true generative capabilities into this latent space, similar like an autoencoder. It introduces some kind of an underlying distribution, so it interpolates between the data points of the kind of different data that it has seen and is able to generate data also mixtures in between. That was quite exciting to see that. And then of course I think the biggest technological Impact was what we have mentioned already. GANS generated generative adversarial networks which are two or multiple networks basically learning adversarial to each other against each other. While one network is trying to generate data, the other network is trying to detect if it's a real or fake. In this fashion they're both getting better. Throughout the learning, throughout the training period, these models have shown great, great results. It was probably the biggest step. And then of course Ian Goodfella just introduced the first vanilla gan. Yeah, the blueprint of it. And multiple variations of gans have been developed. Big GANS cycle, gans, B Partit, Gans, Wasserstein Gans. There are innumerable, I think if you go on arxiv.org for all these submitted papers there you can already see like I think something in the 8,000, 9,000 papers taking various kinds of GANs into account and come up with their own variations. So there's a lot of attention, research attention also happening there. And then you also mentioned diffusion models which sort of I see as a next step in terms of image generation where they introduce noise to an image in the learning phase and then reduce the noise step by step and generate out of this at the end or in the middle there's white noise and they generate then the really sharp good pictures computationally sort of also leaner. And lastly I think an important trend also introduced especially in natural language processing kind of applications. So text of voice sequential data in general are the transformers. Where we have for instance the GPT series from OpenAI there is like GPT I think 1, 2, 3 and they're working on the fourth version, GPT standing for generative pre trained transformer where they also have an encoder and a decoder. It's trained semi supervised, meaning it has first unsupervised, so it doesn't have any labels as it's training. So it's the models are supposed to understand the problem at hand and try to solve it without labels. And then it's fine tuned with labeled data which is also quite powerful and it has this attention mechanism to ultimately lift recurrent neural networks, etc. So this is huge algorithmic steps happening here. But also computationally we have a huge leapfrog. Companies like OpenAI, they are backed by Microsoft, they have almost like unlimited amount of computational power and data storage is getting cheap. And yeah, I think all of this comes together into a great time. And that's why it's almost impossible to keep up with all of the trends and technologies happening in Generative AI and achievements. It's just a good time right now.

It's definitely a good time. It's crazy seeing just how fast the state of research is going in this space and just how quickly we're moving through different modalities. Just like two years ago GPT3 came in, everyone thought, okay, we're hitting a next peak in text generation. Then Dall E2 came in, we're hitting a next peak in image generation. And now with Meta and Google releasing vex to video models, it seems like we're heading to the same space in just a few months, maybe if not just a year or two in video. So now that we have covered the technology trends that have shape the generative AI revolution, I'm excited to talk to you also about the actual implementation and operationalization of this technology in the wild. We've definitely mentioned a lot of use cases at the beginning. Whether, you know, image generation for creative teams, contract generation for procurement teams, for example. What do you think are going to be the most impactful use cases that we're going to see from generative AI in the short term?

Okay, so the most impactful use case of generative ies, what just recently happened was AlphaFold. I'm not sure how much you are in the picture of that, but it is basically it's along this alpha series of Google DeepMind where they have first came up with the AlphaGo beating Sedol this like sort of like the world champion and then there was other alpha models and now or Alpha fold it's not the end because then they also came to Alpha Tensor but now especially Alpha fold is interesting because what Alpha folds aimed to do or has aimed to do is to unfold protein structures. Yeah, there's a huge, huge database of proteins and what research scientists have issues or like had big problems with this finding the right folding of these proteins and alphafold within a very short period of time, I think like a year since it started, has basically almost unfolded all of the proteins that are known to us so far in the universe. It's very, very impressive and have open sourced it for everyone to use for every researcher.

Maybe just as an aside here From a generative AI use case, AlphaFold model generates protein structures and by doing so has been able to uncover a lot of protein structures for molecules. Is that correct?

That is correct exactly. So I think this is when it comes to impact, this is a huge impact. It's a gift to humanity. As I heard in this space, I'm not an expert in this field So I cannot talk in great depth there. But other things that I have seen just recently is. And also we see a productionalization here of Google Pixel. They have included this translator mode. Just a friend has shown it to me how great it works. Turn on this translator mode and you can literally in time talk in different languages to people. It then switches the side and it waits for the answer and then it switches back. And so this is. I think it tears down all the language barriers. Feels like it just literally the next step would be some kind of a Babelfish like from this movie, the hitchhiker's guide through the galaxy where they stick the. The fish in there in their ear and then they understand basically every galactical language. Yeah. So. And I think we are not that far away from that. Yeah, why not? Having like some kind of like AirPod device that listens in and translates it into your ear. And I think it's. We are really not that far away from that. And I think that would be a great device. Then one other thing that has a lot of impact, at least I think that. And also what I'm observing is applications from GitHub Copilot. When you start coding, you start with a certain command. Yeah. With a hashtag and then you write the command that you are going to code. But as you're writing the command already, it suggests you the right piece of code. And for administrative code, more or less, or some functions that you want to include that are more or less standard, it's getting actually quite good. And this administrative coding time gets close to zero. You just then accept what is providing to you, or you continue with a command and then it provides you maybe a different piece of code and you can implement that. And this is. Has also I think a great impact. It reduces development time significantly. And also on this note, I would like to say I think that prompt engineering in general will be a very important skill in the future. We see it, we have talked about it, talking about Dall E. The prompt dictates the quality of the image. And prompt engineering itself becomes like this important skill to that to know how the model reacts to what I'm giving, what I'm putting in. And it's similar with Google Copilot in the way I'm writing the command that's also dictating the quality of what's coming out as a suggestion for code. So all of us developers, I think have to then take this into account in the future. These three things I think are just on the top of my head. There's a plethora of different applications that have a great impact, but these are for me quite substantial.

That's great. And especially co generation and co generation is creating these tools that remove a lot of the boilerplate task from creative work or technical work I think are going to be very useful. I think one thing that's difficult for me to square is maybe the business model of generative AI and how it will, how will we see it play out? Because in this space there's a lot of research, a lot of hype as well. It's difficult a bit for me to reconcile between the hype and sustainable business models for generative AI solutions, especially when it comes to stuff like image generation.

Right.

For example, looking at the parallel over the past 20 years when social networks were invented, it took a while for social media companies to really understand how to monetize, best grow and scale their services. And you know, this is regardless of the moral implications of the business, business models of social platforms. I think there's a lot to criticize there. I think we're going to experience similar growing pains for companies looking to monetize generative AI technologies. So what do you think the path will look like and what are viable business models for generative AI solutions?

So I said in the beginning that I think we are still quite in the beginning. I think a lot of fields are opening up. They haven't even opened up and I think there's going to be a lot of potential to tap into. For me the most dominant idea is it's a two parter. So maybe first talking about my current position at IBM, I didn't mention that yet, but I'm a member of the Technical Expert Council. And the Technical Expert Council at IBM is like a global tech council. Expert Council, yeah, as its name says, that tries to understand what are the new trends, the new technologies on the horizon that could be lifted, shifted into the industry and could be used. One of the things that we have also been approached with is some kind of personalized packaging. It doesn't have to stay with packaging, but this personalized approach I think is quite interesting. So with the personalized packaging, the idea was that I cannot name any names but that people could upload an image that they liked a lot. Yeah. Maybe of their family, of their kids, no harmful content that needs to be detected. So it's also a bit, there are some challenges around that but. And then they could choose how they want to transform this image. So here we talk about the second task of generative eye transforming style Transform this could be into maybe some kind of Halloween picture or maybe like Christmas or Easter, et cetera. And then transforming this image accordingly, making it funny, suggesting maybe multiple alternatives. They choose one and then this gets printed on some kind of package of a product and gives it a personalized touch. I think this is how companies can drive a bit more engagement in the future. On one hand. On the other hand, what I see a lot is data generation. I said that as well. That why not having some kind of data generation as a service. I see also with our clients that many clients are missing data. If they are missing data on one hand because it's messy, they are just scraping it somehow together or they haven't thought about a proper data strategy from the beginning in. After all, the problem at hand is just. It's a very rare problem. Yeah. And they just don't have the data. And think about some rare diseases in medical images that we have. Privacy issues. We have. Yeah. Just it's a rare disease, so it's just not existing that often. If there would be some kind of a surface that takes into account images or like the original images as much as possible and then generates just more similar data points there, I think that could be very valuable. It's also not very easy because this needs to hit a certain quality standard. If the generated images are not or generated data points are not good enough, then the machine learning pipeline that is then plugged at the end for detecting the rare disease, cancer, etc. Is then actually getting worse. It's not getting better. So there needs to be a lot of engineering around that. But this is one use case that I'm seeing. And then of course regarding the metaverse generation of various kinds of worlds, lastly, I would like to say Nvidia is working on this Omniverse or they are building this platform for virtual collaboration and they take many like small features into account. For instance, small things like I think it's called a Maxine, that product where when having a face to face meeting through like a virtual meeting, then they make sure that you have eye contact at any time. So even though you're not, you're looking on your screen, your eyes are still adjusted in a way that it looks like you have eye contact and the call itself feels much more immersive. Yeah. These are the things on top of.

My head that's really great. And given how this aspect of AI is evolving, I do imagine a key determinant of how the business models will play out and how the solutions that we'll work with in the future will play out is whether the technology will remain closed source or open source. Right. Looking at GPT3 Dall E2, they're all close source solutions, right? You need to essentially pay for an API with OpenAI to be able to use them regularly. But we've seen tons of movement in the open source community with tools such as stable diffusion, midjourney and stuff like that. How do you see this kind of dichotomy playing out in the future? And which model do you think will most likely succeed in being able to create becoming maybe the de facto solution for generative AI in the future?

I think it's not that one will win over the other when it comes to solving some kind of a specific task and companies want to maybe monetize on something specific that they have solved. I think they won't open source it, or maybe they won't. What I'm seeing also is like for instance, OpenAI has, they have open sourced whisper their model for the translation of the data or for the understanding of its automatic speech recognition ASR. And they have open sourced the code they have open sourced, the model can go to the GitHub and etc. But what they haven't open sourced is the data actually on what they have trained it on. And this is quite interesting because to understand there are these laws from Google DeepMind, these are called the chinchilla laws. And these chinchilla laws, when it comes to natural language processing models, we have to also a little bit distinguish between these different subfields within generative AI. Now I'm talking about the generative or the large language models within the NLP space. And these large language models, one trend that we see is that they have more and more trainable parameters. Here we talk half a billion, then they have multiple billion and the trend just goes up and up. Because I mentioned this also earlier that the computational power is almost unlimited spaces, the algorithms are getting better. But what sort of is a problem? This is now what the chinchilla laws are saying is that in order to train these billions of trainable parameters, you need to have the right amount of data. And actually already GPT3 is under trained regarding these laws chinchilla laws. So now that is at least the rumor that OpenAI is now GPT4, that they have postponed it a bit because they are just missing the data for training the compute optimal the GPT4 model. And so actually that makes a lot of sense to then have this whisper data because there is already a whisper for YouTube that takes all the YouTube videos into account and translates that into text. Apparently this is supposed to make up with the lacking data that they have. What? Also we need to take into account that there are other bottlenecks that are more important than just the open sourcing or closed sourcing of algorithms. However, to come back to your question, sorry for the digression, but I think it's a quite an interesting relationship here. But to come back to your question, I think that these like grand breakthroughs like diffusion models, I can see these being open sourced. And at the end, the implementation as well as the right kind of training on the right data is then the determining factor regarding the quality at the end.

That's perfect. So there's not necessarily going to be a clear winner in mind. There's going to be a lot of solutions that are both closed source and open source and we'll see who kind of competes out at the end.

Yeah, I don't think it will converge just to, you know, open sourcing everything or close sourcing everything. I don't believe this will happen.

That's great. So we talked about kind of the applications, we talked about the technological innovation, the underpinning, this application. Maybe let's talk about the risks and the complications when speaking about generative AI. Another dimension to generative AI technology that adds a lot of complication to the business model is what to do with copyrights and attribution. Think about AI generated art here in the style of a artist that is currently alive. Right. Who is the artist here who takes credit? How do we think. I know this is. Maybe there's no clear answer to this as of today, but what is this the overall trajectory or the thought process around adapting copyright laws to this technology?

That is a good question, also widely discussed in the respective communities. However, I think if there is a company that has built a model that can generate data, so there are different kinds of angles to this, but when a company decides to sell the service, you can prompt some kind of an image, write down what kind of image you want to have in a Funko style, and then it gives it back to you and you purchase this image. Then I think this is your image, even with the, with this credit thing. So the first couple of like 10 prompts is. Or generations is free and then you maybe have to pay. So I think then once you have generated an image with their model, it is your image. This is at least my point of view. Yeah. If that model has trained on an artist that is alive and claims the copyrights here, this is not so easy, because apparently that model then has trained all of these images from that specific artist. And how did the company that develops this model has actually got the data, got the images? The right way would be that they have purchased these images because once they have purchased it, they could use it for the training. Otherwise there are many, many questions in detail, because in the detail there's a devil. Because if the artist then shows his images but then says please don't download them, I think it's sort of like at least a gray zone, if not an illegal act to train on these data points.

There is no easy answer.

There is no easy answer. I'm getting stuck in the details. But I think really my point to this, that it can be solved out if we have like a clear standpoint from the end to end chain. So from the artist itself, if he wants to open sources or make it accessible of his images, you know, then you can't complain that they are being used for training. I think everyone has to be clear about what they want to have at the end. What's the standpoint on their usage of the data?

It's definitely a social and legal pickle that I think a lot of experts in the community is going to have to grapple with in the future. Carrying on, on the complications of generative AI. Maybe another component here is also AI safety. This is something, you know, AI safety, whether generative or non generative techniques, something that needs to be thought of consistently. A key harmful aspect or maybe an aspect of generative AI is the fact that it can have harmful impacts such as bias, perpetuation. It can be leveraged to create very convincing fake news. It can be reused as well to be leveraged to use reputation attacks and reputation assassinations, let's say, through generative technologies. What do you think are some of the ways the community now is approaching how to build safe, responsible generative AI solutions?

So first of all, I think that it should be illegal to harm the reputations of people like with these deep fakes with photographic images. This should be. And it is. I know in California it is already illegal to do that. Especially there is one law regarding Californian politicians that they are not allowed to do that with them. This is the first thing like it should be illegal and if it's detected, there should be consequences to that. Now the question is like how to detect fake news or deepfakes. And there are also technological answers to that on the text level as well as the image level. So in the same way a deep fake is being produced In a reversed way you can classify if a deep fake is actually a deepfake or not. And if something is a deep fake then it's not always easy to answer like who has created it. Sometimes it's actually almost impossible. But I think where it is possible there should be consequences regarding that.

How are companies creating generative AI solutions approaching building safe and responsible generative AI?

I was mentioning this application where we can have personalized packaging. People upload their image and then it gets personalized a package. So one important piece was to detect what kind of image they are uploading. We have a policy where in the policy is okay, no violent images, no right wing or hardcore left wing or some kinds of images. Yeah, in these directions. And I think there is a very important piece that needs to then detect if an image complies to this policy or not. And so it's an image detection element. So what we have done is that we have flagged the images accordingly and then forwarded it to a human to decide. These image detection models detecting if it's policy complying or not should be very sensitive. It's better to take out more images that are maybe okay rather than having the thresholds adjusted in a way that you have. Harmful images are going into the pipeline and maybe being produced and not recognized along the way.

Yeah, definitely. And that's something that we've seen as well with Dalle 2 for example where it's pretty easy to trigger their content filter which is great to see. And this is something that we're definitely going to see moving forward in a lot of these solutions as a potential model. So Martin, as we close out our chat here, we definitely learned a lot on generative AI and where generative AI is headed. Where can people follow your work for more insights on generative AI?

So I have multiple points. So one point is generativeai.net go there, take the online course that we have built, which is an online course, get a solid basis about generative AI. It also touches upon various application fields. I also have there a newsletter where I'm writing biweekly about various kinds of topics that are hot in the moment. Currently I'm writing about text to video generation in this newspaper. Newsletters I always like go into like first of all what is the tech perspective and then what, where could this lead us in the future?

Okay, perfect. And now as we close up, do you have any final call to action before we wrap up today's episode?

First of all, it was a pleasure to talk to you Adel. Thank you very much for that. And so if you listen to this and you want to get in contact with me, want to Discuss various generative AI topics, then please reach out on LinkedIn. Martin Musiol yeah, let's have a talk.

That's awesome. Thank you so much, Martin, for coming on Data Framed.

Thank you so much.

You've been listening to Data Framed, a podcast by Data Camp. Keep connected with us by subscribing to the show in your favorite podcast player. Please give us a rating, leave a comment, and share episodes you love that helps us keep delivering insights into all things data. Thanks for listening. Until next time.