#223 [Radar Recap] Charting the Path: What the Future Holds for Generative AI

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

Hello everyone and welcome to our penultimate session of the day at Data Camp Radar. I am so excited for this one on what the future holds for generative AI There are various ways by which we can look at the generative AI ecosystem today and its future. From how the technical landscape will evolve to how LLMs themselves will improve, to the infrastructure to the adoption, the future of work and how generative AI in general will really change society and the tech space as a whole in the future. So luckily today's guests all come from the generative AI ecosystem, but from different angles and it will be awesome to pick their brains and see how they view the future of AI. So first, let me introduce Edo Liberty ID Ido Liberty is the Founder and CEO of Pinecone, the managed vector database that helps build knowledgeable AI systems. He is also the former head of Amazon AI Labs and the Research Director at Yahoo. Ido is key in building Amazon Sagemaker and has computed to over 100 academic papers propelling AI research forward. His company Pinecone was founded in 2019 and is now valued at $750 million. That's not bad at all. Ido, it's great to see you.

It's great to see you.

And next we have Thomas Tungus. He is the General Partner at Theory Venture. Thomas has been a venture capitalist for more than 15 years. Before founding Theory, he has worked with companies including Looker, Monte Carlo, Mother Duck, Omni, Hex, Arbitrum and Mistin Labs. Quite a few well known names in the data and AI space. And he writes@tontongush.com, which I highly recommend that you check out his blog. One of my favorite reads of the week. Thom, it's great to see you.

Pleasure to be here. Thanks for having me on.

Awesome. And last but not least is Nick Alprin. Nick is the CEO of Domino Data Lab. As the co founder and CEO of Domino, Nick had a front row seat to the transformative impact that Domino's platform has unleashed for its customers. Nick started Domino in 2014 after spending nearly a decade building platforms for data scientists at Bridgewater, the world's largest hedge fund. Since then, Nick has built Domino into an industry leader whose ongoing innovations continuously reshaped the standard for data science and AI platforms. He holds a BA and a Master's in computer science from Harvard. Nick, it's great to have you on.

Nice to be here, thanks.

All of you approach generative AI from different angles. Thomas, you're investing in the space. You do your work at Pinecon. You're literally building the infrastructure that is powering a lot of generative AI use cases today. And Nick, you're building tooling that makes building and managing AI models much easier. So maybe let's set the stage for this conversation. How do you view the current state of generative AI this? Tom, I'll start with you.

Well, I think we're at a really exciting time. You look at the model, performance is rapidly improving. Where small, medium and large language models are, you know, there's a score called the mmlu, which is high school equivalency. And all those models are performing pretty well. The Llama model, you have open source and closed source that are both thriving with OpenAI and anthropic being closed, and then Mistral and then the Llama family of models being open source. There's a lot of innovation at the infrastructure layer. Even the chips, we're seeing pretty meaningful improvements. Nvidia, with the H200, you have AMD launching chips that are 30% more powerful than the current H1 hundreds and then even startups that have architectures that profess to be 10 times faster. So there's a lot of innovation at the infrastructure layer. And I think as that evolves and improves, there's a parallel desire from businesses to really take advantage of this. There's a dominant feeling that this is a big wave for good reason. And so we see a lot of innovation, a lot of desire, like enterprise search is a major use case. Asking questions about a knowledge base. Customer support is a major use case. Applications in legal, accounting, code completion, those are probably the five most active areas of investment today. And then in terms of the future, where we're going is making sure that these models are accurate over multi step processes. And once we're able to unlock that, then I think you'll actually see some massive productivity improvements because we will be able to delegate tasks that take a lot of time and have a computer do it.

Okay, that's Great. And ido, you know you're building infrastructure to power a lot of generative AI tools. Let us know how you see the state of generative AI today.

Yeah, I mean the space of course.

Is rapidly evolving and so on. I think we see a few different trends. I think the biggest one is enterprise adoption which sort of like started in fits and hiccups last year and I think this year people really start to invest deeply in that and for that.

They.

Models and their AI in general to be knowledgeable and trustworthy and dependable and governed and like they sort of like the excitement of the new technology kind of heads and at the same time, I mean I share the excitement. I mean there's a ton of improvements. I think agents and assistants are going to be very big. We're going to improve a lot of them with more knowledge from vector basis and other improved models and better mechanisms and so on. So yeah, there's a ton coming and there's a lot of energy and desire to build on those tools so.

So we're definitely going to unpack a lot of that. And Nick, from your perspective, walk us through how you view the state of generative AI.

Well, maybe I'll be a little more contrarian or a little less sanguine than Tomas and Ido. Look, the technology innovation is very exciting from my perspective, working with customers and most of our customers are large global enterprises. My take is we're still kind of in the inflated expectations period of the hype cycle. And what I mean by that is there's a lot of prototyping, there's a lot of experimentation but most of the real production use case, not all of them, but most of them we're seeing are still sort of internally facing. Like Tomas said, the enterprise search, question and answer chatbots and look that's fine but I don't. That stuff is not world changing. And where I've seen companies really get excited about initial ideas for products based on gen AI that would change a business model that would really be customer facing. They have exciting prototypes but they can't get across the finish line because there's still too many issues around accuracy, around safety or around ROI frankly. And so look, I think there's a huge opportunity for Genai to impact services, heavy businesses, consulting businesses, graphic design agencies. But I think that for a lot of other businesses I worry actually that the infatuation with gen AI has distracted business leaders from the benefits and the potential of traditional predictive machine learning traditional AI and I think that in some ways has sort of set us back on the potential for all AI to really transform businesses and have positive impact on the world. So, yeah, like I said, maybe, maybe a bit more provocative take, but that's how I see things right now.

Yeah, I'll let maybe Tom and Ido react to that. So how do you view the adoption landscape today? So I'll let maybe Ido personally, you take that. Sure.

I agree 100% that there is a ton of experimentation and most of the energy in this field is spent with understanding how to run these things, how to make your applications, first of all, perform what you want them to perform, be governed, be dependable, not make silly and embarrassing mistakes and so on. And that's hard. And I agree with Nick 100% that some people might have unrealistic expectations of AI. That said, I do think that a very large set of new kinds of workloads is now possible that was not possible with traditional machine learning or whatever I would call them, like predictive models whose job is mainly to score and predict and rank and do stuff that we do very well today already. And I agree with Nick 100%, are incredibly valuable and people should invest in them. But the ability to consume language and generate language in a meaningful way is incredibly powerful. And I think a lot of people don't know how to commercialize it yet within their business, and they're experimenting, but I think a lot of them already know what they want to do and already know that this is the answer. Now the question is how to build it and how to get there and when can we launch this?

Tom, I'll let you react as well.

Yeah, I agree. I think we're still early, really. People are still trying to understand exactly what's going on, what the technologies are good at, what they're not good at. If there are security concerns around data loss and there's this dynamic where historically the chief Information Security officer, the ciso, has been the one that's been responsible for securing these companies. And now the heads of data, heads of engineering are a bit thrust into this role. So figuring out how does that work? I mean, even, like the procurement processes for some of these technologies are really tough. Legal teams are needing to create new rules about, well, how do we think about software? And there's been 20 or 30 years, you have a SOC 2 compliant to this procurement process. Now all of a sudden there's this thing, where can a vendor use my company's data to train a model? Is that valuable? What do we want? The data that's on inside of VPC or on prem. Is that something that's really important to us? And so I think even there's like the technology diffusion curve, like what does this technology do? And that's slowing adoption. And there's the who's responsible for securing it? And then there's the how do we get this through procurement in a way where nobody's fired? And so those three things, those three dynamics I think have companies, organizations are still figuring it out.

Okay, that's really interesting. And you mentioned here these challenges either maybe from your perspective, you see technical challenges as well that are hindering adoption today in the enterprise.

Yeah, I would say there are three main challenges and not surprisingly, these are three big focus areas for us as well. One is the adoption of the technology itself. You have a lot of engineers and systems and application builders that haven't had the experience of data scientists or data engineering or ML engineers, and they are now entrusted with building those things. So they need to catch up and learn very quickly. And we're trying to do a very good job at both educating and giving really good tooling and making everything work out of the gate. The second thing is people need to actually make the applications more knowledgeable and perform better. So it's not just, if you could build something by just hooking up to an LLM, great, you're done. But most applications don't work that way.

They need to do support.

They need to know about your documents, they need to know about what a process is. They need to connect to your own data somehow. And so that means you have to somehow provide them with knowledge. And that is very difficult today. Vector databases in Pycon specifically make it a lot easier, but it's not enough. We have launched a system that allows people just load document and run basically text queries and so on and answer questions and complete tasks with their own data. But that's still a challenge for a lot of people who don't use Pinecone. And the last thing really underscores what Tamash said. It's not enough. You can build an application that will blow the CEOs like socks off, but you would still need to be socks compliant and HEPA and, and GDPR and jump through all the fiery hoops. And a lot of them are actually complicated questions, you know, and so it's still going to be hard. So this enterprise readiness is, I think, a big area of friction for buyers and investment. For companies who work in this space.

I think it's time as well that we can discuss outside of challenges in deployment. Let's maybe switch gears and discuss the technical trends that we see within generative AI today. We've seen LLMs become multimodal and LLMs providers improve their different product experiences. And you think GPT 4.0 with the voice mode and the data analysis mode. Claude 3.5 with artifacts. These are great X UI innovations as well. However, we haven't seen necessarily a leap in intelligence comparable to the transition from GPT2 to GPT3, GPT3 to GPT 3.5, and GPT 3.5 to GPT 4 before. Do you think that we'll see this other massive leap in intelligence soon? If so, what do you think will drive this leap? If not, why not? Nick, I'll start with you here.

I was going to yield or exceed my time to answer to maybe, you know, I don't have the most expertise on sort of the LLM architecture details and so he might have a better answer.

Yeah, maybe.

Before he jumps into that, I'll just say my prediction would be that regardless of how some of those next wave of leaps play out, I think we're going to see a lot more juice being squeezed from different ways of combining multiple models together. So more authentic workflows. I know that's, that's very. A lot of people are talking about that right now and I think that's also going to enable better harnessing or extracting more value from smaller, more specialized models. And I think we'll get more, more system AI systems that are using a lot of more specialized or small models potential likely in sort of agentic patterns to improve overall performance of these systems. But anyway, to answer your question directly, I don't feel like I have the best expertise.

So let's switch gears then. Ido, I'll let you take that on.

I'll offer a contrarian view that I actually don't Models are going just the current way we think about models are not going to improve dramatically in where it will qualitatively change what can be done for multiple reasons. The like physics kicks in at some point on how these things are trained. There is the economics of doing that. When model training out top notch model was quote unquote only $10 million, which was way out for any tiny company. But a lot of medium and large companies have that capacity. When you need to invest a billion dollars to train a model, suddenly this is really kept to a very, very small set of players and for reasons that you know it's going to take a long time to go through. I personally Think we're reaching a level of saturation that it's going to be hard to qualitatively change. And yes, there are improvements, but to me, product wise, if some accuracy measures goes from 80 to 85%, it might be a tremendous engineering feat and it could be a big difference. But it's not a qualitative change. You didn't go from 15 to 80 and you didn't go from 80 to 100. Yes, it's a better model, but is it qualitatively different? I don't think not in terms of the applications and use cases. What is going to change is that we are going to make these models a lot more dynamic, knowledgeable, plugged into the right places, and find ways to make them basically improve the ways that they train and make them 10 to 100 times bigger. And that's happening, but it's not the models, you know, and the, the, the, the way you think about a language model, it's going to be completely different. A qualitative change has to happen for that curve to break, basically, in my opinion.

Okay, so models are going to become more useful, but not necessarily raw, intelligence wise. They're gonna have a simple. A jump.

No, no, they, they will, but. Well, it's, it's hard to explain. I might, I might have to stop here. But I will say that I don't think the, the models, the way they are trained today are I think hitting an, an asymptote. An asymptote that we're not going to improve much on.

And Tom, I'll let you also react to here. How do you view the next wave of LLMs?

Yeah, I think there's a bifurcation. You'll have large language models that will cost like a billion or 10 billion. And you'll have a handful of companies that are training them and I suspect they'll probably be government subsidies training some of these massive, massive models because they're of national importance. And then you'll have smaller models that are purpose built, that are much more accurate, that are running on your mobile phone, that are much more. And so you'll have this basically bifurcation and it'll be used for different use cases. Query will come in, there'll be a classifier that says, have I seen this kind of query before? And if I have, it probably goes to a small language model to optimize for a particular task. And if it hasn't goes to a large language model, it's basically a big consumer search index, effectively. And so I think that's where we're going. We call these things constellations of models. We're starting to see enterprises actually develop some pretty sophisticated layers on top. There's like caching layers in order to reduce costs. Because if, let's say you were working on a translation software company saying the word for hot dog in Russian doesn't change and so you don't need to go back to the index every time, you can just cache it. And so that's the way that I see it. I think multimodal is kind of the, I think the two big waves of the day is multimodal meaning use text to produce video or vice versa, or learn from video and use that to improve text models and then agentic systems which Nick and Edo have talked about already, which is basically chaining these things together to act on behalf of humans. The multimodal technologies are new. I think you can see it with clods, artifacts starting to work, people creating three dimensional models of particle physics with just a handful of sentences and then it's rendering, it's pretty cool. And then on the agentix stuff, I think the compounding error rates through multi step processes is really hard to manage today. And there are lots of different architectures there, there are suggester critiques, there's adversarial networks that people are starting to use. There's the use of the suggestion with a classic classifier in order to minimize the error. And that's all research today. It's very, very early.

Yeah. And one thing I think that is are we reaching peak saturation to training data for example as well? Like when it comes to kind of training the next generation of models. So, and you hinted at this Edo, like saturation of training and like saturation of models today. Maybe walk us through your thinking here a bit as well in more depth. I'd love to learn more how you see the next wave of training will happen. What needs to be true to get incremental improvements on models?

Well, I can't talk about it because we're working on it, but.

So Tom, maybe I'll let you hint in here as well.

One way of thinking about training is there's a cost optimization function here which is how much data do I need and how many compute hours do I need to train a model. And there was a paper produced called Chinchilla which created a heuristic and algorithm to determine exactly what that was. And Meta decided on the llama3.8 billion parameter to go way beyond it. Where they spent, they trained it on 15 trillion tokens and the performance there was actually. It was very expensive to do, but the performance was really good, especially compared to larger, larger models. And so I think we're sort of figuring out how do we squeeze as much juice from the lemons that we have and exploring that surface area. We're probably in a local maximum. The other dynamic, and Ito and I were talking about it before, but that Llama model was trained on about 15 trillion tokens. Our estimate is just somewhere with like 20. Roughly 20 trillion tokens on the Internet. And the question is, okay, well, we want to train a model on like 100 trillion tokens or 500 trillion tokens. Where does that come from? And I think that's tbd. Nobody knows. Right. It could be video, it could be content that's yet to be created. It could be synthetic data, or computers are actually identifying where models are weak and then producing data to supplement it.

Okay, so I think we'll find out and then maybe if we kind of look at. Nick, you mentioned this earlier in the conversation that, you know, we're still in experimentation phase for a lot of tools, but all three of you hinted at kind of the deployment challenges with generative AI solutions today. You know, a lot of the infrastructure technology to enable fast deployment of LLMs, think monitoring, debugging, retraining systems, all of that still needs to be built. So how do you feel maybe the maturity of the LLM middleware ecosystem today or like the generative AI middleware ecosystem today? So, Nick, maybe I'll start with you here as well.

Yeah, I think it's similarly early. I mean, and the other thing I would say is I don't. I think a mistake people are making is thinking it's conceptually a completely different thing from like, let's say, the traditional MLOPS challenges we've all we've had over the last five, 10 years. I hear some people think, well, LLM and Gen AI make it so we don't have MLOps problems anymore. It's a different set of problems. And rather I think it's more like the Genai use cases take all the same problems we had with traditional mlops, but they just exacerbate them. Or now it's like if we're playing a video game, we've activated the hard mode so you have a lot more intensive compute resource requirements, infrastructure requirements. It's GPUs, it's not just CPUs. You need to scale these things. That comes with a whole bunch more costs you need to deal with. I mean, we have customers that have gone from deploying again for prototype use cases, traditional predictive ML to now gen AI. And all of a sudden their compute bills are going out of control. And now they. Before they didn't need to worry about things like elastic auto scaling for inference, now they do because the GPUs cost a lot more. You talk about model governance, model monitoring, that's all based on the fundamental characteristic of any sort of predictive AI system that, that they're probabilistic. And so their behavior can, they can have unexpected failure modes, their behaviors can change, their performance can change, so the world around them changes. Well, Gen has a much wider surface area of unexpected failure modes than say a traditional predictive ML model. And so now the set of things you need to do to check and guardrail and the set of human processes that you need to put these things through as you move them through a sort of a quality assurance process that's all much more complicated. So yeah, I mean like the sum it all up, it's like all the same problems we had, we had with ML ops are now incremented or exponentially harder for gen AI. And I would just add, I think a lot of these challenges are not. You asked about middleware, I'd say a lot of these challenges are not strictly technical. They are business process challenges. They are people in process orchestration challenges. If we're going to put a Genai model or system into production in an enterprise that might touch customers, especially in a regulated environment, a bank, an insurance company, a life sciences company, what checks have been executed to ensure that is compliant, that is safe. Everyone that we work with is still trying to figure that out.

And then Thomas, from your perspective, you're investing in this space, right? Like you have a very elevated bird's eye view over the space. I'd love to kind of see how you see the kind of infrastructure space for LLMs evolving today.

Well, there's, I mean, lots of exciting things happening, right? You're building a great business. Vector databases are really important. We're investors in a company called Superlink, which is a, called a vector computer, which allows you to take structured and unstructured data and put it together in a unique way. But it's really early in this ecosystem where as we talked about, buyers are still trying to figure out exactly what it is that they want to build. And until they know what that is, it's hard to have a view on exactly what the ideal architecture is. Then you have OpenAI and the other big companies also deciding strategically what areas of infrastructure do they want to play with? So OpenAI released an open source evaluations framework to help software engineers understand exactly how well a model is performing before they release. A little bit like a testing harness. And that was a move for them into a category where there are a lot of startups already. So for startups, I guess the way I'd put it is you're dancing with elephants. They're these big companies who really care about these markets. Microsoft has a $5 billion run rate business in 18 months. OpenAI is north of 3 billion. And you have others that are growing pretty fast and they're still deciding what parts of the stack they want to play with. You look at OpenAI's acquisition of Rockset last week, I think, and open questions about what did they do with that technology and that team. So the chessboard is not set. There's still a lot of moves to be played and you have to navigate. I think startups have to do what they do best, which is be nimble and navigate the ecosystem as it's rapidly forming.

Yeah, brilliant. And Ido, you're building a company that's at the center of the infrastructure space. And LLMs walk us through how you see this space evolving over the next year. And I don't want to hear product roadmap, that's not the question. But I would love to see kind of how you think the, the ecosystem will evolve in the year to come.

I agree with Tamash 100%. I mean this is the. It's a very dynamic space and I think that the. There's going to be one truism that's going to still prevail here, which is companies and enterprises specifically are going to find AI more production ready and useful very soon. We are working on it. Others are working on it. The elephants in the room are working on it. It's obvious that this needs to happen. And so exactly who does what part of the stack and who gets to own what piece of the pie I think is anybody's guess. We of course want to have the biggest piece that we can own out of that. It's obvious, it's like any other. But be intellectually honest and say that nobody can really tell.

Yeah, And I think this segues to my next question really well because everyone's working on this problem of deployment, making sure that models are very productive. I think Nick mentioned earlier that we're at the peak of inflated expectations, but we will reach the slope of enlightenment at one point in time here. Right. And once we reach critical Massive adoption. If generative AI reaches critical mass in terms of adoption, how do you think it will shape the nature of work, or what does that future look like? So maybe, Thomas, I'll start with you.

You know, the way that we think about it is there's lots of toil within work. There are lots of repetitive actions that we do, and each discipline has its own right. Within sales, there's the role in sales, development of understanding leads and qualifying leads and deciding which ones to pursue. Within the world of legal, there's paralegal work, which is the administration of documents. Within the world of accounting, data entry across different kinds of tax forms that have different formats. And within the world of software engineering, it's remembering which arguments go in which order in a function call. And I think the future of work with AI is having computers solve a lot of those rote tasks and work alongside of these copilot systems that we have. And today, the productivity gains, at least from some of the early companies, suggest it's about a 50 to a 75% productivity gain. Maybe it's a little bit aggressive. Computers will not love each other. That's true, Luis. I think if we look to the future where work is actually automated and we can delegate tasks to computers the way that we delegate tasks to humans, the best analogy is looking at automobile manufacturing lines where robots have replaced a lot of the labor. And there you see about a 275% improvement in productivity, where one robot can produce about the output of 2.7 humans. And I think we should be able to get close. But that's not based on anything except a rough guess, but I think order of magnitude. We should be able to get to a place where a lot of this toil, this rote work that is unappealing is automated. Right. Okay. 100 years ago in America, there were 4 million human dishwashers, people who wash dishes for restaurants.

Wow. Okay.

And then the first robot introduced in most people's houses with a dishwasher. And now we don't, you know, the chores that we give to our children is loading the dishwasher instead of washing the dishes. I think that's, that's a very good analogy for future work in White Flower.

That's very, very fascinating. And Ido, I'll let you here also react. How do you see, you know, once pinecone takes the biggest part of the pie as much as possible, how do you see the view, the future of work here?

I think we are producing great tools. I think we're very far from any Model or set of models or agents completely replacing somebody's job. I think that's very unlikely unless your job is so menial and just whatever, like, except for extreme cases, okay, all the lawyers and doctors and accountants and analysts and this and that, they're not going away. What they are getting is fantastic new tools that make them work more efficiently, more correctly, offload the grunt work that maybe they don't want to do. And that's going to happen. I mean, I'm 100% sure of that. And history shows that when you have something like this, that actually improves overall productivity for society, which ends up being good for everyone.

Okay, great. And then with productivity, when the paradigm changes for productivity generating technologies, you mentioned this, Thomas. The chores one used to give to their kids was, was the dishes. Now it's loading the dishwasher. That skill has changed, right? The skill from washing dishes to loading dishwasher. What would your advice be for individuals today to build the skills needed to kind of adapt to an era where generative AI is widely adopted. So, Nick, I'll start with you on that question.

Yeah, well, I think, I mean, building on what Tomash and Ida both said, if Genai plays out the way everyone's expectations hope it will, it's going to be leveraged for people, it's not going to replace people. And so in any given industry or any given job someone has today, there are a set of things that can be delegated away, but there's going to be a core essence of that job that is the insight part or the creative part or the architecture part that is going to be, that is going to remain undelegatable to one of these AI systems. And so I think, yes, if you're a software engineer, then you don't need to remember the order of the arguments to a function call, but you do need to be able to understand a customer requirement. And based on that understanding, design a software architecture that appropriately reflects the abstractions you need to model the customer's problem domain. So the conceptual or more insight based parts of any job are going to be increasingly important. If you're a graphic artist, you'll still need to be able to have a picture of something in your head that you want. And then you can interact with, you can coach, you can instruct an image generation system to create that, but it'll be getting a computer to create what you are picturing in your mind's eye. So maybe the technique of how do I use Photoshop to compose this thing I'm picturing maybe that role will go away. And just combining that thought with your question a minute ago about what this all means for the future of work, the interesting potential dynamic I am thinking about is what will this mean for a whole wave of disruption that flows across every services industry. So if you're a consulting firm, if you're a graphic design firm, if your business fundamentally a services business, I think that you are at risk over the next five years of being disrupted by a much more efficient, much more scalable, much more high leverage model of executing that business.

Okay, that's great. And Tom, I'll let you also react. What do you think the future skills to adapt are?

Well, I remember when all the search engines came out, right? Google, Metaweb, Yahoo, AltaVista, and you needed to figure out exactly how to use them well. And the people who could use them well were a lot more productive at work. And so that's where we are. So I think prompt engineering is probably the skill of the day today because if you can do it really well, you can produce a blog post, you can produce marketing copy, you can produce images, you can produce code. And those marginal benefits, if you're 50% more productive than a coworker or competitor, you will be promoted. And that's sort of the name of the game, right? I think all software is sold because it's a promotion in a different form. Somebody buys the software and champions it because they can be promoted. So I think comp engineering is the skill that is the most broadly applicable. There was this awesome study of I think BCG or McKinsey. He won the big consulting company and studied the way that, yeah, it was called Centaurs and Cyborgs. And one of the most interesting use cases there was asking the AI to simulate a potential software buyer and be the foil in a conversation. We're seeing this in education too, where students are creating their own personalities to talk about how to solve a calculus problem or a chemistry problem set. That's probably the broadest and most applicable use case today.

And then either I'll let you finish us off with how do you view the state of skills in an era where adoption has. Has matured?

It's very hard to. It's very hard to know to be honest. And I might not have the best like a very strongly held opinion on exactly what that's going to look like. I remember also when search engines came out like people imagined all sorts of professions that ended up not existing and so on. So think prophecy was. Is on on this topic is a bit Hard. I think it's, it's obvious though that like a set of very powerful tools are going to be produced and professionals who don't master those set of tools are going to be left behind. If you're that lawyer that rejected the use of a PC and kept using your, you know. Yeah, you didn't, you didn't stick around for a long time.

I couldn't agree more. And I think we have time for one question to take from the audience. So do you think that we might see agentic approaches increasing in effectiveness to the point that it reaches the level of predictive? So, for example, can you reach a point where agentic systems are so good that you can predict accurately that they will work well most of the time? Are we headed to that type of future? Because, you know, Thomas, you mentioned the compounding error rate across the different steps of the agentic workflow. I'd love to see where you think agentic systems are headed in the near future. So let's start with you maybe Tom.

I'll put it a different way. I think we'll get to a place where we will brute force innovation. In other words, we'll kick off massively parallel jobs that are trying to figure out what is the next generation chemical compound. And it's not that the systems will predict, it's that we will just throw so many computers at it that at some point in time they will find a solution. And so it'll discover huge solution space and one of the answers will be there. I don't believe computers can predict very effectively. And you look at classical machine learning, they've had a really hard time. They can predict things that are very stable and repetitive and cyclical. And Google made a really beautiful business model based on very predictable patterns. But for things like poor innovation or creation, that is not just recombination is just not the right technology to solve that.

Nick, I see you're nodding in agreement. I'd love to see your thoughts here as well.

Oh yeah, I was just going to agree with Tavash about the brute forcing innovation. There's a hedge fund we work with that's doing that exact thing using Genai to come up with candidate investment algorithms. And then of course it's easy to, you can test and check each one. Their point of view is, look, if we generate 10,000 and 9,999 are bad ideas, but we get one good one that's worth it. I think that brute force innovation, you only get the ROI for these areas where the innovation has a big payoff and you can kind of justify the investment to search a big space because the vast majority of the ideas will be bad.

Eido, I'll let you finish us off here. How do you see agentic systems evolving?

Well, I'm both in support and violent.

Disagreement with so much. I mean, I think brute forcing doesn't work. Exponents tend to be exponential. And anybody who wrote like a five nested for loop knows that it gets pretty terrible pretty quickly. And so I don't think this is solved by just going, well, hellboarded. For the love of God, we've been spending literally billions of dollars worth of energy on this thing. You know, we didn't solve everything, so I don't think that's the way to go. I do think that some innovation comes from other systems that then agentic systems can actually discover interesting things because humans also can produce exponential amount of compute in in our own heads. And the people who do research are also energy bound. And so whether we do this in brute force, I don't think so whether it actually produces interesting insights and innovation, I think 100 I think this is.

A great place to end our chat. I want to make sure everyone send as much love as possible to our speakers today. Thank you so much. Thomas. Thank you so much. Ido, Nick especially Ido and Nick are both on vacation in Greece and Italy respectively. So I really appreciate them making time out of their precious vacation times as leaders of their organization. So I really, really appreciate your times and thank you so much for everyone who attended and again, thank you all so much.

Thank you.

Sam.