Wolfram Technology Conference 2023: Stephen Wolfram's Keynote Address

One. Welcome to the keynote for the Wolfram technology conference of 2023. So it would have been fun to do this in person, but the good news is doing it, virtually many more people are able to attend our whole conference. And that's great. In the future we'll be sort of trying to figure out the best of both worlds, of both in person interaction and wide access. Maybe we'll be doing some regional meetups, things like this. We'd like to hear from people about what suggestions people have about that. All right, well, I got lots to talk about here today. I'll talk about some big picture kinds of things. I'll talk about some new features and technology kinds of things. I'll talk about some future kinds of things. And. Well, one thing, 2023, kind of the year the LLMs landed, we didn't know they were coming, nobody did. But our technology was very much ready for this arrival. And all sorts of exciting things have become possible. All sorts of things both that we've been able to do and all sorts of things about the ways that people look at technology. Surprises about technology, possibilities of technology have happened this year that are really very aligned with things that we've been talking about working on delivering things for many decades. So this is also the year that not only do we have lots of human users, but we also got lots of AI users as well. So, you know, in early January, I started to describe kind of sort of the story of how we might connect to ChatGPT and the kind of complementarity of the technology that we have compared to what is provided by a large language model. And actually just two months later, we were able to release the ChatGPT plugin for Wolfram language and WolframAlpha. And this was actually built in just a few weeks. It's not totally trivial to build a major kind of piece of technology that's in a very new direction in a matter of a few weeks. And of course that was only possible because in addition to those few weeks, we had three and a half decades of work on the technology stack that we've been building all these years. And that's were things that the fact that we had Wolfram Alpha, which could communicate in natural language, the natural language, so to speak, of the LLMs as well as us humans, the fact that, that we had Wolfram language set up, that we had documented Wolfram language, that we put all those things out on the web that kind of explained how to use Wolfram language, and that all of you had put out all sorts of Information about using Wolfram language for all sorts of purposes. All of those things were read by the LLMs and that's what allowed them to be able to write Wolfram language code that allowed this connection to be made as efficiently as it was. In addition, there's a lot of software engineering that has to go into something like this, and that was something that was delivered primarily through the cloud technology stack that we've been building all these years. So it was a remarkable thing to see this come together in just a matter of a few weeks. So I think kind of the sort of bigger picture of this is, well, lots has been happening with kind of the integration of Wolfram language and with our general technology stack, with the kind of new world of AIs and LLMs and so on. I think we're sort of in a spectacular position here of being able to, with sort of the unique thing that is able to bring kind of computational superpowers to kind of neural net based AIs. It's not something that the neural net based AIs can do for themselves. It's like the hopes that, well, eventually the AIs will be able to learn everything. I think that's similar to the hope that eventually us humans will be able to learn everything. And the fact is, us humans find it really useful to use tools, computational tools to do things. We don't get to run code in our heads, we don't get to compute all sorts of math. We don't get to know all of these things that are part of kind of the corpus of knowledge in our civilization. We don't get to know those kind of in our brains. And nor do neural nets get to efficiently do that. Instead, the kind of the workflow that seems to be fantastically efficient is neural nets for what they're good at and actual computational knowledge and computational language for what it's good at. And that's been the story that's been unfolding over the course of this year. So this whole idea of kind of formalizing the world in a computational way is something that we've been kind of building to for. Well, I've been building to it for about 45 years now. And we've been as a company have been building to it for about three and a half decades now. Kind of the concept is when we have things that we can just sort of think about in our minds, but how do we formalize our descriptions of the world? And in a sense, there have been several formalizations that we have seen develop over the course of Human history. The very first one is human language itself. Instead of just pointing to different rocks, we have the word rock that can abstractly represent sort of all rocks, so to speak, from language, we get things like logic, a way of kind of lifting sort of a formalization of what one's talking about above the particulars of individual words and so on. And then we have the idea of mathematics, the formalization of the world in mathematical terms. And in the past century, the big formalization that's the one that I. And we have been so deeply involved in, is the formalization in terms of computation, the idea of sort of representing the world in a precise computational way that allows you to kind of systematically figure things out, so to speak. And in a sense, what we've been trying to do. I'll talk some more about this later. The whole kind of concept of Wolfram language and our technology stack is to provide that kind of notation for computation in the way that, for example, notation of mathematics was provided a few hundred years ago. That kind of led to the development of mathematical sciences and so on. Well, in terms of where we're at now and the relationship to AIs, I think that one of the things we're seeing is that kind of the technology of LLMs, which has been very much pioneered by a few companies, I think will become more generic. But the thing that we have, this kind of stack of computational knowledge and computational language is really a completely unique thing. We've been building this thing for all these decades, and people kind of. It's just something very different from what anybody else has been building. And now we're in this great place where in addition to the human users of this technology, we have the AI users. And it's a capability that sort of all of the AIs need the ability to actually get access to knowledge, computation and so on in a structured way. This is something that we've set up and that sort of AIs need, just as humans need to use these kinds of tools. I would say that I think as we look at kind of the development of AI technology and the things that we're doing and so on, particularly on the AI side, I think the great value is going to be in the kind of detailed use cases that are built around the raw technology of LLMs, the raw kind of linguistic interface that we now have. With LLMs, we've had sort of the development of things like graphical user interfaces. We now have this kind of linguistic user interface. I would say that in Wolfram Alpha, we have the kind of small utterance, high reliability linguistic user interface. We can deal with small pieces of text and do it with very high fidelity. We are not set up to deal with either getting input or getting output of big essays and so on, but that's the domain of LLMs. Although the LLMs can kind of crispen up those essays to generate some kind of small question that can then be fed to WolframAlpha and so on. And where WolframAlpha can kind of catch that piece of natural language and with very high fidelity, convert that fragment of natural language to sort of precise Wolfram language, computational language code. Well, I think, you know, I've talked about in earlier years the sort of feeling that we have. The kind of Wolfram language for many people is still kind of an artifact from the future. It's something where a lot of ideas that we've had over all these decades are things that are kind of, kind of inexorably obvious from some way of thinking about things. The ideas of symbolic programming, ideas of computational knowledge, the kind of idea of coherent language design and so on. These are all things which in some sense, I think are inexorable ideas, but they're also ideas that. Will seem more obvious decades from now than they are today. It took a solid, oh, nearly three decades for the idea of notebooks, which always seemed to us fairly obvious to be obvious to other people. These other ideas are much deeper, in a sense, and it'll take more time, probably, for these kind of artifacts to not be sort of purely from the future. I think that it's kind of fun to see in this time of LLMs and AI and so on, that a little bit of future arrived that people weren't expecting. And I think we were very much ready for this. And it kind of wakes people up to the idea that there are things from the future that can arrive. Now, we happen to have had many of those things for a long time, but this is something that. Is kind of an interesting dynamic in the world over this past year. Well, of course, for many of you, as for us, we've been kind of grabbing the future and using it in Wolm language for, for a long time. And it's, you know, I, I use, I use it every day. To me, it feels like kind of a superpower. You know, I imagine something that I want to do, and then if I can conceptualize it in computational language, then I have this ability to just sort of, with my own fingers, get that thing to happen. It's, it's very much a, A Kind of a, A, A sort of a superpower feeling. You imagine what you want and then you can actually make it happen. We're kind of reducing out the mechanics of doing things once you have conceptualized what to do. And for me, for example, as a person who built this stuff in order to be able to use it myself, I think that that ability to kind of have this sort of computational superpower is what's led to things like our physics project and all sorts of things like that. I would say that AIs. One of the things that's happened is that AIs have kind of clarified a bit what one might call the programming story. I think one can distinguish. There's computational thinking and there's raw programming. Raw programming is about taking the capabilities that computers intrinsically have and kind of poking at them to get the computer to do what you want. Computational thinking is something more abstract. It's about conceptualizing the world in a computational way, thinking about how to kind of formalize what one wants to know or how one wants to describe the world in computational terms. And in a sense, that thinking gets done by people. And sort of the thing that is the thing that people have to contribute is, so what do you want to do now? It's our job in building our computational language to take those computational thoughts and make them as executable as easily as possible, to automate the process of taking those computational thoughts and being able to execute on them. And it's very different from this kind of idea of programming where you're kind of poking at the computer to tell it specifically, set this variable, do this thing, et cetera. It's something where our job is to try to make a computational language, which is this kind of bridge between the way we humans think about things and what computers can intrinsically do, but where we are kind of automating that process of getting the computer to do these things. Well, I mean, for decades we've been working on sort of automating the process of programming. That's kind of the whole point of our sort of computational language technology stack. And now people realize that, yes, with an LLM, you can write a slab of Python code or Java code or something, and maybe it'll even be correct. And that having described what you want, you get this kind of slab of code. Well, our point in our computational language is, yes, it's a slab of kind of boilerplate code, but that's a piece of code. If that's something that people want repeatedly, something that we can describe in natural language. That's something that we should have implemented a precise kind of computational language function to implement that. And that's kind of the idea is that we are trying to, just as human language tries to take the types of things we want to talk about in the world and has words for those kinds of things. So similarly in our computational language we want to take the types of things that one can think computationally about and have functions for all of those things. And that's what we've been, what, what we've been working on all these years. And that's something that gives one this, this very different dynamic in, in, in interacting with the world. This, this kind of idea where, where, where we have kind of the. Once you can imagine what you want to do computationally, we try to automate actually executing that on a computer. But in a sense that moves kind of the, where's the real work to be done? And you know, in the idea of RAW programming, there's lots of work to be done about tapping out all those lines of code. And the typical programmer is spending most of their time writing code which, yes, the LLM can just put in that big slab of code that it produces a piece of boilerplate. That's not the way that people who use Wolfram language have been doing things all these years because most of that, that slab of code, you just, you just don't need that. And it's, it's. I think the thing that sort of interesting to me is that, you know, when Mathematica first came out 35 years ago now was a, it was a very interesting dynamic because there were a lot of kind of mathematical scientists and so on who had had this methodology of calculating things with pencil and paper and so on and suddenly they had this kind of computational superpower. But their involved today. I want to calculate this. Once I know the answer to that, then I've immediately got another thing I want to do. Because they were the people who were figuring out what they wanted to do and that allowed kind of very rapid absorption of our technology there. Because people. The part that was kind of the, in a sense, in some sense the hard part, which is figuring out what do you actually want to do. That's what those people today are familiar with doing. That's the story of what they do. Programmers, it's a different story. Programmers, it's like you're going to implement this collection of dialogue boxes and so on. And there's kind of a process, a mechanical process for doing that. And it's not for the programmer to Figure out, okay, what's the next thing I'm supposed to implement. So I kind of realized this increasingly and it's actually been sort of made very clear with the sort of arrival of LLMs and the kind of programming automation that they make possible. That what we see is it's very typical in companies and other organizations to see that the CEO CTO of the company uses our technology every day. I'm sure that some of some examples of that here today. Similarly, kind of sort of intellectual leaders in lots of areas use our technology every day. But the people who are kind of in the trenches doing the sort of more mechanical programming are not. They say, no, no, no, we're just going to write low level code and so on. What I realized is what's difficult for those people is they're not used to thinking about what to actually do next. They're not used to doing this computational thinking part. They're used to RAW programming. Computational thinking is the thing that is sort of the ultimate kind of strategic part of what has to be done. The raw programming is what we have been working on Automating, what the LLMs provide some automation for as well. But that's the part that is in a sense the part that really there don't need to be humans doing that. That's been kind of an anomaly of history. A typical thing before automation really takes over in an area, lots of people are doing kind of that low level work. And I think the thing that is what we realize is once you can kind of do the computational thinking and use computational language to implement that computational thinking, then you're in such a tremendous position to make forward progress in lots of kinds of areas. But you, first of all, you have to actually learn how to do that kind of computational thinking. You have to actually learn how to figure out what you should do next, so to speak. And I think that's what kind of realized is that people who've used Rothman language for a long time, it's kind of almost by definition. These are computational thinkers. What we have tried to do is to provide kind of this notation for computational thinking, this way of being able to sort of crystallize computational thoughts. It's kind of analogous. I've used this analogy many times. When mathematical notation was introduced, it kind of gave one a streamlined way to talk about mathematics, which had pretty important consequences. Like it's what led to the development of algebra and then calculus and pretty much all the modern mathematical sciences. Well, so now we have a notation for computational thinking which allows us to build sort of computational X for all X. And I think the thing to realize is it's that computational thinking that is the critical seed. Yes, you might have a company full of programmers, pure sort of raw, low level programmers, but the fact is the real value is going to come from the computational thinking, from being able to represent what one's thinking about in computational language. That's where one can actually. And it is our job to make sure that that's something that you can implement with as well as think in terms of. I mean, a very important feature of our language is that it's something that provides you kind of this notation for thinking in. It's a language which is intended not just to be written by humans and fed to computers, but also to be read by humans. When it's written well, it provides a good way of expressing kind of computational ideas in much the same kind of way that well written mathematical notation provides a good way to express mathematical ideas. So it's the. I think one of the things that we see is. In kind of the educational side of things, there's been a tremendous push for people to learn computer science. It's not completely clear always what that means. Part of that is learning raw programming, part of it is learning theoretical computer science, which is really quite different kind of area. And there are some applications that have been sort of now rolled in, in some places to computer science. But I think the real future is what we might call sort of the computational X direction, not cs, but cx and learning kind of the computational thinking and computational language to support that computational thinking that allows one to really work in the direction of CX computational X for all X rather than sort of traditional cs. And I guess all these years we have in a sense been building the infrastructure for cx and that seems to us to be kind of where the future is headed. And in a sense. What we've tried to build still remains in many ways kind of an artifact from the future. In a sense, what this conference, I suppose is about in some ways is pulling that artifact from the future into the present. Talking about the amazing things that one can do with our technology and these ideas now, hearing about what some people are doing with our technology in the world at large. That's what this conference is about. Well, it's, I think the thing that, yeah, a few other things to say. I mean, I think it's been, as I say, I kind of originally started building this whole technology stack because I wanted to have it to use myself. And I've had the Good fortune of being able to kind of alternate over the years between building that technology stack and using the technology stack typically to build basic science. And it's been quite exciting to see that that basic science has ended up in turn informing what can be built in the technology stack and then allowing us to go further in those directions. And I think it's kind of. The thing that has happened is we've been sort of building this taller and taller tower of kind of a mixture of basic science and technology. And I think one of the things that I found interesting in all of this is it's really all about whether it's the basic science, whether it's the technology, it's all about really understanding the essence of things. Language design is really about what is the essential point. How can you as clearly as possible represent some particular set of ideas? I mean, I myself have written a lot of text that is expositional text, blogs, books, whatever else that are trying to explain things to people. But in a sense, I think the purest form of that sort of expositional effort is language design. That's kind of the ultimate way in which one takes sort of understanding and crystallizes it into something which then is not only a representation of one's own understanding, but also something that can immediately be used by many people. And as many of you know, we've been live streaming a lot of our actual process of doing that language design, that the effort of sort of grinding things down to get to the essence of what's going on, to be able to build the right structures in our language, in a sense, sort of. I view it as being the case that sort of every function encapsulates some achievement of our civilization, whether it's knowledge, whether it's algorithms, methods, whether it's even just a way of thinking about things. It's kind of like these are kind of encapsulations of achievements of the civilization. And sometimes they're things that we discovered ourselves, but a lot of the time it's kind of existing achievements of the civilization. So we're kind of proud of this whole structure we built in the 7,000 functions and so on. And I hope you all enjoy using it. I think for me, the sort of thrill of using this kind of superpower has not gone away even after all these years. It certainly helps when one gets fluent enough in the language that one is kind of thinking directly in the language. And one can start typing, often language, language, without being able to even say what one is going to type. It's just kind of A natural, fluent thing to do. I suppose, in some sense, maybe, as we built all these functions, I sort of slowly have gotten a bit jaded in the sense that it's like, does the function only do that? That would have itself been an impressive function. Maybe it would be a function which would have supported a whole company just running that particular algorithm or something. But now the kind of what we end up building has gotten bigger and bigger. And in a sense, that's been possible because we have been using this automation that we've been doing as a way to allow us to implement more, to build our tower of technology more and more quickly by automating the pieces of what's needed for that kind of building. All right, well, let's get down to some details. So at the last of these technology conferences last year, we were about to release version 13.2 of Wolfram language. It came out on December 14th. Right now we're about to release version 14.0, and middle of the year, June 28th, version 13.3 came out. I have to say that our 0.1 releases are absurdly large, and 13.3 was a kind of a whopper of a release. But. And in a sense, what we're doing is we're trying to deliver a couple of releases every year. We've been on that rhythm for a little while here, and sort of at every version, we're kind of delivering whatever has been packaged up to be part of our permanent system from our sort of R and D pipeline. And kind of the idea is that our Integer releases try to have more parts that are kind of where there's a whole section that can be kind of packaged up. And then the point one releases, we're kind of releasing the things that got nice and finished and polished in and of themselves, but don't necessarily make such a big picture in the whole system. So it's kind of fun to see if we look at kind of what's been developed even sort of, since version one, so to speak. This is kind of the big picture of what mathematica as of 1990 looked like as compared to the Wolfram language of today. And in a sense, this is part of why we've been making the effort to sort of rebrand what we're doing as Wolfram Language. And the Mathematica part is the math part, which is what was begun some parts of. There was more of that, percentage wise, by a large amount back in 1988 than there is today. It represents a rather small fraction of what we have today. Even though it's been something where we have been kind of the leaders and putting a great deal of effort into further development of kind of mathematical capabilities. But the full picture of making this kind of computational language for everything, the seed for computational X for cx, that's what this whole picture is about. And the Mathematica mathematics part is a small part of that whole picture. The whole picture is something vastly more ambitious than, than in a sense, quotes Just that even though the just automating mathematics and mathematical computation is much more than a just. It's a very big thing and it's a thing that we have been very focused on for all these years. Well, so let's see, let me start off and talk a little bit about what's new in. What's new in version 14. And there's some things that are not in version 13.3. People who've been watching our live streams will have seen some of the things that are coming for version 14 already there. I have to say for me personally, after all these versions of our system, it always takes me a little while to remember what I don't necessarily know. In what version did we introduce that? Is that 10 years old? Is that 5 years old? Years old? Mostly what I remember is the sum total of what's been done. Well, let's see, where can we start? There's so many different places to start. Well, let's see, this was the version 13.3. Lots of things in version 13.3 that we have to talk about. But let's start in on some version 14 kinds of things. Well, so here's an example. This is something where the heavens wait for no one. And one of the things that's coming up in April of next year is a solar eclipse. And so we have been building up our astronomical computation capabilities in recent times. And so in version 14 there will be what you need to, to sort of know about this upcoming eclipse. So let's see if we can get a map here of the eclipse. There we go. A nice map of where the eclipse will be. And if we go on and ask questions, this is kind of a high wire act of astronomical computation. Eclipses are a hard thing to compute for here. I don't know where it thinks I am right now, but anyway, we can see, we can always, always use the tech to find out. Let's see where it thinks I am based on my Internet connection and so on. Oh wow. Where is that? No idea where that is. All right, well we can, let's look at the five Nearest cities to where I am right now. Huh, Fascinating. I don't know where those are, not where I actually am. But anyway, let's, that's. This is telling us information about the solar eclipse for. Next year. And let's see if I say take wherever the here is and just plot the magnitude of the eclipse, how much of the sun is covered by the moon as a function of time in the couple of hours around that eclipse. Or maybe we can even ask a question like let's plot where the umbra, where the shadow of the moon is. Okay, now we're learning where this is. This is somewhere near New York City. But that's now computing how the shadow of the moon will fall on the earth. And lots of stuff has to come together to make a computation like this work. And I think it's really neat that we can kind of express that in this kind of computational language way. All right, changing by the way, this whole astronomical computation, celestial mechanics, this is a good example of something we've been building over a series of versions and we'll see it kind of fully come together with orbit computations and deep space kinds of things and galactic astronomy and so on over the next few versions. Okay, well, let's see, let's turn to something quite different. Let's go to something a bit mathy. Talk about finite fields. Finite fields have been implemented as kind of a package and have been on our to do list for decades. Finally now in version 13.3, we introduced sort of finite fields as a built in thing, as another type of numbers, just like numbers with errors, numbers, complex numbers, numbers and so on. Finite fields, numbers of finite fields are another kind of number. Here's a finite field over GF17 squared. And we can just do computations with this. So there's a number in that field. We take it to the Power 100, it'll be another number in that field. We can, for example, now we can have that. We can have an equation represented in terms of a finite field and we get results in the finite field and so on. Also in math, something that's coming in. Well, here's an example coming in version 14. This is if we want to kind of. Maybe I should make this a little bigger. If we want to kind of represent a sort of, sort of truly symbolic sum that hasn't been done, it might look like this and we might want to say let's have an approximation to that sum, let's have a truncated version of that infinite sum. This is a function for doing that. Something that came in version 13.3 is contour integration in the complex plane. So this is now sort of combining computational geometry. This is a regular polygon, but it could be any kind of shape. And this is now doing the contour integral of that complex function over that region. So, you know, we're always trying to add kind of convenience functions to the system. And actually one new source of convenience functions is none other than our friendly LLMs. Because when LLMs try and write Wolfram language code, they're doing it by kind of looking at what's out there in the code that already exists in Wolfram language and kind of seeing how people describe it, seeing every other kind of library of every programming language and so on. And sometimes they make suggestions about things that sort of are obvious, things that should be in Wolfram language. And we recently started taking some of those suggestions. So digitsum, which is a function we certainly talked about for a long time, was something that the LLM kept on saying, there really should be a function with this name. So we actually implemented it another in terms of kind of thinking through the language and so on. Another function. This is a function that as we've kind of gone over the years, we've built up these higher and higher levels of abstraction about what we mean by a function and how we use functions. This is a new kind of thing. We have map F over a list. This is comap map of a list of functions over a thing. And it'll be interesting to see to what extent this ends up being a really useful thing in typical programming. That's always a question that first you clarify the idea, then you implement it, then you see just how widely you can think in terms of that idea. Okay, here's a quite different direction. Something we've been building up for quite a number of years is partial differential equations, PDEs, and being able to do planet element method computations. This all very much relies on things like our computational geometry capabilities. So here, but now also kind of a modeling language, this is representing pieces of the fluid equations. So now let's see what can we do here. We can actually compute the velocity field here. And now let's make a plot of the velocity field over the region that we've described here. So we describe that region as the region difference between a rectangle and a disk. So that's a rectangle with a disk chopped out of it. And then we describe the fluid equations here with various boundary conditions. Then we solve these equations. And now we're plotting the the velocity field from Those equations. So it's taken a lot of sort of building up to be able to get to the point where we can do this. We've added, we're sort of gradually adding kind of modeling functions for different kinds of domains. One that, well, I like is Schrodinger equation and quantum mechanics and things. There's a Hamiltonian here. And now we're going to compute the over the stadium shape. We're going to compute eigenfunctions. And now we can make a contour plot of those eigenfunctions and takes longer to make the plot than it did to do the computation here. So there are the eigenfunctions of the Schrodinger equation over a stadium shape. And we're gradually adding these different domains, more solid mechanics, hyper elastic materials, things like this. And it's kind of a sign that there's a big area because each of these individual application areas we have a whole monograph devoted to the use of our language in those particular areas. So that's been a sort of a big build out that's built on things that we've been creating for the last at least decade and a half. Well, let's see in another direction, let's talk about geometry. So this is something we've been introducing in recent years. The ability to kind of represent geometric scenes kind of as Euclid might have written them. But we basically are giving a set of constraints about some angles are equal and some distance is this or that. And then we're saying, okay, given those constraints, make us a random instance of that geometrical scene. Well now a new thing that we can do is we can solve for things in this geometrical scene. So given those constraints there, we're now asking what is the Euclidean distance from A to C? And although this is sort of simple Euclidean geometry, it's actually a piece of non trivial heavy lifting kind of algebraic geometry type work to be able to compute this. Well, The. So that's another type of thing. We've also got things that are happening in terms of being able to generate geometrical proofs. I don't think I can yet show that to you, but that's something that's coming in. So both computational geometry being able to describe computational regions and so on, and the synthetic geometry of kind of Euclid style descriptions of geometrical figures and so on. Okay, another completely different area, system modeling. So here's a system model. This is of some electric circuit. And now we can solve that electric circuit with. And you see here, there's A resistance which has an around associated with it. It has an uncertainty. And so here, now we're now solving this system, this rather simple electric circuit with an uncertain parameter. And that will allow us. It's going to grind away here, solving the differential equations and so on. Oh, it's going to, it's doing, in this case, it's doing a Monte Carlo simulation to try and find the results of that uncertainty on the behavior of the system. And now it'll plot, I guess this is a function of time. Maybe this is a capacitor. I do see a capacitor there. Or we could do the same kind of thing for a chaotic system where we'll see the kind of the error building up over time in this chaotic system as we run it with uncertain initial parameters. And the whole point is that you can do this not just for simple systems like this, but for big complicated systems with 10,000 components and so on. That's the story of our system modeler technology stack, now with the capability to simulate things directly built into Wolfram language and where we can have this symbolic representation, just as we have symbolic representations of so many other things, we now in these years have symbolic representations of systems engineering systems. Let's see, another type of thing that we have is, Okay, so machine learning is getting everywhere. So a lot of functions. So here's just a function for highlighting a part of an image. But now we can say, let's remove the background for that part of the image and we're using now some kind of newer machine learning. Oop. This is showing us that, yes, it is a newer machine learning model and it's a 1 gigabyte neural net which it seems was not downloaded to this particular computer. So we can wait for that to happen. But in the end, all these computations are going to happen locally here and perhaps why. Wow. Why is it so slow to download? It's only a gigabyte. Well, okay, I guess I don't get to show you that. All right, let's go on. We can look at another example here. I hope there's not going to need the very same neural net. We'll find out. Oh, no. Oh no. Well, anyway, this would have been beautiful if I'd been able to show it to you. Basically being able to get these, carve out these actual. The cats and dogs and so on from the image in an effective way. Well, let's see other kinds of things. Oh, I don't know. We've got new capabilities for making videos now. This might again take a little While this is going to make a video touring a Geographics. Okay, here we go. So the. It's now generating frames in a video. And this will be generating kind of an optimized video here that will show us some kind of tour. I don't know why those particular cities were picked, but let's see what it makes. Okay, so here's a video. This is a video embedded in our notebook. And we can see that now it's moving across this geo scene. Very easy way to make a video like that. And we can make quite elaborate videos. I'll show you some that were made with this technology a little bit later related to our physics project that are pretty elaborate. Okay. Another thing that's coming is we have all these different kinds of entities representing things in the real world. One of the things that's kind of new is the notion of hierarchical entities. The first place where we'll see that is in species, where the tree of life is kind of a typical example of a hierarchical setup. We can see among the eukaryotes, for example, we'll see different. We'll be able to drill down to different types here and we can in general do all sorts of things. In this particular case with species. What are we doing here? We are asking for the entity class of taxonomic species which are predators, for which the cheetah is a predator. And then we're saying, give me an image associated with that type of critter. And so this is a collage of things for which cheetahs are predators. Again, I think it's pretty neat that it's possible with the combination of computation and knowledge to generate something like this as easily as this. Well, let's see other things. Another thing that. In a different direction. This is about sort of user interface that we've been adding. Kind of just as if I say list plot, I'll get an autocomplete menu that shows me list plot. One of the things that's new, if I just type lp, I'll also get listplot as a completion there. And that's a convenient thing. I don't know, something like resource function. You can just type RF and hit enter and get that resource function. And then here you can go and open the resource system and go and pick your resource function and so on and keep going. This is just a convenience in entering things for this purpose. So other kinds of conveniences. Well, another thing to. Yeah, let me actually show you a different. A different thing here. And then we'll get back to more sort of typing Conveniences. Here's one that's just a simple thing, but very convenient. This is a very common thing. You might type color function arrow, and then you're about to type fish Ampersand. Okay? And as many of you will know the way that the precedence has to work, that Ampersand will not just apply to the fish function, but will apply to the whole color function when I type the ampersand. Now, why didn't that happen? I would have expected it would have selected this whole thing. I am running the latest version. Am I not here? That would be terrible. Yes, I am. Well, it's a convenient thing that I'm just amazed. Didn't work here. That this shows you that Ampersand is applying to this whole range rather than just to the fish thing. And that will save me a bunch of mistakes, at least just to talk about other kinds of things. Okay, so just making a plot. Okay. We thought we might have. That's a thing that's existed for a long time. One thing that's new in version 14 is this default kind of highlighting of the plot. And this works both on the desktop and in the cloud. And there's a lot of control of that highlighting. So for example here I could say as the plot highlighting, I want the red directive, so when I go to that line, it will highlight in red. Or I can give more detail here. I can say make an X slice highlighting. So I'll now see. And of course, there are lots of cases that have to be dealt with here. And that's why these things are not just, oh, we'll just slap this together. It's a thing that has taken quite a lot of design and effort to get something that will work in all these different cases. And let's see, here's another example. We're just highlighting one particular curve here. And with this particular highlight on the curve. Well, another thing along the lines of graphics is something that's sort of coming in the world is more and more augmented reality capabilities. So this is something where you can publish this to as an object that will now, oops, I think I need to just make that bigger. Ah, okay. This, this cloud object comes with a QR code. You point your phone, camera or something at that QR code and then what you'll get, I think I just have to show you here because you can't kind of see through the phone and so on. What you'll get here is an augmented reality version of that geometrical object now renderable through the AR system of your phone for Example, So it's a very convenient way to sort of get the things that are created from Wolfram language and make them sort of accessible in augmented reality. Let's see. So another thing that perhaps I should show here is something we've been doing. We've been progressively enhancing a lot of things about the documentation of our system. So for example, if I go to something like Plot, you'll see a number of things that are of relevance here. One is, well, there's a lot of complicated options and things. There's a new section here which just shows all the options. So Plot sort of inherits a bunch of options from other functions. This is a way to see kind of all the options, including what it inherited. That's a useful thing there. Another thing that we've added is a lot more cross linking not only of functions to other functions, but also functions to other kinds of things. So graph, for example, you'll see here see also these various functions, but also entity types, formats and so on that are linked from these pages. As, let's see, I could show one more example here. Again, lots of different things which are formats and entity types and so on, as well as just other functions linked from that page. And they're also will be showing up things in the function repository, linked from built in functions and so on. Let's see, another direction has to do with elegant code. So let me try and show, actually I can do that for this notebook here. Let me just do that for this format. I go to the screen environment and instead of picking working, I pick elegant. And what you'll see is all those pieces of code that I had, they all change their formatting. And kind of the idea here is we want to be able to format code so that it is as elegant as possible. So if you put it in a paper, for example, it really will look not like this ugly computeristic thing, but as something which is kind of an elegant thing in the same way that, for example, something like math can look elegant in a document. And so it actually doesn't take a lot of tweaking because Wolfram language already is very kind of word based, it's very easy to read. But it turns out that bringing back a little bit the kind of the infrastructure of the language, the square brackets and all this kind of thing, changing the fonts and so on, this makes it kind of a more elegant presentation of code. And this is something that, in fact, I've already used this in a number of places or previously versions of this in a number of places. And things that I've written. But this is kind of going forward. This is what I certainly will be using when I'm presenting Wolfram language code that just makes it easier to read. And the thing that I think one can use this as a working to do one's work or one can use the kind of monospaced, kind of more kind of code writing type mode in which to do one's work. And then when one's presenting it, one can use elegant code to do that. But this is for people writing books and articles and things. I strongly recommend the use of this kind of elegant code style as a way style environment, as a way to present all the great things that you can express in computational language. Let's see, another thing we're doing is to do with testing. This is now we have had a testing framework for a while. This is a kind of more symbolic testing framework. So this becomes a symbolic test that we can go ahead and evaluate. Yes, it's correct, two plus two really is four. But we can now do that test object that we got out there has many different properties. And for example, we can do all sorts of elaborate tests. We can build our tests separately, manipulate them symbolically and then subsequently, for example, generate a report from these kind of symbolically represented tests. And this is an important thing in terms of being able to do sort of large scale symbolic software engineering. Well, from testing the one's software to testing humans. One of the things we've been adding as part of our whole question, our assessment framework and question framework system, this is something relevant for making quizzes for humans and things. And there's a bunch of technology we built over the last few years for this. There's one recent piece has to do with, well, what does it mean? Is 2x plus 3x squared really equal to x times 2 plus 3x? Well, if you're doing a calculus course, it probably is. If you're doing something where you're studying polynomials, it probably isn't. And so that's a kind of part of the kind of infrastructure that's needed to build this kind of large scale kind of human assessment system in WM language, which is now, I think, something that's become quite sophisticated. All right. Another thing that's become more symbolic is parallel computation. So now we've got this kind of object that represents a kernel configuration and it becomes much easier to set up kind of. For me, I have quite a lot of machines with quite lots of different kernel configurations on them. These can now be sort of described symbolically and manipulated that way in another direction in software engineering we've got a very streamlined foreign function interface now. So let's say you've got a library that's written in some C like language and it exposes itself with a. As a dynamic library, for example. Well, we can now directly specify a call into that dynamic library from top level Wolfram language. Now that dynamic library is a C level thing and it has things like type information. So this is for example a cint type. And this is saying this is a very basic function that just is a compiled C function. And now this is all it takes to load that compiled C function into the running Wolfram language system. And now we're calling it and it's doing all of these type conversions and so on here. For example, we've got OpenSSL, which is specified as a dynamic library. We've got a particular function in OpenSSL and then it has a declaration in C which we're now giving a symbolic version of, which says this is a raw pointer to an unsigned char and it takes an integer and it returns an integer. Yes, that's low level. Most of the time the people who use Wolfram language never have to see stuff like this. That's what you have to see if you're writing C code and things. But you don't have to see that. But if you're trying to connect to a piece of C code, this is super convenient. So that's this kind of symbolic representation of a foreign function. And now we can go ahead. And now this is really, you know, diving down into the machine. This is allocating memory that we're going to send to that C function. We can now go ahead and actually write out. Actually this is importing from that memory. Okay, so that must be importing the raw memory. And then we could call this foreign function. Let's see, what do we do? We call the foreign function on. How does this work? I think we would call it on. Let's see. The problem with calling foreign functions on things is that by golly they could. It could just crash the whole system. Oh no, it didn't. It gave a one. Okay, I'm not sure what the significance of that is, but that called into that C function through this whole sort of raw description of what was going on. But directly from top level Wolfm language and with 100% efficiency, so to speak. Talking of calling external languages, we can also now we have built, when you are dealing with Python, for example, it's got this giant bizarre zoo of dependencies and libraries and all this kind of thing. We've now built a technology for encapsulating instances so that you can encapsulate all the dependencies you need and create an environment that can then be run kind of as a standalone thing using external evaluate from. Look at all this crazy stuff. Okay, this is what you don't have to do if you use Wolfram language and you don't have to worry about. Let's see if it actually works in this case. But this is now building some environment, it's installing some libraries. I don't know what it's doing, but at least we get to just describe it in pure Wolven language code here and encapsulate it so that we can just call it directly from within our language. And this is something where you can take both in the foreign function interface and this encapsulation. God. Wow. You need to do all that to do this. Okay, let's see. Produce one character there. Okay, good. I think there's. I think I have another example here that's slightly more elaborate. This is for setting up syntax coloring. So now. But now having set this up, we can now just say format this piece of code and hopefully. Oh no, oh no. Oh, there we go. It will use that whole encapsulated stack of low level code to do this operation. And there are an increasing number of functions showing up in the function repository and so on, which underneath are using C libraries, Python libraries, things like this. The real value is that they connect with all of the other capabilities of Wolfram language and we're kind of describing them at a high computational level. Takes work to sort of wrap these things and find the correct description and you know, untangle kind of the low level, low level structures to make it something that can actually be sort of validly called from. From Wolm language. But we now have all of the sort of software infrastructure to, to do that. Well, okay, other things in software engineering. So another, another big thing we're doing is making a system for creating standalone applications. So we have Wolfram Engine, which is freely available to developers. It's a downloadable version of the Wolfram language kernel and we are creating a capability that lets you just define an application. So with Wolfram Script, for example, you can say, I want this command line function to run this piece of Wolfram language code. Well, now you can say, I've got this piece of Wolfram language code. I just want to run this. And we'll create a standalone application that includes a stripped down version of the Wolfram engine that is just set up to run your particular piece of code. So this is a way of delivering essentially a runtime version of Wolfram language, but it's been stripped down, so it doesn't even include everything. It only includes what's needed for the particular code that you're trying to run. So it's a way of creating standalone applications using Wolfram language. And you can either distribute the whole Wolfram engine with your application, or you can do something more like a generic runtime. That way you can have multiple applications calling one runtime. So that's the thing that's coming. And let's see. Well, talking of deployment, the Wolfram Cloud has gotten sleeker and sleeker and more and more efficient. And it's something. The fact that we have the cloud allows us to just build all kinds of different things. If we need kind of a playground for people to just evaluate Wolfram language code on the web, we have that. If we have something that we need to call APIs, as we did for the, for the plugin for ChatGPT, we have all of that stuff. The cloud is a very important enabling technology and something that allows users to just be able to sort of walk up to a web browser and use Wolfram language. Let's see. Well, talking of deployment, again, lots of things happening with our enterprise Private cloud now getting much more in step in terms of releases with the public cloud. We have really a whole stack of technologies for deploying our system on servers. And let's see, lots of different deployment options depending on the sort of size and shape of the application that you have. For us and our consulting team, for example, it's really just a. Well, somebody's got some application that they want to run with these parameters. Which of these various different server technologies should we use to do that? And. But we now have sort of all the different cases well encapsulated so that it's possible to do that. Let's see, another thing that's happening is, well, lots of containerization of both Wolfram Cloud and Wolfram Alpha actually to make it easier to deploy in different places. We already have deployment in lots of cloud services of Wolfram Language, Wolfram Engine and so on. More of that is coming so that you can just sort of spin up an Azure instance, an AWS instance of Wolfram language, or of the Enterprise Private Cloud to be able to run your applications against that cloud service. Okay, well, so a big story is the whole LLM integration story. And first thing I should say is that LLMs and the chat systems around LLMs are kind of a new user interface for Wolfram Language and for Wolfram Alpha. It's, you know, intelligent assistants were a thing for a while. Now they're back with more strength, so to speak through kind of LLMs. And you'll see integrations of Wolfram Language and Wolfram Alpha in a series of kind of Chat LLM applications, a whole collection of them that are coming. We also are integrating the ability to call out to LLMs from within Wolfram language. We have versions now for OpenAI and Anthropic, and we have a whole spectrum of different kinds of connections out from Wolfram language for different LLMs. So a big thing is the notion of a chat notebook. This is a sort of new user interface for us. Let me show you that. And in a chat notebook, you're talking not just to Wolfram language, but also to another kind of evaluator, an LLM evaluator. So, you know, draw me a circle that is half green and half, I don't know, purple. That's some. So what this will now do is pull out to an LLM and now it will use that, but it'll write some code for this and probably you never know what the LLM is going to do. So now it's telling me how that works, but that's not really what I wanted. I wanted to actually do it. Thank you very much for that long explanation, but just do it. Okay. All right, let's see. Okay, now it's calling Wolfram Language Evaluator as a tool. And let's see if it does it right. Let's a little bit surprised it's taking that long, but let's see what's happening. This is. Okay, let's see, let's see what happened. Oh, very good. Okay, well, let's say, let's say. Can you rotate that. 90 degrees counterclockwise is see if it can figure out how to do that. And by the way, it's, it's. Okay, let's see what it does here. It's being a bit slow to write this out, but. But anyway, the point here is that one of the things that's important is that we can read this code. By the way, if you press that button, you can execute that code here and hopefully it'll run that and see what it does so that it works one the. Okay. Ah, terrible. Well, what did it do wrong? Well, I think it doesn't know. Okay, so I could say that doesn't work and I can start complaining or I could just copy down that code. And I actually know it's wrong because that rotation, it thought you could rotate it inside there, but you can't. And so now I can read the code, correct it, and with luck. Oh, my gosh. No, no. Maybe that's what's wrong. Oh, that's very wrong. What on earth is that? No, no, no, no, no. Well, let's just go ahead and. Okay, let's copy this so. And we can edit this again. Maybe that's what's wrong. Let's see. Oh, yeah, that. Right. Yes, I think that's right. But what's important here, and maybe this isn't the best example I could give, what's important here is that the workflow is you kind of tell the LLM roughly what you want. It's a bit vague. It may do the right thing, it may not quite do the right thing, but you can read the code and then you get to kind of build the sort of solid piece that you want and then go from there. By the way, one thing I should mention about chat notebooks is that unlike an ordinary notebook, where the line you type is the only thing that is being executed at that moment in a chat notebook, the LLM gets to read everything above there in the notebook. So, for example, unless you put a separator, a chat block separator, in which case. So if you say here, what was that? Is that number prime? And then it will. Okay, now it'll refer to the number 59, but it's not going and referring to things above it. Now, okay, now we can also. Here there are many options for what we can do. You'll see here it's a little bit small perhaps. There are all sorts of different Personas that you can use, all sorts of different models, you can use all sorts of different settings, you can use tools and so on. So let's see, this is one called Bernardo. Let's rerun this. Because we can always rerun. That's one of the nice things here. I can just copy this, for example, and actually I could copy that here. And I could say I put a different number in here. And I could just say, is that prime? And now we're talking to Bernardo, which will now have a different personality. And. Okay, and it'll give us this result belt. You get to pick all sorts of different. Well, there's a lot of. Yeah, a lot of things that are both. So chat notebooks are this kind of exposing kind of chat capabilities directly in the in to our user interface and the ability to get essentially a code assistant that's a useful on ramp for Wolfram language and even something which occasionally I found useful as a way to not read the documentation. By the way, one of the tools that we have is something that allows the LLM to read the documentation. Another thing to say is that when there's an error, like here, for example, let's say what went wrong. There's a good chance that it has seen that. Let's see. I don't think that's the problem. Sorry. LLM, let's say. No, no, no, no. Let's see if it figured it out. Oh, yeah, it did figure it out that time. Okay. I was just going to say it's like, okay, now it figured it out for itself in that case. But what's happening here is that it's seeing a lot of information about errors that. So for example, if I just say, okay, okay, okay, yeah, yeah, yeah. Let's stop it here. Okay. Oh, wow. What did it do there? Well, if I, if I type in something like 1 over 0, I think it will should show us here. Oh, wow. Okay, let's try it again. That's surprising. All right. It will often tell us, Actually. If I say what went wrong and I kind of make a chat block here, it will only look at that one over zero when. Okay, okay, okay, Yaks on about this. Let's just tell it. Okay. I don't know why it's yacking on about these things, but we could tell it, read the documentation. And I don't think in this case it might not do anything terribly useful. But in general, when it's made a mistake, you can tell it to read the documentation and it has a tool that allows it to do that. By the way, I should say that one thing that's sort of a general feature of a lot of what's going on here is the kind of linguistification of kind of symbolically represented things, sort of making it so that it's kind of LLM readable. LLM friendly. Well, and I should also say that the chat notebook capability that I've been showing you here is available not only on the desktop, but also in the cloud, which is a sort of major technical achievement. I should say that there are lots of functions that kind of apply LLM technology kind of under the hood. So text summarized, for example, will be running an LLM underneath. And why is this so slow? Something is going unbelievably slowly with this LLM. This is the life of the modern. You can't tell what the thing is going to do LLM world, it's kind of. I've, I've talked quite a bit about AI governance and questions about how to think about AI and this question of sort of knowing what it's going to do. Well, okay, here, let's try a different summarization of just keywords. Okay. Now it's doing things like chunking it up because it has to send it in pieces to the LLM and so on. Maybe it'll take a while to do this. So maybe we don't want to do this. Okay, okay. In any case, I mentioned that we've got different Personas that you can use from within chat notebooks. We have a whole prompt repository with few hundred prompts. There are Personas that say, you know, act like the Mad Hatter or something. There are functions that do things like suggest a title for this piece of text. You can use those functions. So for example, I could say, well, let's say, let's start here. Oops, that's not what I want. I want this and I want to. Where did I have this? Let's say I want to use a function here. I could say in this, I could say I could start a chat cell and I could say exclam and I could stay informal, Rephrase. And then now I can say in here I can say, request the pleasure of your company. And now it will call the LLM using that as a function. What am I, Earth? Is this my. Oh my. That LLM just did something totally bonkers. How fascinating. Well, actually the. Huh, okay. Can happen. You never know what the AI is going to do. All right, let's see. So another thing I was mentioning was our tools. Oh, I should say, by the way, in the prompt repository, another category of prompt is a modifier prompt that says something like, oh, here's one that translates into emojis. So for example, if I do this, I can say, well, this is. Okay, this is using that as a piece of. That's using it in a programmatic way. You can also use it within a chat notebook. This is synthesizing a piece of text with a emoji translated modifier. Okay, there's. I mentioned we also have the tool repository. This is something that's going to be officially released soon. This is something for the LLM inside the operation of the LLM for it to call different kinds of things, whether it's text retrieval, whether it's. I think it has some web lookup capabilities and other things that are tools that the LLM can call from directly inside the LLM to be used within the overall wrapper of, for example, calling the LLM from within Wolfram language. Well, let me see. So I should mention there are also kits for making plugins for things like ChatGPT that call back to the Wolfram language. So in a sense, you're creating your own interface. So for example, here I am deploying a ChatGPT plugin that calls this API function that can use local information on my computer along with the LLM to produce results. Okay, so now there's sort of a programmatic access to LLM. So there's an LLM function. This is like a function function, except that instead of executing Wolfram language, it's executing an LLM evaluator. And so here we can say rewrite in passive voice, in active voice, and then that's applied to that argument. So we can do all kinds of things with this. We could, for example, let's say we make an LLM function that converts the tense of it tries to make a table of tenses of verbs. We could try doing that. It's going to call the LLM a bunch of times and hopefully eventually it will come back with a result which it will then format in this way. Another thing, let's wait for that to do its thing. But there we go. So now that's using the LLM to generate, in this particular case, some linguistic data. Another thing, you can use LLM functions. There's an llmexample function which says, here's an example of how I want to translate this data. Now let me give you some actual data and see if you can standardize that data according to the example. Something useful for kind of data scraping kinds of things. Okay, so there's the result. So it took these kind of messy the inputs and standardize them in the way that followed that example. So now we can have all sorts of interaction between the LLMs. So this is using the resource function that's from the prompt repository moviesuggest. We're asking it movies starring LLM starring AIs. We should maybe ask it. Wow. Okay, let's try it again. See, this is the problem with using LLMs that you have no idea. I mean, we could look inside and see what it actually produced. Huh, how very odd. Huh? That's strange. Well, it seemed to have gotten the movies, but for some reason it flubbed on converting those into a list to get us the result we needed. Okay, so. One of the things that I mentioned in sort of this is an example of something where we're using, we're trying to sort of clean up and structure textual material to make it more computable. And one of the things we kind of defined a while ago, this kind of series of levels of. Wait a minute, That's really odd. Hold on, I need to. Wow, that's really odd. Somebody must have modified something on my machine even as I have been here. That's very, very odd. Okay, let me try and do this again. Okay, there we go. So we've kind of defined the. We kind of defined these levels of data curation of levels of making data computable from the data has been collected to the data is suitable for repeated systematic computation. And so this is something that we as a company have kind of specialized in doing over the course of many years, both for data that we use for things like WolframAlpha and For Data for outside organizations and so on, used for various enterprise applications. And one gets to different levels in this kind of hierarchy of making things computable with different amounts of effort. And for example, our built in functionality gets sort of all the way to level 10. The things in our data repository get maybe a couple of levels below that. So one question is how much can LLMs help with this kind of hierarchy of data curation and making data computable? What we found is they can help a bit. They don't. It's not like they make all of the work go away. They help a bit, I think in round numbers, kind of a factor of two speed up in being able to curate data. That's a thing that we can readily expect and sometimes it's a lot more than that. Another thing that's been important, we have an enterprise version of W that we built for a bunch of large organizations over the years. And one of the things that's now happening is that with an LLM you can kind of do. You can kind of get to that point more cheaply, more easily because the LLM can basically just take the. Take the internal documentation of a company and essentially just read that to give at least a candidate for how you can sort of connect the internal databases to the linguistics and so on. So that's kind of a use of this. One of the things I mentioned that is sort of a feature of our cooking LLMs to Wolfram language is this sort of meta LLM engineering of hooking all sorts of different LLMs. And so we've been doing lots of testing of LLMs and so on that is all done using Wolfram language, using the Testing framework and such like. And in general, this whole sort of integration between LLMs and Wolfram language is really an exciting thing. It's really sort of bringing the two things that people imagine might sort of turn into AI. The kind of the symbolic approach and the kind of neuro approach, bringing those two things together to get the best of both of those traditions. Now, having said that, in some cases it will work out, in some it will not. You know, LLMs are much less reliable than computational language. Computational language, we're specifying something precisely. It's all formalized. We just compute things. LLMs, as you could see even from these demos that I was doing, it's a little bit hit and miss. And if your application is something where 80% success rate is a win, like, well, I just want to kind of know roughly how to do this, and if it tells me that, I'm happy. If it doesn't tell me, I try again, and so on, then using LLM is a good idea. Idea. If you need 100% success rate, you need to use computational language. You can't just rely on the LLM. You have to have something which is kind of an actual piece of formal computational language code that really does what you want. But I think, and as I said, I think a large part of the sort of story of the use of LLMs is understanding these use cases where sort of some percentage is good enough. Now, one use case we're exploring is, for example, tutoring using LLMs, providing personalized education using LLMs. I would say that the number one thing to say about that project and that problem is it's hard. People say, oh, I've got an LLM. We're almost there. We can build an LLM tutor in two days and release it to the world. It will not work. It will give you, you know, it will teach you how to add fractions in a way that doesn't even work. It will start having crazy conversations with you. It's not easy to take. Take the great capabilities of LLMs, their great linguistic interface, and sort of package them up so that they can provide sort of personalized education where they're kind of talking about a topic in a way that's personalized to a student, where they know about the student, where they can assess the progress of the student, where they can look at the working the student is using to try and work something out, and they can comment on that. That's a new thing. We've never been able to do that before, to actually look at some chemistry, calculation A student is doing and saying and having something that says that's where you got it wrong type thing. And let me explain the key thing that you need to know to get that right. These are things that are potential from LLM based sort of AI tutors, but they're not easy to achieve. And I think with the whole infrastructure that we have together with our actual underlying computational language, we have a chance. I don't know if it's going to work, but we have a chance. I mean if you look back at the history of kind of applying computers to education, not as tools, that's the story of Wolfram Language and Mathematica over the course of many years been a great success. But applying computers as this kind of teaching tutoring machine, it's been kind of 70 years of failure. And I think we now have the chance to make that a success. But it isn't easy. And we're exploring, really doing that and seeing whether we can build something that I think will be important for the world if it can be done successfully. By the way, I should say that one thing we seem to be doing a lot of is kind of strategy and prototyping for companies and other organizations about okay, LLMs have arrived. Now what can we really do with them? Don't do the silly things with them. Do the things that are actually going to work and do things which are as I say, the kind of the use case, the harness for the LLMs. And that's what we've got in a lot of our technology stack and a lot of things we've now built out to connect LLMs. That's what is really going to make LLMs very useful, really be productive as opposed to just kind of a cool demo and a nice piece of kind of hype, but not actually useful. Well, let's see, let me, let me talk about. So I talked a little bit about the world of LLMs. Let me talk about some other things that are, are going on. We put a lot of effort into making WOLM language into, you know, designing WOLM Language, making it work in kind of a, A, a perfect way. Let's see. The. One of the things that has been important in the last few years is that we have kind of an on ramp for Wolfram Language functionality. We something where you can make use of the whole infrastructure of Wolfram language without having to kind of without sort of the final integration and the final commitment to permanence that we have in the language. And so the function repository provides that sort of intermediate point and for Example, as I've been writing things in, whenever I write write something, I've kind of had the rule that when I write a. Where is this? Let's see, let's see, let's go here. For example, whenever I write something, I always set it up so that you can click the thing and paste it. And okay, actually that's using packlets. But very often these things will use function repository entries as a way. So the idea is this is using packlets rather than the function repository, but sort of the same idea that you can click this. No, that's not a good thing. That's very bad, actually. Let's see. Maybe we can just use one of these. I think that's going to do the same thing. Dear, dear, dear. Bad example. How could I have had such a bad example here? Well, anyway, in principle you can just click these and get something which will reproduce the picture. And part of the way that that's being achieved is sometimes there are more complicated functions that it's useful to define. And one can either privately or publicly put them in the function repository. You can deploy a function so that it operates like the function repository, but it is operating out of your cloud account, a publicly shared object in your cloud account. Or you can submit it to the curated function repository. And there are now nearly 3,000, I think, functions in the function repository, 2,872 functions in the function repository. And the important thing there is that you just have to implement that function. The whole infrastructure of how the function is delivered, the infrastructure of. Making use of all these different kinds of capabilities in Wolfram language. That's all automatically there, there. All right, so lots of things in the function repository. The other big story is the packlet repository. The function repository is something which is a curated place where we are making sure that the public function repository has functions that have been reviewed and curated and so on. The packlet repository, anybody can upload something. Anybody can upload a private function repository entry. Anybody can upload a packlet. A packlet is a whole collection of functions with its own whole documentation structure. By the way, one thing I should say about the function repository, it's really good for a single function with documentation. It's kind of a just do this thing. Packlets are for a bigger story. And quite often function repository entries are backed by packlets. So if we're using packlets, the packlets for all kinds of things, this is a way we can just get to a packlet. Here's the chess packlet, and we load up the Chess packlet. And then we now say we load the actual functions here. And then we could, for example, say import some chess game here. And now we can say, show us what is this? Show us the actual moves in that game. Okay, that's not that exciting. But show us graphics of that. Okay, that's more interesting. This is all things that are in the packlet. These functions are defined in the packlet. And that shouldn't look like that way. Why this is misbehaving in this way. But this is now an interactive element that is defined by that particular packlet. There even seems to be a thing in that packlet for playing. Let's see whether this will, what this does. Can we get it to. Oh my God. Ah, what do I do? Well, I should be able to just move here. What do I do? Oh, I can pick this up and move. Okay, great. And then it's going to respond in some way. All right, I'll show how incredibly bad I am at this. Let's see what happens. Oh, let's go there. Splat. Okay, that's an example of a packlet. Let's look at another packlet. This is one. And some of these packlets are kind of things that are being sort of prepared for integration into the core system. Here's a packlet for protein visualization. And this is something that I think is intended for a future built in function, but it's still being developed and it's already available as a packlet. And there's a nice little rendering of a protein. And let's see another example. Here's a direct access into a packlet using the packlet symbol function. This is directly accessing, kind of reaching inside a packlet and in this particular case making a periodic table plot using the space group of the crystals, I guess for each element there. Well, let's take a look at another packlet. Here's another big packlet. I don't know why we're quite doing this, but this is our quantum framework packlet and we have. Let's take a look at what we can do with this. There's a quantum circuit operator and this is implementing that Boolean function. We can go and we could make a diagram of some gate, for example, here. We can then convert that into, for example, a tensor network and start using some of the other capabilities of Wolfram language for dealing with graphical objects. We could also do something where we're converting this into a multi way system from our physics project and we'd be doing quantum mechanics directly using multi way Systems and using all of the infrastructure, both scientific and technical, of our physics project. I don't know what, let me just see, what does this do? Oh, oh, okay. We can also make. This is a causal graph. This is again, physics project stuff of essentially compiling quantum mechanics into lower level structures that are the kind of thing that show up in our physics project. Also, this quantum framework connects to various quantum APIs. I think there's a quite separate issue of just what it means to have a quantum computer. That's something deeply informed by our physics project. But I'm not going to get off topic here by talking about that. Well, so doing lots of stuff, lots of, for lots of sort of artifacts from the future kinds of things. And one of the big challenges is to understand what we're doing and even for us to keep track of all the things we're doing and to be able to make the best use of it and to be able to communicate it in the best possible way to the world. I mean, it's really going to take a huge army to kind of realize the promise and many years to realize sort of the full promise of computational language. It's sort of inexorable future that we get there, but rather fun if we can get this delivered, you know, today, this year, this, even this decade rather than many decades from now. And so, you know, we, we certainly appreciate any help that people can give in sort of explaining what this is and trying to communicate it to the world. Whether that's writing about it, writing books about it, talking about it, whether that's, you know, filling out that place on the web where people are listing kind of how to program this or that thing, and making sure there's a Wolfram language entry, making sure that the Wikipedia page about this or that thing points to the way to do that in computational language form, or makes use of a lot of the illustrations and things that we provide in a way that's appropriate for inclusion in things like Wikipedia and so on. And we encourage people to, to go and make use of those things and spread the stuff that we've put out there. Well, we've been sort of steadily trying to communicate things better ourselves, whether it's okay, this is a live playground for using Wolfram language. On the Wolfram language homepage, we have been adding these kind of core area pages, talking about different kinds of capabilities in Wolfram language. These are a convenient way to introduce other people to sort of the core features of the language. Kind of easy to get, easily accessible kind of description of some core Areas that people find important. We've also been adding to our core documentation, more diagrams, more kind of pedagogical materials so that you can land on a page about some function and learn about that function as well as learning about that function in Wolfram language, so to speak. That kind of pedagogical material is useful actually, not only to humans, but Also to the AIs who've been eagerly reading it. Well, another big area for us is Wolfram U. We have lots of classes, so lots of new classes and lots of mechanisms for accessing these classes, whether it's daily study groups, also things like certifications. We have been, I think one new class here is Introduction to Probability and another one Introduction to Statistics. We have a bunch of classes that are coming that. And we're really setting up kind of for us, this is something we're kind of doing for the world, so to speak, to provide these kind of classes about a lot of different kinds of things. They're made. We are leveraging the technology that we have to make these classes easier to deliver and easier to actually explain the topics. So we've also been doing, our academic Innovation Support team has also been doing quite a lot with individual universities, groups of universities and so on, providing webinars with things like how teaching is getting transformed by AI and computational thinking, kind of generative AI for students, you know, upskilling for the AI driven future. These are all kind of webinars that our academic innovation support team has been providing to particular universities and so on. Another thing that we've been doing kind of in the education area is our summer school. We had our 21st year of our summer school and it's always interesting to see the kinds of projects people can get done in three weeks with kind of the superpower of computational language and perhaps even the more so our high school summer research program. This is a two week program for high school students and it's remarkable what they can get done as sort of a new generation fluent in computational language. What becomes possible in a couple of weeks. Weeks. We've been doing a lot. We expanded our summer research program, formerly known as our summer camp this year. 60 students and we were in person again this year. We've kind of tried to develop sort of a pipeline approach to growing computational thinkers in the world, so to speak. And so for example, we didn't have what we thought was the right gender balance in our high school program, so we started a middle school summer camp for girls. And that was very successful in, in sort of feeding later Parts of our pipeline. We also have our Emerging Leaders program, which is a semester long and year long program for high school students. We now have a version of that for college students as well, kind of providing kind of this, this pipeline of helping people to become fluent computational thinkers. We have a variety of different parts of that program. We have a spring program in the UK that's about this. And we also have now now a computational adventures program that will be launching for the world at large in January. Six month long subscription program for kids, teens, et cetera, to learn computational thinking through a variety of activities. Well, lots of kind of outreach of all kinds that we've been doing. One that I've been deeply involved in is live streaming. This year I gather I have done about 202 hours of live streaming about a lot of different kinds of things. Both our language design live streams and I've been doing kind of Q and A live streams twice a week. One is a ask me anything about science and technology, other ones about future of science, history of science and so on. For me it's helped me to be, I think, a better explainer of things. And I think this is providing kind of a, well, a lot of raw material for lots of expositions of lots of kinds of things. And I've been doing myself, I've been doing some sort of education for kids also more locally in Massachusetts, but that's sort of just my hobby at this point. Well, I've also been doing a lot of writing and one thing that was sort of a collision of my worlds of technology and science was a, a something that I wrote that turned into. That's now this little book about what is ChatGPT doing and why does it work. And that book has now made it into. Well, what's this one? There we go. It's made it into 13 languages I think. Now I don't know how many of those translations were done by an LLM, but it's we actually from the time that I posted the sort of explanation of ChatGPT on February 14, it only took us three weeks to actually publish the book version, which was kind of, kind of impressive. I noticed just today the book won a non obvious book award, which I think is complimentary. The. The. I think the. One of the things that that book kind of forced me to do was to really think about why does ChatGPT work? And I realized kind of that, that what it's really showing us is that there's sort of structure and language beyond what we knew. That there is a kind of not only a syntactic grammar of noun, verb, noun type, thing in language, but also a semantic grammar of kind of how different lumps of meaning get put together. It's something which is kind of a. Story. It's one of the pieces of what I think is the sort of emerging field of LLM science, something for which I think there's a lot of Wolfram language capability to study LLM science. It's kind of a thing which physicists should get involved in LLM science. What is the temperature phase transition in LLMs when they stop making sense? How does the structure of human meaning and so on work, work in LLM space? All those kinds of things. Well, and that's led, by the way, this whole sort of LLM science questions have led to all sorts of questions about AI governance, which I've been thinking quite a bit about. I think it's kind of the few hundred year opportunity for political philosophy, so to speak, and for the modernization of. It is thinking about sort of how do you roll AIs into this whole picture. A lot of interesting and very important questions. But let me not get so deeply into that right now. And in terms of talking about, I've been kind of, in terms of talking about CX and computational thinking, I think I sort of have forced myself into a situation where I'm going to try to write sort of the definitive college course about computational thinking. And hopefully that will be starting to show up in small pieces in the next few months. Well, it's. Let's see, as I said, I've been using our technology a lot as well as, as well as building it. And in fact, for example, earlier this year I was able to finish a project that I started 50 years ago about understanding the second law of thermodynamics. And again, it's really a big story of kind of using Wolfram language. Now let's try this one. Let's see if this works. Let's try and prove my claim that you can take any picture in one of these. I think again, I picked a difficult example because that's a complicated picture to reproduce. Let's see if it works. But anyway, we'll let it run for a little while. Oh, there we go. Yes, it was true. And in any case, for me it was exciting to be able to sort of finish this project that I started 50 years ago about really finally understanding the origins of the second order of thermodynamics. And in fact that was part of that turned into now this book about the second law that Just came out in fact, earlier this year, my effort on metamathematics. This was my effort to understand the kind of the limit of mathematics. When you don't just have the 3 million theorems that people have written down in the history of human mathematics so far, but you think about the kind of limit of the structure of metamath mathematical space and what that means for mathematics. Actually, talking about publishing another thing, we have the third edition of my elementary Introduction to Wolfram Language with some additional material. And we've been publishing other things as well. We've been publishing other people's books. Here's one of the nice books that's come out from another author from our publishing program. And back to, back to kind of the story of kind of things I've been. Oops. I've been working on. I have to say in our physics project I had sort of thought that there would be no applications of our physics project for at least 200 years, but actually I've been wrong. And this whole story about expression evaluation and fundamental physics and the analogy between kind of space like events, time like events, branch like events, and the way that expression evaluation works, and this is a direct application of a physics project to language design. And it's going to lead to some very interesting new kind of multi computational approaches to language design and the use of parallel computation there. For me, it also has a certain degree of closure because back a long time ago when I was building the forerunner of mathematical language, a system called smp, I had an example of this. I think I have. Oh yeah, here we go. In smp, I had tried to figure out sort of the structure of recursive evaluation. When you do recursion, there's a question of sort of do you evaluate? That's what all these pictures up here are all about, is kind of the sequence in which you evaluate a recursive function. When you are evaluating, for example, a Fibonacci function, do you kind of recurse all the way down one side and then go up in a kind of depth first way, or do you do more breadth first recursion and so on? How does that work? Can you parameterize kind of the way in which recursive function evaluation works? And I tried to do that in smp and it was, it was incomprehensible to me or anybody else what happened there. And what I now realized is that what I was doing was directly analogous to picking reference frames in relativity. And now we have a way to sort of, in an organized way understand that through the Physics project. And now we have a way to actually implement these things in Wolfram language. So that was kind of exciting for me, and it's been sort of remarkable to me the extent to which all of these different things that that one works on end up being connected. And so here's a thing that I wrote about this very abstract concept of the ruliad that has come out of our physics project and now relating it to generative AI and looking at the question of what is the space of images that sort of occur in the mind of an AI. So this is what I've been calling cat island. And that is kind of the part of generative AI space that corresponds to the human concept of a cat in a party hat, as it turns out. And away from that point, there's all this kind of generative AI space that represents things which are not yet described by human concepts. One of the kind of shocking things is Even with this AI that's been trained on human concepts, only 1 part in 10 to the 600 of its space are associated with things for which we yet have words. So there's a lot more to explore than what we've explored so far. So just to say something about. I might say something. Something just about a little bit about our physics project, I tried to kind of summarize recently. I gave a short talk. I'm sorry, this today was not a short talk about kind of trying to pull together the things that we've understood from computational language from our physics project. I would say that our physics project, which kind of burst on the scene in 2020, has been doing spectacularly well. I think that we're really understanding kind of every month more and more about how this kind of machine code for the universe works. And we've been sort of breaking off our basic science activities into our Wolfram Institute that's now up and running and sort of studying lots of things in basic science sense. I might show you a little bit just things that created with Wolfram language and so on. These are videos that sort of illustrate aspects of the physics project. This is, if I go back to the beginning here, this is, in a sense, the emergence of space from the kind of underlying structure of things. This is sort of pure computation, building space in the very early moments of the universe in our model. And so this is something where we are representing kind of these computational elements whose aggregate effect is to produce something like physical. To produce the physical space that we experience in much the same way that sort of from discrete molecules, we reproduce the fluids the continuous behavior of fluids. We now know that matter is discrete, light is discrete. We now think that space is discrete and that this is how it works. And that instead of producing the equations of fluid mechanics as the kind of large scale aggregate limit, we produce the equations. We produce Einstein's equations for the structure of space time. But this was sort of a video of the kind of very early moments of the universe. In this model, we can also go later in the history of the universe. Let's see if we can play this. Gosh. Okay. This is the structure of discrete space. And this is showing two little tiny black holes in that discrete space. And space is wiggling around a lot. And those two black holes are getting closer and closer and eventually will merge. And if you look very closely, you can see little ripples of gravitational radiation produced and seems to agree pretty well with the predictions quite well with the predictions of general relativity. These are very tiny black holes. They're very fact that you can kind of see all that other stuff going on in the structure of space, in the kind of dynamics that knit together the structure of space kind of suggests a bunch of things about how this really should work. Well, just to show you another one more picture here, this is kind of a story of quantum mechanics picture. I think we finally understand something about how quantum mechanics really works. Let's see. This is showing kind of the branching paths of history theory in quantum mechanics. And there's this thing that we as observers of this are kind of spread across many branches. And we get to kind of experience this in the aggregate, Just as we get to experience space in the aggregate. And that is what leads to quantum mechanics. There's a whole very interesting story associated with that. Just to this, the. I will say that the kind of underneath all of this, kind of underneath all of these different structures of space and so on, is this thing we call the ruliad, which is this kind of entangled limit of all possible computations. And we kind of as observers of the universe, get. We are part of this kind of this computational object, and we get to kind of of experience the universe as entities embedded within this computational object. And one of the big results is that for observers like us, with the kind of limitations that we have, it becomes inevitable that we observe kind of the core laws of physics that we observe. And so there's a whole story about kind of observer theory that gets built from that which bizarrely intersects with a lot of things observ about neural nets as kind of models of observers, and also about things about Computational language and how that's designed. Well, so let's see, what do I have here that I could also show you? Maybe one more thing here talking of. I think this is a hot off the press picture picture of the beginnings of quantum black holes in our model of physics. That's like a one or two day old kind of picture that kind of reflects the beginnings of actually being able to have the merger of quantum mechanics and general relativity explicitly computable in our models. So that's kind of a coming attraction action. But in a sense, this whole story of kind of these very abstract things about the ruliad, the structure of physics, the computational structure of physics and so on, this all comes back to this sort of big idea that sort of everything is computation. And then the question is everything is computation. In the end, everything is embedded in this ruliad. That's the entangled limit of all possible computations. How do we mine that? How do we as sort of humans get access to that? How do we turn that into technology? Well, that's the story of computational language. In a sense. We can see that as being the attempt to humanize kind of what's out there in the computational universe. That's kind of what we're doing as we build our computational language. We're defining kind of the human description of what's out there. Just as human language started off describing kind of what we saw out there in nature and elsewhere, computational language is now giving us a more formal way to describe what's out there, both in the physical world and in this computational universe of possible computations. So in a sense, that's at a very abstract level. That's kind of the big picture of what we're doing with computational language at a very concrete level. It's the thing that lets you do all the kinds of computations you want, lets you make progress and, and do the projects you're trying to do and so on. Well, I would say that again, it's been exciting to be on this journey for such a long time and with so many of you as well. I want to kind of thank you for joining us on this journey journey and trying to kind of pull some of these artifacts from the future into the present day so we and other people get to use these things and do all this cool stuff. Well, that's what this conference is talking about and hope you enjoy other parts of this conference. I'll have a chance to answer questions and have more discussion tomorrow. So thanks very much and bye for now.