Workshop: LLM Tech

Okay, so this is. I'm not going to go in presentation mode. I think it's better. This is kind of a working presentation, so. Okay, so this is to basically help you all, like get set up and play with and understand our LLM Tech stack technology stack. And so to do that you would need like, for example, an OpenAI API key or anthropic API key or like any API vendor access token. We don't expect all of you to have it. So I think every. Do you have one ready that you can post in the chat? So Avery is creating one that, don't get cocky, that will be disabled in one hour once the workshop is over. But this way, if you don't have your own, you can still try that. So let's see. Hi, Dimote, thanks for coming.

Hi, everyone.

Okay, so you see this is pinned, I guess. Is it sticky? And you see this API key, you can just copy it. How do you use it? Well, there are several ways you can just create a new connection to OpenAI. You don't need the new if you never did it, but this will force a new one to open. And this will open a window like this. And you can just paste the key here and this button here saved. This connection now is not super useful because this, this will stop working at the end of a session. But in general, this will store your key securely in your cloud account, so you don't have to put it in every time you create a new service connection. So this thing here I'm doing explicitly, all our LLM functions are doing this implicitly. So it will still work the same. It's just done under the hood. Another way, the one I use personally to, to store the key, easy. Just set system credential. I think this works by the out of the box on Windows and Mac and on Linux you have to set up some keychain provider. But I'm not super familiar with Linux, so I cannot really tell. And so you can say this string has to be exactly that, the OpenAI of Yikey and you just paste it here. Another thing I forgot to put in here, if you're not in a notebook, but you launch the Wolfram language from script from a console, you can set up environment variables that will be read in. And so you can do it the same way you can say environment this. So you can even set this in the terminal and this will be read in by our functions. So we have what we call an authentication flow. So we try to read this and we try to read this and then we Try to check if you have safe connection in the cloud. So we try to be quite flexible in finding at least one connection that works if you have set up one in the past. Now there are breakout rooms in these workshops, so. Oh, I don't see my screen anymore. Am I still sharing? No, no. What happened? It was kicked out. Share screen share again. Hello. Okay, so let us know if you cannot set it up and one of us can open a breakout room and do some debugging. 101. Can people hear me? Rohit cannot hear me. Yes, we can hear you. Okay, so it's a separate issue. Sorry Rohit. Maybe try to debug in the chat. So okay, once you are all set up, what you can do, let's start high level chat notebooks, very front end based. What you can do here? First you need to open one. How do you do it? Well, you have two ways. You can go up here and do new and open one of these two. And for the life of me I never remember what is the difference? Can somebody else here that is in the know explain what is the difference between these two? Which will go away soon I assume.

Chat driven notebooks have a default input of a chat input cell and have a default Persona of plain chat. That's the main difference.

Okay, well this one here is a code writer Persona. Yeah.

And you need to use the quotes to have new chat entries. So chat driven notebook is really like chatting in a chat room. Whereas the first one is having a code assistant more or less.

Okay, cool. So for the rest of the. I wasn't aware of this for the rest of the presentation and while I clarify concepts that you just mentioned. So if you don't know these words, Persona, assistant, don't worry, I'm going to use this one here, which is the one I've opened on the side. And I'm going to show things here in a moment. Secret and you can use shortcuts to open one command option N on Is it option? Did I copy the right one? Yes, it is option okay on Mac and control option N on Windows Linux. Which thing I can show by clicking Bam it open in my other screen but it's here. So this is what you get and you can say hi there and you get stuff out. Amazing. Now how can you start a new chart and if you have an empty one you press quote? Yeah, Anton, you're right. We could have asked ChatGPT about that. So we have like two shortcuts here that are like quotes and until the. What are those? Oh Sorry, this is my bad. Copy paste. This is ins. Sorry, correct. Live. This. This line, insert chat section. Who's called chat section. Break.

Chat block. But chat section.

Or like, okay, let's say start a new chat block. This is probably fine. So what does these two things mean? Let's check out here. So here I go. Let's see, I go here, I type this and I get this, and then I can start typing. Now I've already typed something because this stuff here is slow and unpredictable, at least in Italy, where I am. So here they tell me one thing about association. And this starts writing, what is association with Wolfram language? Here we see that we do something nice. And this is kudos to Rick who's here. This is all parsed nicely for you. So this is a symbol. You can just click on it and it will open a documentation page for you. This is actual Wolfram language code. And you can do a few things. You can click this one here and it will just copy and evaluate. This will just copy it. And this one here will just copy to the clipboard. So you can paste somewhere else like here and let's undo this here. Or if you click on here, like paste evaluate, bam, you get the output out. So yeah, as I create a new one, interact with it some formatting you can follow up. If I say what is the name of the function you told me about? And this is going to say, the function I mentioned is called association. Now why ask this stupid thing? Because I want to show what the delimiter is. If I press tilde, I get this thing here. And what is this thing doing? This is resetting the chart history. So this is useful because every time you type something in the input field, everything that happened before is sent to vlm. So this not too long, but it might get long or you might want to ask some unrelated thing. So now if I ask what is the name of the function you told me about? It says, oh, I don't have any context. I'm not sure what you're referring to because this thing is completely invisible to this call, which can be very useful. So we have a bunch of time here. We have one hour. So I'm going slowly. Stop me if you have any question. I'm trying to check in the chat. So yeah, just let me know here. Okay, so let's go back to our things here. We mentioned Persona. What is this Persona concept? So you can LLMs, they will read what you write and try to do something based on what is written. So you can shape the Conversation by putting in specific text in the prompt. Now conventionally having a piece of text that describes how the LLM should answer and sort of giving it a personality is called Persona from lading, I assume. Not Italian, but somebody unmuted. Do you have a better, a better definition? No. Okay, I'll just go with mine then. So Personas can be, you know, like always act like a, you know, funny comedian or my boring history professor or speak like a robot or something more useful like give precise answer using the least amount of words as possible or be verbose and formal while you write your response. So these things can be used to shape the tone of the conversation. Let me show here stuff that you can do here. Okay, so if I click here, you see that I have some Persona that are pre installed. So this one is I added. So sorry, I added it later. But these are the ones that you find by default. So this thing is a bit more verbose. Yeah. Rick, while this is more spot on on the code, I haven't played extensively the second one.

That's right, code writer tries to stop all the, all the chatter.

Okay, thanks. While plain chat is not about code per se. So you know, if you don't want stuff about code, so let's try, try some difference here. So if I say code the distance and say my usual set one, like write three colors, let's try British spelling. So LLMs are extremely lenient with spelling. So if I do this. Yeah, it still works. But you see, this is very focused on the Wolfram language, which is what this notebook is about now actually, let's change it so it doesn't have any history. If I go up and change this to plain chat and evaluate. Bam. So this is very generic. Now this has nothing to do with the Wolfram language per se. So you can also have normal conversation and ask about ideas for gifts for your boss. So let's try live. This should be safe enough. Give me some gift ideas or my boss birthday. And this starts writing, oh, very stereotypical. Which is typical for LLMs. Okay, so yeah, this is like normal chart, nothing to do with it. Okay, we have some feedback here. I don't see any turbo model options in my notebook. These ones I think these depend on. I mean first the version you have of Chatmoto Packlet and also the key you are using like GPT4 is not available for everybody. Still, I'm not sure it used to be the case. But all the models that are available depend on these two things. So let's go on a bit. What you can do you have default Personas. And you see they have a new one here, text support bots. And this I added simply doing. Basically let me do it from scratch. I did that. And this opens this nice window. I can start typing and get auto completion. So I did this tech support bot. If you don't have it, like I only install this one, but I have like Travel Advisor. This will take a moment because it's installing it from the web and now I have it. So let's go back to the tech support bot. Now what I ask is how do I turn my computer off? And this gives helpful advices which are acting a bit like tech support. But this is probably not going to give personal opinions. And this is more like.

Why don't you ask tech support what boss? What gift to give your boss? See what it says.

Oh yeah, this is very good. Let's see the power of. Of prompting plus stupid questions. Well, this is, you know, this is. I think this prompt is. Is not over specifying that it doesn't have to give personal opinions. We could update this. Now some people are asking, oh yeah, every pinned another thing which is packet install Wolfram Chatbook. Like this command here. I'm also gonna paste here so you can see it. So this one here will make sure that you are running the latest version of chatbook and pick the latest updates. So for these packlets about LLMs, we are doing something a bit different than our usual release model. It's more like a continuous deployment strategy. So you don't have to run this. Of course, every time once something is tested, we put it on the packlet server. So you just load it automatically. It's downloaded automatically on your system. This is just to force it. Basically. Now I think this is probably enough for Personas. There's funny things you can do with Personas. Like we have our one and only Bernardo. And I can say, oh, what happened here? Bernardo space. Yes. Hi there. Bernardo is very funny and moves. And then I say, I don't know, what do you think about, can I do this? I hope so. I think we can do this. No, this is tech support bots. There was a way to do this. Bob or Rick, I'm doing this wrong. Well, I leave this here so people can actually guide me in how to do this. I'm more on the programmatic side of things. So all the fun stuff. Oh, don't use it. Okay, Well, let's redo the session. Okay, so this we. Okay, okay. Yeah, yeah, yeah, yeah. And this has to stay Bernardo Added.

Manage Persona and then select.

Oh yeah. Okay, let me show you live. If I want this to be persistent, I can do this menu. Bam. This one is here. Okay. And now I can select it from the menu. Okay. Oh, okay, cool. This is what I want to do. Great. Okay, so we've seen how you can personalize your chat experience using Personas. Okay, this is what I showed. I was a bit fast, but this is all here in the handout. Cost of Tools so tools are an extremely powerful way to extend alignment capabilities. Basically they teach the LLM how to call external APIs to get the answers that they can then use to craft your answer. So GPT4 works with tools out of the box. I think we have some hacks to enable them in GPT 3.5, but GPT 3.5 is not so great in following tools. GPT 4 is much better because it's been fine tuned for that. One thing to keep in mind is that if you are on a budget calling tools involve multiple lamb calls, at least two because you have tool call. I mean you have the part of the answer that calls the tool and then you have to another call to the LLM to actually do something with a tool result. So keep this in mind. Now how do you use tools in here? I set these two. I set this to GPT 4 previously and now I can also redo this. So this one here by default if I go to this menu add and manage tools, you see that the default code assistant has this. Sorry, my thing is a bit small. It has Wolfram Alpha enabled documentation search enabled documentation lookup. So it has a bunch of things it can do and evaluate worth language. It it doesn't have web search or web image search by default. So I mean I can enable it and we see what it does later. So now if I evaluate this, what is the stock point for Alpha right now this should be calling WolframAlpha. I did it like three times out of three. Yeah, four times out of four. So now you see that this is actually asking WolframAlpha. So even if the LLM doesn't know because it was the training Data stopped like 12 years ago. You can still use WolframAlpha to get accurate data about contemporary things. And this is November 2, 2023. Bam. And tells me a bunch of things. So this all got from the wolframar vi. If I click here I see what happened and you see this is a tool called input stock price for Apple. This is what happened and this is what it gave out. So this nice formatting of course is not what we send to the LLM. What we send to the LLM is this row thingy. So you have a tool called Wolfram Alpha inputs and tool call result and you have this. And these things is parsed by the LLM which is then used to put this nice textual format here. So LLMs are actually great at rephrasing information. So if you can. One thing to keep in mind is you are never sure they are not making up the things that they tell you. So I mean it could be that we send this out like + 3.5, 2.7% up from last close and I think my screen stopped again.

Yeah, stopped sharing.

I don't know why. Do you see the screen share icon below? Okay, yeah, but it's the second time it happened. I don't know, didn't happen to me in this presentation. Whatever. So I mean you see like these things here, they are used like here. But of course we have no control. Like the LM could make these numbers up. So wal it's extremely powerful, extremely convenient, but also, you know, a bit shaky. So this is, I'm telling you because you have to keep it in mind. And we advertised clearly that the tool was used because it's also a liar sometimes it says yes, I'm calling WolframAlpha and I'm telling you this but. But it never did that. So this way you can at least you have to open it and verify. But if you want to be super sure, but at least you know that the information actually came from Alpha. Now what I did is I enabled this web image searcher here. So in here if I create a new. Let's get a new section just to be sure and a new section like give me an image of the crazy monkey. Now these. Okay, so I think you need to.

Click OK in the menu to the left.

Oh. No. Oh, it's not here. It's like embedded. Let's try now. Okay, nice. I don't want to run this. Let's try to force it. Use the tool. Let's try this. Sometimes you can force the bit. Okay, let's see if any help from.

The chatbot people here enabled.

Oh, I create a new section. I'm stupid. It's crazy face template. Yes. I'm not going to do this live. I'm going to do this after I stop sharing my screen. Okay, now you see GPT4 is using tool by default. So bam. Webinar searcher and we get the crazy monkey out. Ready to stop. Okay. Okay. This is still safe for work. Okay, nice. This was broken. I don't think LLM fault. Oh well, underwhelming. Okay, one out of three. But we got actually a real image from the web. So like the quality of results depends on what's on the web actually and if the images are still there or not. Broken links I think. But this way you can see the power of this. Yeah, you can. This stuff here, what it was doing before, it was trying to come up with a way to write code to do those. But actually what you want is, you know, typically image search and you can do this embedded in the notebook. So you can just copy these and do, I don't know, color negate on these bomb. Now let's go ahead. Oh, I have one more thing that I want to do in here. Here. So I wrote this function which I use sometimes actually is a nice idea. Expression color. So what is this doing? This is taking an expression is like a connoisseur thing is adding unevaluated. This way if you pass an unevaluated expression here you can, you can hash the unevaluated expression. So you know one unevaluated is stripped here. This one here stays unevaluated. It's computing the hash. So if you pass 1 plus 1 it's completely hash of 2. But if you. Let's actually do this. If you do this you get this nice color. If you do this you get the same color but if you do this you get the hash of one plus one. So you get the hash then. Yeah.

This is hash U from the function repository to spin us with a sneeze.

Yeah, I mean I used this way before the function repository was a thing. Sorry, I didn't know. I never stopped checking. But thanks and well anyway this, this creates a color using this balance upalette and you know I could probably add more. More intervals here now why I wrote this, I wrote here a brief description in the comment and I'm asking can you please write few unit tests for function above because we should always test code but it's boring so I'm asking the LM to do it. And here some very basic U ideas. These two are the same. These two are different. Let's actually check. Yeah, nice. So he's not using the whole test create business because it's new. He's not even using the diffusion test which is a bummer. I was expecting that. But the boring part is coming up with ideas and this case too. And every told me ask for more tests. So here, test for different symbols. Check that all true. This is actually a very nice use of a function. But. The function outputs are RGB colors. And then this one here is saying the function is deterministic. This is actually very nice test. The color for a particular symbol expression is always the same. And this is testing it using the, the actual output of the function. And this approximate equal which while it was generating people told me, oh, this is hallucinating the name of a function. This function doesn't exist. But actually the definition is here. So it created some test utility for you which is not too bad. This is L1 distance in RGB using a tolerance. That's nice. I guess. Let's put it here. Does it work? Oh yeah. I also need this one here. So evaluate and then this. Why is that Happy? I do have up equal.

Does it generate RGB colors?

Oh, this is lab color.

Yes.

I didn't know. I didn't know. Poor guy. Balanceview is returning lab colors. So I'm gonna actually just change the definition here. Color convert rgb. Poor guy. And now not here, but here it will work. Yes, actually very nice.

You could also try to have the LLM fix the mistake, but that's probably a long story.

Should we. No, that's right. That's right. This is nice. Let's do this. This was the last interactive example. I mean everything is interactive but non programmatic use. So we have this, we do this. Give more test. We evaluate this, we evaluate this. And what did go wrong with the test? Let's try. Oh.

Well, actually not that long.

Well, no, no. Who sent their applause? This is the opposite. This is the opposite. What is true? X Pro color returns LEV and Apex SQL expecting rgb. It's the right problem.

It's the right idea.

Yeah, exactly. So yeah, okay, let's, let's stop this.

Is that explanation based on the fact that you changed the code and it's reading the south bomb.

So now. No, no, but I, I change it back. I change it back.

Oh, you change it back.

Okay, I changed it back. Yeah. I don't know if I actually left something in here that confuses it, but. Well, let's try it once more and then we move on. This one is correct now. Well, you know, it got confused. Understandable people even like real people have difficulties wrapping their head around color spaces. So.

But that is an important point to consider because when you do this, it's not using evaluation, it's history. To figure out what has happened is using what's in the notebook at the moment that you evaluate it. So if you've gone back and changed things higher in the notebook and not reevaluated, it has like a false sense of history that you know, it'll.

This is very correct. Bob's audio is a little muffled for me so I'm just going to repeat it because it's a very important concept. So by default in a normal notebook, which is I bring you the normal notebook, If I do 1, 2, 3 and then I evaluate 1 and then I ask for the last evaluation, I get one because the spatial order of these, it has no importance whatsoever. It's all important in the same cell. So if I have like this and I have sorry, two times this actually this is very boring. Let's say six. Yeah, no, then this actually is important. But this is a single expression. Yeah, but have you seen in this case if you read this and you see last output, I get one because it's not this last output, it was this because I went back and reevaluated this chat notebook works differently. Everything that is here literal is converted to string and sent to the LLM as history. So if I change something here, I literally change history.

So.

Yeah, this is something to keep in mind is quite a different mental model compared to how you're working with normal notebooks. Okay then let's see what's given to a kind of changing topic. Let me take like a two seconds break here. Is there like any question that is unanswered? What are my, my helper saying in the chat? What about new functions? No open data? Ah, no, I don't think so. So is asking about functions that we added in 14? I don't think, I don't know. Have we solved that yet? Well, I mean the documentation lookup is looking up things in the documentation.

So they have tools, so through the tools.

But if you do the plain chat and you ask for something that was added recently, of course that is just relying on the knowledge base of the original training and it's not going to be. There. Is an LM tool the same as an LM plugin. I would say that they are close. I mean the plugin concept is for the website, the ChatGPT website. But they are very similar in spirit. Yes, I'm going to cover LM tools. So yeah, the answer to the last question is in the rest of the talk. So yeah, 8 out of 19. So let's go ahead. Programmatic usage. So starting in 13.3 and 14 and onwards. I don't think we're gonna drop these any soon. We have added A bunch of functions to basically have this experience but programmatically through normal code writing. So given that we have several, I just listed the main ones here split by the task that you know, why should you want to use one or the other? So starting from simple to complex like basic text completion we have LLM synthesize so you know, prompt in, text out. Then parameter based completion is LLM function. So something that quacks like a normal function but is using an LLM. I'm going to detail all this later, so don't stop me if it's not clear, but just wait a second. Example based completion LM example function. This is for few shot learning programmatic chart section compared to interactive notebook based chart section. Chart object chart evaluate. And they have few specific functions that are application based and are built around image synthesizer is not LLM but it's transformer. So like very similar technology. And text summarize it is UN14 which is going to grow dramatically in the next versions as soon as we finish building all the infrastructure that we need. So let's look at this thing a bit more in detail. So basic X completion, very easy, you know, LM synthesis, the best movie ever is, you know, this is exactly what you could get here. But you know, string input, string output, normal function. So this is very boring answer. Very, very polite. You can ask, you know, least free strange colors. It's gonna, you know, do things roughly formatted. You don't have any formatting here like in here, like this custom code or things. This is just plain string suitable for your own manipulation here. Now this is very basic. What if you want. I mean sometimes it's what you need, but what if you want something more? LLM function is basically the marriage between LM synthesize and string templates or template objects. So this can be used to turn a prompt into something that is a bit more like a function. So here I create like my. I assign it to a symbol and I say a LEM function the opposite of is. And then you see this is wrapping. This is wrapping a template. And I think I can even take it out. Like let's say if I do information. I hope we implemented this. Yes, we did. So we have a template object which is exactly this. So nothing crazy going on here. And then I can use it like a normal function. You know, like this is powered by LLM trademark sign. So opposite of dark is light. Opposite of anyway just map. I can just use like a normal function. Up, down, left, right. So this is a way to encapsulate this functionality for functional programming. Now I mentioned also example based or few shot learning. So few shot learning is a technique where you have few examples in mind of what you want your task to be and then you teach the LLM through examples is like do as I do, not as I say. So here I do this very typical sentiment analysis. I love it. It's crazy. This is positive. This was okay, but it took a while to get my orders neutral and I'm disappointed with the product quality for the price I paid. Not happy, negative. So this is creating something like this is resolving into an LLM function but it automatically does this input output thing and your input will go here and it kind of queues the LLM that it has to put your output here. So now if I say this is a right blah blah blah, it says neutral. It has learned from the examples. So why this is useful. Sometimes you don't exactly know the right words or you're not sure about how you would like to explain this better. Or you have to be verbose, say use only one word. It has to be positive, negative or neutral. An example is worth a thousand words. Sometimes a clear reason why you would like to do this is like I asked for colors before, so let's do it slightly differently. So I created a list function which is list two animals, cat, cat, dog, list four cities, Paris, Cairo, Bogota, Tokyo. And I'm using the same format here. And now if I say list three strange colors is the same thing I said before. Now it respects the format and this is useful because I can use now the second argument of a LEM function. For example delimited sequence string to call interpreter on this. And now this. You know, with this new definition it automatically becomes a list of strings. So this is a list of strings that I can use programmatically which is, you know, in a programmatic environment way more useful than this or sorry for the scrolling than this is interesting, but maybe not what I'm looking for. If I not, you know, interactively reading.

Did you try it with the limit is even smaller.

No. What do you want me risky business. Does it know about these strange colors? No, I didn't know about this.

Worth a try.

Worth a try. Yeah, these are strange colors indeed. So I'm not surprised. What's periwinkle? I don't know. I would never know. I didn't actually bother reading the output. Poor guy, I'm not even paying attention to the LLM. Now let's look at charts so you can do the same Thing here. Let's say that you want to build your own application. You don't want to use chat notebooks. You want to embed this into your own application for a customer or for students or for whatever your own statement. So you can use the same technology programmatically. And we created two functions, basically one that is used to instantiate a stateful. No, I could be stateless. Sorry a stated object which is called chat object. And this is stateless in the sense that if I do this and I say chat, this is not changing the original chart. So you see this gives out. Oh yeah, but the original chart was changed before. So let me start from scratch. Yeah, never use state kernel. So I get this thing. Oh yeah. And I also did. I did too many things while I was setting this up. Do it again. Okay, well let's just quit. I did stupid things. There's nothing pre baked here. Let's just start from scratch. Okay. Periwinkle blue. Thanks, Danny. Okay, now this is plain. What is the highest mountain is Mount Everest. Blah blah blah blah blah. Now if I look at chat, let's put it here. This is still empty. So I can use my new favorite syntax which is evaluate and apply to. I think it's called this. So it evaluates this on this and redefines chat to build it up. So now if I do this, this is using the empty one. And now I say oh yeah, I asked where is it? Because it didn't tell me before. So let's see if I can. Oh no, I have to reset it. Well, we can go on. I'm quite stupid. Where is it? Is it any malayas? I can keep asking and keep adding it. So this is. Where is it? Where is it twice. But yeah, you can basically, you can basically do the same as this with your complete control over history and you can check things. I think it's called history messages. Yeah, you get everything when it was evaluated, what was the prompt, A few things. I mean this is all here for you in all the gory details for programmatic assets. So now I was playing with this evaluation thing.

Can you also show usage content?

Bam. 388 tokens. I think we have also full usage. Let's see how much was history and how much was completion. So the other one was the sum of this. Thanks Tim. So what was that evaluator thingy I showed before? We've introduced this concept of LLM evaluator. The idea is taken from the fact that you know, like if you press on a button or like A cell or dynamic expression. They can use a specific kernel as evaluator for that code. And we want a way to define what the LLM evaluator is for one of these functions. So my default one was now better reset. It is OpenAI GPT4 but. Let's do something different. So like here for only for this one. This is using Anthropic Cloud Instant 1. It's like a fast model from Anthropic. So let's actually re evaluate. It should be quite fast because it's fast. There it is. So you see Entropic has like a Claude 2 which is slower model, a bit more powerful. Claude Instant 1 is very fast. They are both very verbose. So I ask Rizzi Highest Montaigne and I get this out so we can use the configuration to plug in prompts. So I had one here answered in a brief and concise way. And you see the output was only one sentence. So you see how similar to Personas prompt are the superset of Personas. They can really help you shape the conversation. You can, you know, these are just strings. So you can put whatever prompt you like. And if I, you know, go to the prompt repository and I grab the numeric only prompt, it tries the best it can do and it answers with the height of the of the montage. But it actually respected what I what I asked. So you know, this is again not 100% guarantee, but if the element is good, it will try to respect the prompt as much as possible. So yeah, I had the dollar elevator here. What I want to show is if you're using some things over and over, like for example, I want always Entropy instant one and I always want brief and concise answers. I can just take the configuration and assign it to this $llmvaluator which is the default for these functions. So now if I say LM synthesize what is montaigne it is actually briefer than I can try to get. Let's evaluate this and it should always be brief. Yeah yeah. Now stays here. Now other settings that you can do well, settings that affect sampling like most of you could be familiar with. Temperature is it temperature zero will make it repeatable if they implemented it correctly because he's actually always picking the argmax in the probability space. And you can put a hard cutoff like stop sampling after you picked five tokens. Tokens is not really words but sort of words like the unit of thought of these LLMs. So it says it 1, 2, 3, 5. I don't know what was it? Yeah, this is one token per word and then it stops. So of course you can put this as like 1000 to be sure that it never goes above 1000 or 9000 and you can do the same with the stop tokens. Keep in mind that we are going to tell you with a message not every API supports every combination of parameters. So in this case anthropic supports both. So I can say stop at commas and periods. This is still I left temperature zero in so I know that it's going to write this sentence and it stops here and if I remove comma it's going to stop at China. There it is. So tools, somebody asked me in the chat. So finally we get to that answer. You can use tools programmatically. So here I am writing using lmtool expression expand use interpreter. This is going to be an expression and I'm just going to expand it. So very dangerous. But I think interpreter is sanitized against something like quit. And yeah, this sorry, sneak peek into the future. Look at this. So if I call this, it says it's calling OpenAI then it's calling OpenAI again. And this is gets nice series expansion, sorry, polynomial expansion of X + 4 to the power of 4. Now this is something I did like a moment ago. I wasn't expecting it to work, but it works at least sometimes. I define this 0 Expand and I also give it a description compute Taylor series. So the RLM can actually read this and know a bit. And this has more things like this has an input expression, an order which is an integer and a variable which is an expression. And what he's doing is evolving these things. He's doing series of input in variable from zero to order, is normalizing it and is converted to input for strings. So I can get, you know, I don't get those full form plus minus. I just get, you know, these symbols in this is what I want the LLM to read. Now if you ask compute the heavier approximation of W sin W to the 10th order. And I pass these with, you know, I put in these two tools. It should be let's do it again from scratch. It should know that it has to call the right one. There it is. So I'm asking here a few properties. You see, I have a second argument. I can ask for the full text but it's going to be very long with all the prompt to call the tools. So I'm just going to give you the completion text. This is the basic answer that I would get with the default property that you get out if you don't put anything here. And this is also parsing the. The text that went back and forth to identify where it called tools and put these in nice formats. So this thing here is us parsing the text and telling you what happened. So at a specific part of the prompt or actually of the answer that when I use it as a prompt in the second case it calls use expand with these parameters. And then you got an answer that was this. So the answer to this request was this response, and this is the output. And you can see that this is exactly what it used in the end to produce the answer. So again, no much magic. But yeah, this works. Oh yeah, there's quite a. Like, if you don't know about this thing, but now you know There's a generate LM2 response that is very useful to debug this stuff. So what you can say is you can do series and you can take the request and put it in and this will actually generate the response. So, you know, this thing went through the whole LLM thing. But this request response was just a Wolfram language evaluation. So it has nothing to do with the lamps. So you can actually. I mean, this is what we use in here under the hood, but you can use it. You can use it manually here then. I see there are a few questions. So did I spell highest wrong? Danny? That's highly probable. Don't let non native speakers do these things. So specific applications. I'm going to show it very quickly. Image Intelligence. This is a workshop about LLM tech. And yeah, people are pretty happy about it. No, they are not. This guy is unconvinced, but everybody listens to him and text summarize. So this is some engineering at LLMs is taking text, splitting it, chunking it, counting tokens, sending out, getting the results, combining the results, redoing this if it's too long. So all this I did when I had anthropic as a default. Anthropic always tells you how many words the absolute is. I actually never counted, but I don't know. It's OpenAI.

Usually wrong.

It's usually wrong. Okay, so OpenAI. That doesn't do this, but anthropic does. So, yeah, you can do things like keywords. And this is kind of built in prompts. You see, this was five chunks. This is the whole book. So this is doing things for you basically, so you don't have to bother checking things. And then of course, in the options you can specify how long do you want it to Be this is a nice. Okay, this is. I optimize the prompts for OpenAI too much and now on Tropic I really have to force not giving the description. So this is the bug for me. Well, this is not too bad honestly. But I don't want this description. So you see this is actually showing you one of the risks of working with LLMs is not a program is not going to give you the same output every time and of course it is an API so you are reliant on what another company is doing. So caveats. Caveats. Just keep it in mind. So this is the end. I have one minute but we can overflow a bit so things every day. This. Can you also show us how to reset the key? Well, let's go all the way up about magic. So this one here, just you know, set it to something else or do this and it will go away. I'll not do this because my own key is saved for the one saved in the cloud. You can haha. If you went to my service frame of thing, you know that the service frame is not super documented. So you have two ways. Either you do this again and this is going to do this and you can just save another one or and bear with me. Service connections, saved connections. OpenAI I don't know if I actually saved any. No, I don't but yeah, this will list all the not the keys explicitly. We'll list the connections that you have done and then you can do Bear with me again. Service connections delete connection and this call here I can actually. How do I answer you every compared to Q and A. Now it's in Q and A. And how can I paste it? Wait, Avery. I converted it to Q and A. You disappeared from. Oh no new. It's not unanswered answer paste send. Okay, sorry. Live. So yeah, this is how I deleted then other things. Oh, nice philosophical discussion about tokens and words. Well, let me just evaluate this for the sake of your argument. Very useful. Thank you LM and thank you Danny. Then I can click on. Sorry.

It's much harder to get him to misspell things than misspell things.

Right Bob, I cannot really hear you well, but. So this is what recreated. Oh, nice definition examples use the control lookup verification test. This is much more beautiful than what I did. Perfect. So yeah, this is in the chart. Lot of tests, few failed. Is it me or no fix? Okay, great. Nice. Amazing. So yeah, you can. If you put a bit more effort like they say, you get out what you put in. And this might be true for a Lemma as well. And on this note, if there are no more questions, I think we can wrap it up.

Sounds good, thanks.

Well, thanks for all of you attending and well, have a nice end of a conference at this point. Bye.