Live CEOing Ep 698: Language Design in Wolfram Language [Machine Learning]

Okay. Hi everyone. This is Orphan Language Design review talking about machine learning for version 13.3. And the advance of things in the world means that we've got all kinds of exciting things happening in machine learning. Image synthesize, how's it doing?

Well, that's in Proto should be working if you want to test it. So this still not. It doesn't have service credits. So if you have used Christopher package and. Okay, so let's see. Yeah, this is using OpenAI. So this still has this thing in place. If you have your key you can use it. Otherwise I can demo it. This is going to use service credits as soon as we set it up. Set this up.

But we're going to provide access to other models than this one, right?

Yeah, there's a method option which I don't know if I ever to expose that because right now we only support OpenAI but we will support different services as soon as they are we hook up against other APIs.

But we'll also support something that is a downloadable thing on our own site. Right. A small model that's downloadable for us, right?

Yeah, we don't have a small model convert yet with good quality, but we are looking for a decent one to add to the pool. Yeah, I mean these are all options on the table. I mean not on the table option on the pipeline.

It seems that that space is evolving quite drastically as we speak.

Yeah, but I don't think we have a good idea. Okay, so I just entered my credentials and it took them. So now what's going to happen? There's a crocodile.

Yeah, so this is like a reasonable. I mean until we have service credit, this is kind of decent in a sense that you don't. You can enter this only once and then you can use all the services with that authentication. So kind of a fire and forget. If you want a bigger image you can say image size or medium.

An evil LLM.

Have you been reading lots of prompts or sort of what people.

A bit. I mean for these things. Yeah, yeah, a little bit. I mean I'm not an expert, but.

Should be called not prompt engineering, prompt hack.

Let's. Let's try something. There's a different endpoint which is variation. So maybe after with the Matisse one, you can take this and put it back in image synthesize and say we Synthesize that image 5 for example and get 5 variations of this one. The model decides. So there are sometimes more variations, sometimes like kind of bigger variations.

I really want to say getting data from. But I don't want to say retrieving data if that's what it's doing. Or is it actually running at that point?

No, he's waiting for the API to answer. Yeah, we could say retrieving or waiting.

For if that's what it's doing. Because maybe the actual transfer time is very small. So. So we don't want to make it seem like it has to spend all its time retrieving the data.

Okay, makes sense.

Waiting for. Waiting for results from. Okay, so if I say text synthesize. Oops. Okay, so why don't I get more here?

We need to decide here what to put the maximum number of tokens. The default one is 16. I put it that 50 is somewhat arbitrary. You can say max item to a stop.

It'll get to a stop token. Watch it. I mean, on ChatGPT, for example, it runs until it hits a stop token.

Yes, correct. But in this case you are paying per token, so it's.

Why didn't that work?

Oh, no, no, that's. We use. Use that argument for different variations.

And so how do I. But we need a way.

Oh, yeah, yeah, yeah. Sorry. Yeah, yeah, we have a way. It's. It's like if you say max item goes to whatever you want, like 1,000. It's a temporary name for the option. We could say I use box items. We have max item, max iterations. We could introduce a new symbol.

I think that that should be the. You know, I would have thought the second argument. Yeah, I don't know.

I mean, this is the second element is the variations. So if you say.

I don't think that's right. I don't think it's right because I think. I think it's either going to run to the stop token by default. Should run to the stop token. I think it's what people expect. I mean, these won't go on forever, right? They get to a stop token, don't they?

Hopefully. I mean, yeah. What Statistically they do. We could say 1000 to be sure. I don't know.

But then we should have. We can have a max items option, but so long as it's not usually hit and so long as it usually is limited by the. By the stop. I keep on saying codon, but I mean, token.

Yeah, yeah.

Pretty good idea, though. That's kind of what it is.

Yes, yes, yes, yes. There are a limited number of ideas in the pool in the universe. Okay. You know, it is very amusing the way that this is. You know, it's sequentially generated. So when it said firstly has no choice but to say secondly, but it doesn't know it's going to say secondly. And yet. Yet we all think that it has a global plan.

Yeah, anyway, that's true.

Okay, are you sure that we want to do variations being the second argument? And by the way, what other arguments are there? Can I see the docs for this?

Yeah, sure.

Source. Okay.

So there's one other use case just before we move to the second argument, which is the inpainting where you can give a transparent image or an image and a mask. You see the third case and a description of what you want, where the transparent part is and that will fill it in.

So in a sense the notion of a pattern here is transparency in the image or something.

Yeah. And is that is a missing thing here? It's like a. It's like in paint. It's literally like in painting.

Right, but so if I were to take. Anyway, It's not too bad. Is there any way of getting higher resolution?

Yeah, image size small, medium, large. You can say large if you like it. You can even put that in and say image size large to get a larger like higher resolution. But it's kind of a super resolution.

Let's try it, Say running. Running computation or something. That's a pretty sinister creature doubling as a leopard while it's at.

Not even truly frolicking. But yeah, whatever.

Hardly frolicking. But then wolves probably don't frolic. But I like the super resolution stuff. That's really nice.

It's not strictly speaking super solution in a sense because it's really, you know, it can change whatever it wants to change, but for many purposes it's fine enough.

Right. Nice. Okay, let me just look at some of the other arguments here.

I'm keeping this purposefully simple because we don't want to tie our hands with the design, which is too, too tight on an API that will change.

No I. Parameters on a sock. What's in a sock?

Well, text image and mask. So I'm hiding the inpainting if you don't provide a transparent image. I'm hiding the inpainting in the association because who knows, right now we have like three things that you can pass. Maybe in two months we have like five things that you can pass to the API.

Yeah. And when do we get 3D images?

Yeah.

Okay, this is really cool. Let's just make sure we get more APIs supported and the obvious ones. And together with the ability to have at least some small model that's local.

Yeah, yeah, we are working on all those fronts actually at the same time. So, yeah.

So this is very much the same thing, except that, yeah, this.

Is a kind of a simpler design in a sense. It has more parameters. I'm exposing two which are the same that we expose in our class Net decoder, which is temperature and top probability cutoff, which is the top P. Nucleus sampling. Again, just to keep the design simple. This stuff is related to sampling, so it's very likely that it's going to be available in other APIs.

Okay.

Random seeding. I put it as future because it's something that I really, really hope OpenAI is going to add midjourney has it for the Image API in order to be able to retrieve the same result if you call the API twice.

You know, I'm not convinced about the second argument here.

It's going to save a bit of bandwidth and if you have a long token. Sorry, long token. A long prompt is also going to save you money because you're going to send it over only once.

Okay, well, these things will be experimental. I'm not certain that that's the right thing. I think this should be automatic or all this max items thing. Okay. An automatic basically means go to the stop token, unless it's more than, let's say a thousand or something. And all should just mean go to the stop token, whatever happens. And we're not responsible if you end up with a, you know, an AI in a loop that's charging you money.

That makes sense.

Cool. Okay. Okay. Are we looking at this now or are we going to look at this separately? The chat stuff?

I mean, if you want, we can look at this like briefly now. And basically we have been meeting with Design to decide a bit what to do here and most of all how to integrate this with Chat notebook. So yeah, design is moving fast right now. The idea here is that we are keeping this as a kernel backend for other functionalities. So we are focusing on conversion capabilities. You can extract a list of messages from the chat object. You can use the create chat on the list of messages to create a new chat object. So it can be used for extraction, conversion and handling all the API part in an opaque way so you can kind of abstract away the specific service you are using.

Right, Okay, let's keep going. Image identify, new network. Great. That's pretty pathetic. Accuracy.

Well, we are not using. This is a much larger collection of classes than the standard stuff like imagenet.

Okay.

This is why it's lower. This is meant to be useful for Kind of like feature extraction and other applications. So we move to a model which is much more generalizable because it's been trained with noisy data and in a teacher student way. Okay, so it's kind of interesting.

What's the performance goal by default?

Quality. I mean, we typically do quality.

I think that's right. Yeah, but that's only six. You said 8% slower.

Yeah, it's not significantly slower, but it has much better performance overall.

The big problem is the first use of image identifier, which is absurdly slow. How do we speed that up?

I just realized that there was one downloadable packlet that contains the ontology that was still using. It was never actually pushed to production, was still using some WDX format. So I updated that to get, you know, better alignment data and move that to mx. So that is cutting like half a second.

Why is this slow?

This is downloading the network itself, I think. And I don't know why this is actually fading for you.

That's the dude who is put in there. Shall I try it again? But. Okay, so how do we not. I mean, I already complained about the speed thing before, but we didn't find that.

I mean, we have. I mean, there's a bunch of things that need to happen. The first time you run it, you need to load the MXnet library and then, I mean all our packages now are mxified and with this thing that I hope it will be much soon or like I push to production soon because the packlet we are going to cut another half second from the, from the.

It's not a half second we're dealing with. It's 30 seconds.

No, it was never 30 seconds for me. If you already have the network downloaded, it is on my machine is like three to four seconds with a fresh.

Kernel, you know, including Framework, first of all. Okay, is all the stuff that needs to be downloaded in the cdn?

You mean downloading the network?

Yeah. Is the network in cdn?

I have no idea. I mean, that's using the source object. We can check. Yes, okay, please check.

I. I think most of that stuff is in cdn, but I don't know.

I remember we had a discussion about like putting the. The models used by internal functions on a fast track.

Well, let's just check that that happened. Who's PMing?

Yeah, Avery, let's double check. Yeah.

Yeah.

Okay, but I assume when you say that it's low, it's not just a download, that is. Or you're factoring in the download as part of that.

No, I mean, so long as I can see an up. You know, so long as I can see a progress monitor, I'm. Okay.

Well, you had a progress monitor there that's under.

Yeah, I did.

Yeah.

Okay, here's a question to you. I was. What's that?

No, I just tried to run it on my prototype, and it doesn't fail, but maybe I have an older prototype. I'm downloading a new one to figure it out.

Okay. Avery says try to build from last week. It took 10 seconds to download and get the result. But was all that time spent downloading, Avery, or was it spent. Yeah, bulk of the time downloading. Okay. But so the time when it was invisibly going off and grinding was minimal. Yes, that's what we have to measure. And Stefan needs to make sure that we actually measure that. Right. Okay. And, you know, if it's. If it's more than about a second of grinding, that's no good. Right after. After the download. Okay. So, Giulio, do you expect it to be more than a second of. I mean, what's it doing after.

Yeah, I think three seconds.

Why? It's too long. Well, it means that we can't use it in an API cascading.

There is the package where the image NTFI is defined, which is very small, and that's super fast to load. Then that triggers neural networks, which in turn triggers mxnet link.

Okay.

And then you need to load the. And this is just to get the code.

Good enough. I mean, so we need to figure out how to make that more streamlined. What is the. What is the issue? All those things are maxified, right?

Yeah, it's just that you need to load the library in memory.

Which library?

Like, let me see, for example, if I quit and I do a get of a maxnet link, how much time does it take?

Let's profile it and find out what it's actually doing, because I don't think this is right. It doesn't take that time to load a dynamic library, if that's what it's doing.

Okay, so getting a max link is like 0.65 seconds. But I can. I mean, let's not waste time here. I can give you a notebook. I did this several times. I have, like, already stuff set up to take the exact time of each piece. I can send you an email with all the details.

But more relevant is let's fix it. Okay.

Okay.

I. You know, and Stefan, could you please make sure that this really is done and not something where we just have a test that says 10 seconds is okay because it's really not make sense. Okay. All right, let's keep going. I have a question for you. If I want to take something like GPT2 or a simple image model and I want to add noise to all of the weights, how can I do that?

I was having a hell of a.

Time figuring out how to do it.

You could. You could extract all the arrays and then do a network place part with a modified new.

Yeah. And how do I find the arrays?

Arrays, like information. I'm trying the syntax now. Information. Let's try the. Net, for example, which is more. NET model. Yeah, I'm just pasting some code now in the chat. Oh, yeah, yeah, yeah. Arrays.

Okay.

So if you replace that with something with noise, and then you do. NET replace part over that model, it should work.

Okay, so you're saying if I take. For example, let's say I take GPT2. This is nice. Although I couldn't possibly figure it out for myself.

Yeah, we try to be super transparent in the framework. So like arrays, and they are like parts, same as layers. So you can just replace them and extract them with the same part syntax. Sorry.

So, I mean, here again, this will give the array. So I just have to map over this, adding noise and then do a. NET replacement.

Exactly. I don't know what we have with that association.

Right.

Yeah. You can do. NET NET replace part if you call. If you call the. NET model Net then you can do NET replace part. Net comma. Information. NET arrays. It would be an oop, but it basically is doing the replacement.

That's cool. It's very cool. Right. That's what I wanted for. Okay. It might be good to show that as an example.

Noted. I'm going to add it in netspace part. Probably.

Yep.

So it's a little annoying always that we need to decide what to do with information. Maybe we need an information section on symbols that support information, like in scope or something. Because. Yeah, it's difficult to document this stuff in the information page.

Yeah, we don't want to do that. Yeah. Maybe it wants to be a section at the end of the details.

Yeah. What is like options? I mean.

Yeah, right. Like we're doing options. Like we're doing the full options stuff.

Yeah.

We have a section about information there and then we could have. That's a. Scope has subsections, right?

Yeah. Yeah, I was thinking about that.

It doesn't, but.

No, no, scope. It does in many pages.

Oh, scope, yes, scope. I mean the examples. Yes. Right. I thought you meant details. Sorry.

Yeah. Actually, I tried to sell this idea to Roger. If we could have subsection in details and options.

Well, I think this is a separate discussion, but. But yes, we definitely need a way to document information. I mean, at the very least, we could have a section that is purely textual, that just list. You know, I. I would say the way to do it is list something like information of dot dot dot comma arrays, blah. So in other words, you have a table where you can visibly see immediately that it's an information. A table about information. Do you understand what I'm saying?

No, no, sorry. What is in the table? Information of net common arrays. And then the results.

No, no, no. It just says there's no net. Or you could say information of net, arrays. And then what we've done is.

I mean, it's kind of like when you have an object with string arguments. It's the same, but you have to use information here.

Yeah, I understand, but I'm just saying when you document it, don't just write a table that says arrays, blah, bing, blah, bing, blah, because nobody will.

Oh, no. Yes. Yes. You want to repeat at every line information.

Yes.

So that it reinforces the idea that you really, really have to use information.

Right, right. And similarly, I would not.

I think it's going to be super tedious. Even we. We did a couple of attempts like that.

I think it'll be okay. I think it's more important to communicate it than it is to be tedious, than not to be tedious, so to speak. I try it. But I mean, you definitely. For the sub values, you definitely want to list them as net of blah, not just blah. Get what I'm saying?

Yeah, I get it. But that's usually what we say in the line before the table.

I know that. I don't think it's enough. I don't think it's enough. I think it's a mistake. I've been very confused by it many times. We can pick this up in a documentation meeting. I'm telling you, I don't think it's the right thing. People browse documentation. They need to see the explicit form. And if we needed to, we could have something where we make this a smaller font or something. Get what I'm saying? All right, let's keep going here. What is this NET model? Dramatic. Speed up. Of what? Of inference. Or what? Or of loading.

Both. We rebuild it using an optimized intel library. Something. But to attempt makes it faster.

Okay.

Across the board. Some layers are a lot faster. Some like in general, it's considerably faster. Like it goes from like 10 to like 100%.

Okay, that's cool. Okay. Pytorch now working for. Net external log.

I would have wished that. The thing is, Pytorch doesn't deliver on their standalone format, so they have something like onnx, but it doesn't. I mean, when you can export to that, it works, but not many models can, like without massaging or stuff. You know, we need to dig deeper in the stack. You don't. You cannot export with a standalone model. Why? We mean, why is it a problem? We discussed that about hugging face. Because the default workflow when you distribute a Pytorch model is that you link to some Python code that you have to run locally to create something in memory and then load in the weights which come in a parameter file. So this is extremely cumbersome to represent in an external thingy, because not something that you just point a library to something and you run the code.

But can't we replicate that behavior by running using a Python link and so on?

Yeah, yeah, yeah. I mean, we can, but it's something that requires, you know, is what do you put in that External object is a much more complicated design because it's not, you're not pointing it to some binary, you are pointing it to like a repo. And what, what, what do you run in that repo?

Well, I mean, sounds like it's like a packlet.

Yeah, basically, yes. They distribute networks as packlets.

Right. The important thing for us is that we can put in the neural net repository, we can put models that are. Net, external object models, how they work inside, so long as they work reliably. I don't think it's so important how they work inside.

I mean, I agree with this, but the thing is, I was really hoping for something that could effortly, you know, convert Pytorch model to, you know, quickly access thousands of them, not, not one of them.

Just because we have a certain standard for doing software engineering does not mean the rest of the world does. So, you know, okay, I get your point. It's probably much messier, but if we can provide something where it will at least work, even if it has to go, you know, grinding through some, you know, loading this, you know, running this, etc. Etc. Etc.

That's still a. Yeah, but wait, wait, wait. That means that you need to have your external evaluator set up on your system with Python.

By default these days?

No. I thought we were given the amount of error I get every Time I misconfigure it.

No.

Oh, maybe not. I don't know. I mean, maybe it's me that's done.

Can you talk to other people? Because I thought we were shipping some version by default.

Okay. It wasn't the way with the ship Python interpreter with the system. Okay.

That is what I believe was planned. I mean, Todd would know about it.

Okay.

Check some recent ECE meeting.

Okay, yeah, that would simplify things greatly.

Right? I think it's going to be. I mean, Ricardo knows about it, I think, and Todd knows about it. Okay.

Nowadays I can just bang on Ricardo's door and ask him. So he has no excuse.

What's that?

Because I'm in Rome again. So.

Yeah, yeah, yeah. Okay, fair enough.

Okay.

I won't make anyone in Rome jokes. I don't even know what they'd be Anyway, never mind.

Okay. Okay, so netstate Object, this is a very nice concept that we had. It was able to basically convert. I mean all our layers are immutable. This thing was able to convert recursion layers into mutable things. So you could basically do the efficient thing very, very easily. You just wrap netstate Object around the network that contains layers with recursion and then you can just keep applying the same thing over and over and it will carry all the states inside. You don't have to do any modification, any unfolding, any tracking. The thing is this thing was never updated for two things that are present in transformers, which is you remember we saw we look at that sequence index position layer, I don't remember how it's called. The one that is basically telling you the position of a token in the sequence. And the other one that is important is the attention layer. So now we are updating both of them to work inside NetState object. And the end product of that is that you could say netstate object of the GPT2, for example. And then the result of that you can get the efficient thing where it doesn't have to be evaluated over the whole sequence every time. I think it's very neat. It's just one command to go from the stateless to stateful.

I'm still a little bit confused. Give me an example how you'd use it. Like for. In GPT2.

Okay, so imagine you want to your blog sentence, what was it like? The interesting about AI is the ability to. The best thing about the AI is the ability to. So you're passing this to the model and the model is generating one token. Then you need to take the longer sequence and pass it back. But the model has already. The model is causal and it has already computed on up to those tokens. It just needs the extra bit in terms of new data, but it needs also all the old data because all the attention layer inside they are doing the remixing. So you can do a trade off. If you save the intermediate product of all the attention layers and this is what nested object is doing, then you can just apply netstate object to the single new token and this way you get it faster.

Right. So when we actually have things in the neural net repository, are we Planning to insert NetState object things into them somehow.

We can show it at this point. I mean, if it was a netstate object, it's literally a one liner. So.

Well, so in KPT2, what do we do? So how. How do we turn GPT2 into a mutable. NET state object kind of creature?

Well, you just. You just wrap it. You just wrap the NetState object around the. NET model. If you open the documentation page for netstate object, you see, you see how it works? It's actually quite easy. Okay, we see the first example, for example, is NET of a basic recurrent layer.

You know, you really shouldn't use AT signs for these. You really should use square brackets.

Okay, well, I think, okay, I agree about this, but I have one point to make about this stuff. We had this stuff like right composition and left composition and this came up talking about data set and piping operations. And that's a very nice syntax for piping operation. But we never show it in the documentation because it's kind of too exotic. People know it.

Yeah, I agree.

So there's kind of a balance between showing too much and not showing cool stuff we have. So it will be forever new if we never show it. But anyway, so you see, in this case it's just one line. The state Object is just NetState object wrapped around. NET and then you just keep applying it.

That's cool. So could I do this right now? I mean, if I say no, this is not all.

In prototype we have the position thing merged and because we have a new developer and we're training him on this, be done by freeze time on attention layer. We already have it working the branch.

But. But like the example I had of of netlist nest list of, you know, GPT2 applied to a sentence, adding words and so on. You're saying that with this that would happen much faster.

Exactly, exactly.

Is it the same. Do I type the same thing with nest list applied to. I mean, how would I Do that?

Well, no, because you don't need to join it. Yeah. It was a nest list where you were joining in the input. You were joining the. The past with the present.

Yes, yes.

In this case you can do the same without the string join. Okay, so it's still going to be a nest list because you want to show all the intermediate. If you want to show the intermediate steps. Otherwise just. Just nest.

Yep. But that's really nice. Okay.

Yeah, I think. I mean, this. This week I kind of forgot that we had this. And while I was talking to Matteo, we realized that actually it was just a matter of exposing state, because these are not really states. But we can generalize the concept of state, which is like an input port or an output port, but it's kind of a side port. You can use it in the. Net graph if you want, but by default it doesn't show up in input or output because it's like it's wrapped around. But this is a very powerful concept and netstate object is basically just making this stateful. So I think it follows nicely. Also the way you explain it.

Sounds good.

Okay.

Dendrogram tree. So this is making an actual tree.

Exactly. We look at this a few months ago, I brought it to kind of a state where it's kind of. It's working well. I added a lot of stuff around tree. So I'm also meeting with Jan to see what can be folded back into tree. Because you see there are like new ways of doing edges and new layouts. You can do circular layouts, for example, right?

Oh, yeah. I mean, these things should be in tree for sure.

Exactly. This is what I'm saying. Yeah, we want to.

Definitely. These should be in tree. Those are nice. I mean, the real test is when you have significant data in each leaf, which you're starting to have here.

That and also when you have a lot of leaves. Right, well.

But which tree has this elision mechanism? Right. Which presumably you're inheriting.

I'm just calling tree with a special.

You do understand that there's an elision mechanism?

No, no, no. Yeah, I. I haven't used that in practice, but I've seen it.

I'm sure you can.

No, no, I've seen what it's doing.

Hopefully that will allow something. Huh? Try another tree.

Okay. I didn't know it was on by default.

Basically, I thought it was on by default.

What if you go to a thousand or something like really big.

No, I mean, you'll see the elision at a particular node.

Or maybe it's when you have a single node with a lot of children.

That'S what it is.

Okay.

I'm just surprised we're not seeing any illusions here. But maybe that's just the nature of a random tree that it tends to branch.

Well, yeah, it would be extremely rare that you get all the children on a single node.

Yeah, but.

Yeah, okay.

Right. But you should make sure that you can appropriately inherit that and that your styles don't collide with that illusion mechanism.

Mm. I mean, ideally I want to backport all these tiles into tree itself as much as possible.

Right.

Okay. So here, this is like initial, we started looking at all these statistical functions in attempt to kind of take ownership, like slowly but steadily. So the first thing is going to be if you scroll up where you have data, you see that in the possible form of data, R, you're just seeing the beginning of it. You need to scroll a little bit more down. Yeah, this one here. So you see, we are basically making it compatible with predict and classify in terms of. So we have a lot more like we had all the syntaxes ready for data set and associations, but we need to have some more thought in terms of how this has an effect on model specification and variable specification, because we start also like series and solve what you give the function and also the list of variables, which doesn't make much sense with association and data set. So instead of rushing this, we're trying to make a solid design, but for the time being we have everything built out and as soon as we decide on a way to do it, and then we are going to start basically overhauling all this typesetting, making modern typesetting with like association properties, kind of like bringing this into like modern times.

But by the way. But yeah, something, something. And this maybe is not for this meeting, it's more for the user assistance meeting is, you know, when I mouse over a thing like this, I would really like to get some kind of autocomplete that shows me the properties I can compute from it.

The properties.

Well, I mean, you know, there's some sub values here, aren't there? Yeah.

This is also something that I thought again while I was dealing with the chat object. The fact that we have in this case, you can pass a value or pass a property. I don't love the fact that they use the same way to get done.

Well, information is our way of doing properties more generally.

Because, I mean, in this case it's safe because properties are always strings. And right. Right now this thing takes only. Only like numbers. But what about categorical variables?

Yeah, yeah, no, I know, I know. So maybe we should go more towards an information based thing and grandfather in the few sort of string properties that are done with sub values right now.

They're not a few. If you scroll up to the details section, you'll see.

Well, it's sparse on the space of all possible strings.

Yeah, that's true. Okay, so you see this is here something I sneaked in.

Yeah. I think that there are lots of things to modernize about these functions, including these. So in those days they often come in threes. Like if you wanted the values, we needed to produce the values. If you wanted it to look good, we needed to have something that now.

We have associations and all these kinds of things.

Exactly. Which we didn't.

Right.

And then there are other things, you know, there are other things that we're coming to. You know, it's like this set of properties here. It's like getting an a la carte menu. You can get your, you can make your own dish by combining each ingredient. And that's fine to have an a la carte kind of menu, but I think we need some standard reports, effectively.

No, I agree. Like an over is. An anova table is a standard report which now we have a nice way of dealing with, with data sets.

Exactly.

Yes.

This could be like, we typically call it summary.

Yeah, I think, I think that that's, that's part of what we need here because like if you think in terms of workflow, that's what you want to arrive at. That first small set of items is probably what you're going to use all the time. Then there's the a la carte for someone that wants to really, you know, special needs, etc.

Of course, a new, a new strange thing that we need to worry about for these summary boxes and so on, which is the linguistic serialization of the summary boxes. New and different thing we have to worry about.

Okay, I see what you mean here. Yeah, well that's kind of. We have output form. Yeah. Which is how it renders in a terminal.

Yep. It's very similar.

We should start reasoning, you know, if our output form should be not just, you know, fitted model, open close brackets, dot or dot, something like that, but.

Output outperform the old.

Oh, you mean the old.

Yes, I mean that.

Yeah. And I mean that is very important because you know, you don't want the full form of the summary box in a scripting environment.

I do love the way that these concepts, you know, output form is invented. Ok. So that is a bit goofy.

Well, that is literally what it is. Which is a bit ugly. But if you do output form of Lynette, for example, you should see something more like very elided or something like that. Yeah, so this just tells you the number of layers. But this is extreme. But my point was that maybe this.

More useful than seeing the input for the input form is a giant mess.

Exactly. But this thing should kind of grow maybe slightly, not too much to be still kind of reasonable in a terminal environment, but also contain enough information to be parsed.

Right. I mean it is amusing that we're in 2023 and things that we built in 1988 for terminals are still relevant. But that's.

Yeah, well, well, you know, terminals are still here. Repls are still here.

Yeah, I know repl's are here. Okay, let's just take a look at some of these. What is the output from pose estimation?

It's a list of. Sorry, what type of object?

We don't have any kind of symbolic pose representation.

No, unfortunately not. It's just a list of points at this point. Sorry.

Okay, Is anchor free object detection, is that relevant for image cases?

Yeah, I mean the end result is going to be the same is just a different model under the hood. So in the end you still get boxes out.

Okay. And these are relevant for ocr. I mean, this one is something we're expecting to use for the.

Yeah, exactly. Exactly.

Okay.

The current models, best models are pretty heavy. So we are training something in house to be kind of both small and fast on a mobile environment.

Yeah, that's good. What was the time scale for that to be finished?

I was hoping to wrap it up in terms of the beginning of this year. But what I was still hoping for existing models to be kind of good or small. But yeah, we saw something like a month ago. We are using it to train a new developer. So it's kind of going a bit slower than if medium is lower.

The training model is used to train the new developer.

Yeah.

We can do it both ways.

Yeah, exactly. I mean we have like people training. AI is the people.

Exactly. Right, Good.

Whatever.

Yeah. Yes. Okay. Stable diffusion. Will that be a reasonable sort of, you know, downloadable.

Yeah, this is what we hope for. So right now we just wrapped up a model which is very nice. It's called Clip, which is basically using transformer to do. I don't know if I show this to you. I think I only showed Roger and Shadi. I mean you're familiar with the classical classification. You have a softmax at the end and you have to pick a class.

Yep.

But now with the, you know, you can. We have powerful embeddings for images and sentences. So kind of a different way, like an alternative take on classification is that you compute the embedding for one image and then using one of those multimodal architectures, you also compute the embedding for a bunch of sentences, like a chair, like a nice bird with red feathers eating a worm. And then you can compute the distance between these embeddings. And this is kind of a different take, but I think it's very interesting because you are not limited to a number of classes. I mean, you can still fall back because if you compute the embedding against a thousand classes, then you can pre compute it and then fall back on the classical softmax thingy. But you can be a lot more inventive in how you classify stuff.

How are you combining the embeddings for the different, you know, the bird eating the worm, et cetera, et cetera, et.

Cetera, with the image one?

Yeah.

The model is trained to produce embeddings with the same dimensionality, let's say like a thousand dimension. So then you just do the cosine distance between the image one and all the textual ones and you find out which one is the closest.

But how are you mixing together an image that has, let's say multiple animals in it? How do you mix the embedding associated with each animal?

No, you don't. I mean the image gets a single embedding. So that's up to the model, what it says. I mean this is still for classification, not really for object detection. So you kind of describe the full image, not parts of the image.

What's automatic speech recognition as opposed to manual speech recognition?

Oh no, that's the name of the task is speech to text.

Okay, fine.

And this is a nice, nice model that works like it can also work with other languages we're testing. This is nice. We are going to push a bit on audio models. I also want to do text to speech after this.

Yeah. So how many languages? I mean our existing speech recognizes only English, I believe. Is that true?

I believe so, yeah.

It'd be nice to have multilingual. And what's natural language ocr?

Oh that's, that's, that's more models to do basically text. Text recognized.

Do we yet have handwriting?

I don't know. This is not in truck in our team.

Okay.

We are kind of like doing kind of a plumbing here sometimes providing networks and stuff. Like that.

Fair enough. All right. Good stuff. I should run along to something else here, but this is great. This is lots and lots of nice stuff here.

Okay, thank you.

All right. Let me just check one thing and then. Great. Okay, see you all soon. Bye for now.

Bye. Bye.