Live CEOing Ep 770: Language Design in the Wolfram Language [Optimizing Table, Multi, and More]

Sam. Hi everyone. Welcome to another Wolfram language design review, part of our incremental language development series. And we have a few topics today. What is auto batching? I don't know what that is.

Okay, so yeah, this is. We have a handful of primarily network based functions that have to do some sort of automated batching for things like entity value, to a lesser extent parallelize, which Tony has.

We have this batch applied concept. Do we not have that? Yeah, don't we have batch applied? Do we never implement that? How do we do this? What?

We discussed the design years ago, but we never implemented it or decided to implement it.

But what was. Wasn't that primarily for database lookup?

So the idea is that switching between map and function, you put something like batch applied and specify how big the batch size should be. And that would, instead of mapping the function over, you would supply a list of elements given the batch size.

Okay, I'm a little confused here. I mean we've got, and I assume we don't have Roman here. Do we have Roman here?

We do not, but Carlo could you see if he's available?

All right. Okay, so I'm, I'm can remind me, I thought batch applied was a thing that keyed into the, you know, relational database etc lookup mechanism. Am I wrong in that recollection?

I think so. So this is so far has been only an internal concept into entity frameworks as an internal utility. Yes. It is basically used in situations where you have to supply a handler function and you can specify that this function should be applied to a list of elements instead of a single element at a time. That means it allows you to reduce the number of handler functions that you might have to handle. Either one for list or one single element.

Okay, let me see what you have here. I'm somewhat confused.

So this is yet another proposal.

I'm sorry.

So, yeah, so this is basically when you have something like a listable function, but you notice that supplying the whole list is too resource intensive. Let's say it uses too much memory or timing. And then you might want to break it down not to one element batches, but to some intermediate batch size. Now, and that this function handles calling the batches, but also making the batch size smaller if there's a failure.

Okay, I'm somewhat confused. So, okay, so Raman is here.

Sorry.

Okay, there are functions like entity value that if I'm not mistaken. So, so walk me through entity value and how it deals with batching.

Yes. So it's an effectively listable function and you can Supply an arbitrarily long list of entities in the.

So let's, let's try this. So let's say something. Whoops. I say, you know, countries in Europe. Let's say. Okay, and then I'm going to get a list of countries. And now I'm going to take this and I'm going to say here, I don't know what number of airports. Okay, so now you're saying that this will auto. That will automatically do a batched lookup. Is that correct?

Yes, in this case, all at once. But as soon as you're getting in the hundreds, then it might break the things down in smaller lists and put a result back together. Which is easy in the list case, but harder because you could specify a list of properties there. And then you have some kind of outer product of entities.

Okay, so let's look at a case that I was looking at. So let's say. Okay, so if I say alkanes here, I just did this yesterday, then. And I say here, so I could have exploded that with entity list. It'll be a few hundred items, right? And then it doesn't. Why is that wiggling like that, by the way, when I mouse into that? I mean, I hate to bring up a different thing, but that wiggle should go away.

It's kind of wiggling.

Okay, that wheel should go away. Okay. In any case. Oops. Oh gosh. Ah, this is the case. Okay, look. Okay, watch this for a non batched situation. Okay, so let's say I say this, it's going to say floop didn't work, right? It probably should have worked. This is the thing we should report to the chemistry team. Can somebody do that please?

Yep.

But this is the. Okay, so now let's try this. I don't know why it says isn't the list of molecules, but anyway, I think, I think if I say rest because that for some reason liquid methane is not. Yeah, I don't know what the heck it's doing here. Now on the other hand, if I.

Say.

Molecule graph, slash at this, It will work, albeit very, very, very slowly. And by the way, do we have the version of monitor up and running now that works in this case? Can I try this? Okay, so that didn't batch, but do I have a hope in hell of getting it to batch? Do you understand my question?

Yes. So there's something in the works for the case of entity, but in general, how that would know which things to collect, that would be hard.

Yeah, but I. I think that molecule graph, I mean Something is wrong here. But, but in this case, Well, okay. By the time the function is listable, it's up to the function to deal with making sure that it's efficient when it's given a list. Do you agree?

Yes.

Okay, so what is the case that you are handling here? The function is not listable. Right. So if it isn't listable, I don't understand how you ever feed it bigger chunks. Do you see what I'm saying?

You mean entity value or the functions that don't support lists of entities?

Well, you. Are you saying this thing is only for entity value?

Yes, that's where it's currently used. I mean the, the application. The, the idea here is to have a framework to make batching more accessible. Something like the progress monitor framework, where other functions could hook in and say, okay, rather than molecule plot rolling its own batching system. If it in theory works on a list of entities, but may want to batch them together into reasonable size requests, having something like batch map with retry, even if it's an internal utility, would potentially be useful.

Okay, okay, let's cover that in just a second. What was the statement about monitor with one argument with map? What's the status of that? Because that's really an important thing.

It's in the prototype build. Julio and I need to meet to work out some details and because of scheduling and me being out, that hasn't happened. So it's in the prototype build, but you know, it's, you know, it hopefully will be merged not too distant future, but it's a, you know, it's not going to be in the next few days.

Okay, okay, fine. Fine. Great. Okay, let me try again to understand what you're, what you're asking for here. I don't get it. How, how this thing, I mean, what do you need to have for a function so that it can use batch map with retry?

So in, in the case of Entity framework, it, it handles batching for you. Now let's imagine someone is trying and a service connection or something like this. Then they might have to handle long lists of requests and then they would implement their own batching. Probably start with partition of up to 500 and then worry about failures. And all of this logic of handling failures is baked into batch map.

Okay.

Okay. So what you're dealing with here is listable functions. Basically, you are giving listable functions smaller lists to eat than they would naturally have. Is that correct?

Yes.

Okay, so this is a, you know, I might have a function F of, you know, range of 100 or something. And what you're doing is instead of feeding that all to F, what you're doing is you're saying I'm going to feed this in chunks.

Excited?

Okay, so this is somewhat complementary to Parallel Map. Right? Right, yeah, same general idea. But now you could imagine Parallel Map doing the same thing that it says I'm not going to send these individually. I mean, or that no, it isn't quite Parallel Map, it's some kind of parallel thread, parallel ma, you know, listable. It could say I'm going to do one of The F. The first 20 things in this F on one processor that the next 20 to another processor and so on. Am I making sense?

Yeah, I think the obvious use cases for it are things where communication overhead is a factor. So parallelism anything network operated and where we can't necessarily process everything at once. So I think parallel submit and wait all is another kind of like batching. A manual batching system, if I remember right.

Yes.

I mean Parallel Map and friends do that kind of load balancing by themselves. So they usually hack the input into certain pieces and send those pieces to individual kernels. And so that is under user control. So whether you want small batches and more control over all the scheduling or whether you want maximal batches and minimal communication overhead.

Okay, but so right, so I think for anything. Right, right. This wouldn't be useful for anything where we don't have, you know.

Sort of.

Some sort of external constraint, I think. Right, because you know, parallelize talks to other kernels. I mean here we're in.

Exactly, we're talking.

We're in a single kernel and. Right. The kernel is not magically going to become multi threaded after exactly years. And so, you know, if we have some sort of external constraint like here we're talking out to the alpha servers, you know, sort of controlling the batch size. Makes sense if we're just doing a computation in the kernel. Well, may as well just what kind of parallelism?

I mean, if you have a single kernel, then what is the. I mean, I'm lacking some background here, but what is the issue? I mean, what do you gain by.

Batching things in a single kernel? Good question.

I have an example. For example, the GEO functions can send arbitrary lists to the geoserver and in many cases we will just get a timeout. So a framework like this could be helpful in organizing an option or a wrapper function that allows the user to say, oh my connection is bad. So send them in batches of 10.

Batches of 100 well, but where's really the limiting factor, the things being sent to a geoserver? The geoserver has a timeout per query. So if you send it, you know, a thousand places or something to process in some way, it's just going to time out.

Exactly.

So essentially this is a throttling mechanism because you don't want to send the whole thing, you know. Now you could say the geoserver itself should queue those things, but this is asking the client effectively to do the queuing. Right.

I think it's giving the user the control of how many because the user may have experience and, or may want to control things.

Right.

And right, right. And, and the second factor, right. The reason this is beneficial, Right. Is that there is a startup cost. Right. The reason why we don't always send exactly one place at a time is that, you know, then we're bearing quite start up and shut down costs.

Whereas just so I understand that the, the semantics here, basically what's happening is you've got this list, you're breaking it into chunks. You send a chunk, there's a blocking operation to send the chunk, you wait for that chunk to come back, then you send the next chunk. Is that correct? Yes. Okay, so that's in a sense the exact opposite of what's happening in parallelize, where you try to send the chunks in parallel.

Correct.

Okay.

And it's a way of sort of auto negotiating the chunk size.

Yeah, it dynamically rescales it. So like if you ask entity value for the populations of every country, those are relatively small values to come back. If you ask for the polygons, the internal logic in entity value will rescale the batch sizes because it asks for 100 and it times out or something like. I mean, it's more intelligent than that in that it knows roughly what size some of the properties are. But if you're getting a lot of data back, it will trim down the batch sizes automatically, which is kind of what you can see in that echo example right above what you have highlighted where it starts with a batch of three and then for some we're simulating a failure and then it scales it down to 1 and then batches of 2. Sort of like what URL.

The logic behind that is that if you fail, then scale down a lot and then if batch is coming back fast enough, then you might still try getting up a little bit again. No. So you divide by four and then you multiply by two and see, until you reach a certain threshold.

Yeah, okay, okay, okay. I don't understand why this batch might with. I don't understand why it says map of F there. Why isn't it just F comma, range of fidelity?

This map of F is just to simulate a listable function. Because batchmark with retry expects the result.

To be a list have map in the name, though. Because it's really batch apply with retry.

Yes.

It's similar to subset map, where you map over subsets.

No, I don't think so, because. No, I mean, look, the conceptual thing that's happening here. Okay, first point. This only works with listable functions, correct?

Yes.

Because it needs a function. You know, the requirement is it's a function that can take a list of any length as input, correct?

Yep.

Otherwise this is dead meat. It's not useful. So now what this is doing is it's asking. I mean, I really would have thought that this should be a headdress to the function itself. That is that what it should look like is much more like, you know, batch applied F comma. And then here it can be the batching parameters of, you know, whatever it is. Range of 20. See what I'm saying? It's really a headdress to that function, isn't it?

I think it's either that or a common option like parallelize that can be used in multiple places.

Parallelism isn't an option. Parallelize is a function parallelized. Is that right? What is the option?

I think there was an option, right?

Yes, some internal functions use. Use various options to indicate internal parallelism.

Yeah, but what about. How can you set up. Can you set a function to use internal parallelism or not.

Only those that support it. I mean, they.

Well, what are. What are those functions? What's the option called?

Sorry, I was confused. No, I don't think there is an option.

It's.

So. It's just. It's just like the old compiled option to plot, which probably nobody will ever set.

Parallelization as an option just for the old compiler, as far as I'm aware of.

What the heck is that?

Oh, yeah, that's.

That's old stuff.

Yeah, I've never seen it, really.

We should do a tie on that.

Symbol.

Make it possibly available for future use.

What does it even mean? I had no idea that was there. Okay, That's a separate thing to work with, Tom Roman, I think that's something for you to go chase.

I mean, that's not the current compiler, right? That's the old.

No, I. I know, but I don't even know what this does. Let's not even worry about this. What is this?

I think this is activating or deactivating parallelization.

Looks like it looks like the old compiler already had some support for parallelizing. Generating parallel code.

I didn't know that. Maybe nobody ever used it. Maybe nobody ever saw.

I used it all the time, but it's been years since I used it.

Roman, could you investigate this and just see what the story is? I mean, maybe we've been. Maybe we set this up, you know, 20 years ago, before most people had multi core computers and we've just never enabled it. That would be crazy. Let's find out. Okay. In any case, what we're looking at here is something, I think more like this, isn't it?

Is that three your starting batch size?

I would assume so. Or something. I mean there's some options here for batch, how the batching works and then this is just essentially an adverb, a headdress to say how F is getting applied. And it's saying use all of this fancy technology to, you know, do what otherwise would be a pure now, I mean in the C. Okay, so I don't really understand why. Oh, I see. The map thing is pretty weird because that's going to do it element by element. Anyway.

I wonder whether we have to worry about individual arguments like batching in the first argument or the second argument or something like that, like entity value.

I see, I see, I see. What do you have to say about that? Which argument are we applying this to?

Yes, so. So that at the moment that would be in. In this design, it would be handled with a function that takes one argument. Yes.

Yeah, but entity value is not such a thing. I mean, entity value takes two arguments.

So. So this one is the most special thing. It's not listable. It's more like it takes a list in the first and the second argument, but it's not listable in the sense of expecting the same numbers of entities and properties, but rather. That's an outer.

Yeah, yeah, there's another subsection there.

On.

On batch outer.

This is a mess, guys. Sorry. I'm sorry. This is not. You're basically talking about single argument functions here, which is not clear.

Maybe it should be like threaded, which also works on listable.

Right.

So instead of batch byte, it would be batched and you apply it to a list and then you use it within some listable function.

I was about to think of something. Yes, I was about to. That's a. You know, so you say four batched and then range of a thousand. Yeah, that Seems much cleaner to me. I mean implementation, it will probably work without values and things, but that's a different, that's a different issue. This seems interesting to me. You could even have two different arguments that are both batched. Right. How would that deal with the batch outer? It would deal with it. Right.

So it depends on the function that accepts to. To list arguments. So would that be a listable function or not?

I would think so. I mean, I, I don't understand what you're. What, what does the batched outer do?

So it effectively does outer, but supplies two lists to the two argument functions. I think that would maybe go a little bit too far for this initial batching meeting.

We need a design that's general. So I mean, the issue is. If you had two. Look, let's take the entity value where you have both a list of values and a list of properties. Okay. Is it the case then that you could essentially do the equivalent of batched entities, batched properties, or does that not work? Is there something you have to do that's different from that? Let's say you did that.

What, what would the batches be or.

Whatever they need to be? I mean, are you saying that there are cross product batches? Okay, you've got a rectangle. Okay, you've got a list, you got a one dimensional list. The batches are getting chopped to a certain size. Right? That's what you're doing with this batch applied thing. Okay, now you are looking at a two dimensional array and you're saying I somehow want to batch the two dimensional array. It's much less clear how you do that. Much less clear whether if you're not independently batching rows and columns, how exactly do you.

Yes, so yeah, you can imagine a grid and then going along line by line on the grid. And then there are the interesting cases where the overall batch size is smaller than the width of the grid or larger.

Well, I understand, but if you. Okay, so the question is what's the space filling curve basically that. That walks through the grid?

Yes.

Right. And. If you are going line by line, something like this is basically going to work. Right. The only case where that wouldn't work is if you want to have blobs that are beyond a single line. Right. Or that span multiple lines.

So yeah, in the proposed example with batched of ends and batched of props, you could specify independent batch sizes, I assume, like. Yeah, yeah, obviously two drops, that means requesting 20 values at a time and.

Sorry, in what case you're saying, look.

Go ahead, what the Highlighted example. Yeah.

So probably you could obviously give different parameters in the batch.

Yes.

Well, okay, this seems like an interesting design idea. My question would be. I think it will be vastly more common. Jose, maybe you can comment on this to be just having a single batched.

Yes, I think so, but I mean it would be very interesting if this works too.

Sure, but there's no reason why this can't work. The only question is whether it's an out of product here or whether those two batch can interact with each other.

I think I would start by doing some sort of double partition and then each one we leave under control of the batching system.

Sorry, say that again, Jose. What?

Yes. So this is like a double partition. We start with little matrices, sub matrices of the full space and then if that's still too large, the batching system should handle it in some way. We don't care how, we just indicate like some sort of beginning of the batching and then the system will do the reconstruction. The flattening the array. Flattening.

What is the connection between the word auto and batch auto with UI and the inner auto?

What do you mean inner outer.

I mean, can you say batch auto with which Y of inner product or.

I think that's just doing like a partitioning of the matrices of the matrix in submatrices and then F is being handled. Is handling each submatrix.

Yes, but what kind of functions does batch auto with reply handle besides auto?

Yeah, it handles any function that takes two lists.

Okay.

It returns an array of the correct dimensions or matrix.

Sandra on a live stream is commenting that in image rendering people sometimes do successive blocks on a Hilbert curve. I don't think we care about that in this case. Probably. Well, I. I don't know. That's an interesting question. I mean if we were doing this for images, if we ever wanted to use this for Dynamics, for example, then we might care about the visual ordering of things. Did that make any sense?

Yes, if we were presenting the results dynamically, I think we would care about the order.

Yeah, so I mean the combined ordering here of those two batch might want to interact with each other is basically the point. But these are factored things I think we can deal with. The question is what happens if we want them to interact? Is there any syntax we can imagine to have them interact?

Did you have a single batched wrapper that's holding all the arguments where you can specify how it interacts?

Yeah, but how would you do that? Because batched is a multi argument function. I mean you'd have to have a batched sequence thing. Well, I mean, you know, you could do this rather bizarre. Wait a minute. Actually, hold on a second. We have the concept. No, we don't quite. I mean, we have the concept of. We have this concept. Right, but so almost what we'd be looking for is something like batched sequence ents, props and then some complicated, you know, Hilbert curve thing here and then say F of that. See what I'm saying?

That it would basically auto flatten after batching.

Yes.

Yeah, I think certainly the, the basic case is probably where you're only batching a single set, but there are cases, you know, for.

No. So the question, can we have the syntax be extensible? Because the, the second argument of batched is going to be its parameter. I mean, unless the second argument of batched, I think you're going to want the parameters all the time. You don't want the parameters to be options. Right. Is there any. Okay, here's a bizarre idea. Here's a bizarre idea. To use splice as a holder inside batched. See what I'm saying?

Such that it would batch A, B and C independently.

No, it would batch them together. It would do a combined batching of A, B and C, but then it would splice them into the outer F when it, when it, when it did the unpacking.

And so F would always see three arguments.

Right, Exactly. Yeah, I think this is reasonable. I mean, it's a somewhat corner case and I have wanted to extend splice anyway to deal with other functions other.

Than just list does deal with other functions.

Yeah, it takes a second argument. Oh, okay, fine. Well, then this is a perfectly reasonable thing to do, it seems to me. Okay.

I'll admit I got a bit lost here. So what are we suggesting that this splice here does? Because if it, if it's just digging things into a list and it's not changing its meaning and it should just work, but if, if it has some meaning that's different from just splice, then.

Here'S what it does. Here's what it does. Here's what it does. Okay, that does that, right?

Yep.

The whole point here is this is not a pure splice, it's a batched splice. And what does that mean? That means that the batched is going to do some fancy footwork feeding different sub matrices of abc. ABC is some tensor. Right. Rank three tensor, let's say. Okay. They could be even deeper lists and so on. Actually, that's another question is when you say batched in if you just say F of batched of X, if x is a rank 5 tensor, what does it do?

I think you'd have to give a level specification too, right?

Yeah. So I mean there's two different ways. Tony, have you thought this through? There's two different ways that batched can involve a traversal, a non one dimensional traversal. Case number one, the thing that you feed it is not. It might even not be a be a know rectangular array. The thing you feed it is a deeper array. That's point one and point two, you have essentially the outer product of multiple arrays fed as different arguments of the outer function. Do you agree?

Yes, I. I'll have to think about this.

So yeah, okay, look, we're right about this. Okay. I think this is a reasonable way of handling the somewhat corner case of multiple, you know, multiple outer, you know, wanting to co batch, so to speak, multiple function arguments. Do you see what I'm saying? If you want to outer product batch them, this will be fine. Right? If you wanted to co batch them, you have to do this.

So, so you're using splice here as sort of an inactive of map thread.

I guess so. I mean it is the same meaning as a splice. It's just that, you know, this will be a no op. If you had this splice here. We could put the F there if you really wanted to, but I don't think it's necessary. If you had the splice there without the batch, this will be a no op. Yep. Right.

Mm.

But the whole point is to. Because you want to diddle with how this is actually being done, you put in a splice so that you can feed it in one, in one gulp to batched. I like this actually.

Yeah, I can't.

You know, you're.

You're working with some service that's column major and so you don't want submatrices, you want the batches to go across the properties or whatever you're querying, but you want it to be sub slices that don't mix. Multiple columns.

Right? Okay. So now the only question or question is, I mean there's, there's presumably a bunch of arguments to batch like timeout options and things like this, Right?

Yes.

So.

So there need to be certain signals to indicate what went wrong if a batch failed. So that can be timing or memory used.

Okay, that's fine. But there's got to be multiple pieces here. So for example, if you've got even a rank 2 object in F of batched of X. Right. You have to say, how are you going to, you know what, Are you actually going to feed in? Are you going to feed in if F is listable? Okay, just to remind ourselves, if F is listable, what's a good example of a listable function? Sine or something. Okay, It drills all the way down. But we could be doing that by having sine of the state, B comma sine of list, cd could we not? Right. In other words. Or we could be doing the sign of the whole thing. We could be breaking it in a whole variety of ways. We could also be potentially breaking it in columns. Am I making sense?

Yes, but let me make a comment here. The idea of batch applied was leaving the responsibility to batch applied to control the whole thing. Here we are leaving the responsibility to the outer function to look for the batchette, the batched and do things. So it's slightly different.

No, no, no, no, no, no. I mean, look at the psumup value. I mean, it. The. The main point is this is going to create. Having the batch there is going to create a bunch of F of various arguments, various lists. It will then evaluate those and reconstruct them.

Yes, so. So we cannot have, for example, F of list of batched.

Yeah, that's correct. It's a top level argument thing. Yep. Which I think is just fine. I mean, just like listability. If I said F of G of a list, it's not going to know the inner list inside G.

It's just implementationally this is different. We need to. For example, if a function gets too batched and they have different options for error control, It's not so clear.

I understand. Okay, but so then you need a specification of what are the specifications of batch, in this case, batched. What are the things? What are the second arguments? What's the second argument? Tony?

The second argument to batch would be the initial batch size, I say, or the maximum batch size.

Well, which is it? Is it initial?

Let's say it's the maximum batch size. And optionally you could also specify an initial batch size.

Okay, but then, okay, and then, and then you've got things like timeout, arrow, whatever.

Yes, but they could. There are typically two. If, if you are a timing base, then there are two, two values. One is the high timing, which means you're getting close to some time constraint, and the low timing, which means you can start increasing the batch size. Yeah, let's say if the network resource.

Well, in that case we should probably have. If it's that ornate, we should probably have an association With a bunch of those parameters. I think this is a decent thing.

I would think double spec would be a more useful second argument, at least for the type of things that I could think of using this for.

I mean, I have to say I think the batch size. What is the real point here? What's the real point? The real point is, okay, so one approach is just wait for it to fail. You know, pick an initial batch size, wait for it to fail. And the initial batch size could be. There is no batching at all. But the problem with that is there could be a timeout. I mean you could do that with a time constraint. With a default time constraint.

Yes.

But the thing is the remote service might, might be upset if you send it too big a batch.

Yes.

Or he might block you.

Yeah. And another use case is if you are for instance, trying to implement those kind of regular batching, rectangular batching in terms of linear batching, then from the first line you might have learned what a good batch size is and you feed that to get the second line done more directly.

Okay, but this is all the reason why batch size is a complicated issue. And maybe the right thing to do is indeed, as Rick's saying, to put a level spec in the second argument, which is clearly useful. And then to have, and then to have all these parameters about how the batching is done be in an association. Am I making sense? Initial batch size, Arrow, blah. You know.

There could be things about like batching methods, like if it's rectangular, maybe you want to do like ad trees or something.

Yes. Or Hilbert curves or whatever. Yeah, batch ordering, David Hilbert.

Or handler functions.

Yeah. Okay. Roman, do you have any comments about this based on your experience with parallelize?

Well, I mean parallelize, it's all about the, the batch size or the number of batches, but there isn't any ordering stuff involved. We're still trying to figure out for which applications this elaborate mechanism is actually used.

Okay. The basic thing is you're calling a geoserver. The geoserver has a built in timeout. You can't call it. It is efficient to call it with as many places, for example, at the same time as you can. But you don't want to give it too many or it's just going to timeout. You don't want to give it too few because there's too much startup cost.

That's exactly like parallel.

Right?

I mean the start, I mean there is no timeout, but the startup cost and the communication overhead and there's an additional level in parallelism and that is load balancing.

Right.

If you have uneven jobs, you won't have a lot of jobs so that they can schedule themselves. But if you have small even jobs, you want to have as many as possible to minimize communication costs.

So this is, you know, courses grained, finest grained items per evaluation. This is somewhat similar kind of issue here. And it's interesting that there you're putting those things in a method option and maybe we should modernize this and have an association as a possible thing there. I'm not sure. But in any case, it seems like one should just build out whatever the batch thing naturally needs. But that's the case of interest. Right. Is. Is. Okay, so what do you know, for example, for the geoserver, what can you say, Jose, about the parameters? Because you know what its timeouts are probably right.

It's about the 20 seconds of Alpha in general.

Okay, fine. So I mean it. Since you know that it, I mean, and, and you know, my, my question is, let's say we have a service and we have a system services entry for that service. Okay. Or a service connection entry for that service. My assumption would be that that service could advertise its batching parameters. Am I making sense?

Yes.

So the best thing for batch to do would be to somehow know, you know, this is the service that's going to be used by. I don't know exactly how that communication happens, but that would seem to be the most sensible thing to do. Even if it has to be hard coded somehow in certain cases. I mean that would be the, that would be the nicest thing. Right. You know, if you change your geoserver that it would advertise different batching parameters and the batch would do something different.

Some sort of like service connect data or something. Something we know what the parameters are when Talking to Alpha versus this GeoServer versus open street maps or whatever. It could even just return the association. That would be the third argument.

So how would you return it limits.

And automatically pause as necessary too.

Yeah. I was just thinking if there's some, if there's some way to, you know, curate that data, then you could have a lookup function, you know, service connect data or service batching data, even if it's an internal thing and it could return basically what the third argument of batch would want.

Exactly. That's what I'm suggesting. Yep. Yeah.

There are also other strategies. For instance, a service can return a partial result when it knows of itself that, that it can't handle the whole request. And then there are also different Strategies, let's say a key to continue receiving results or the other is simply return the overall size and the recommended batch size and the function.

Also, listen, I think we have a winning general concept here. Many, many details to fill in. Okay? But I think we've got a direction for the design. Sound reasonable, Tony? Yes. Okay, let's move on. We don't have much time left, so let's optimizing table.

Yeah, this is a somewhat related, if kind of the opposite problem of another collection where we have a suite of functions that do some sort of network operation and they are optimized where if you feed geoelevation data a geoposition with a bunch of points in it or a bunch of geopositions, it knows how to send them to the server and return one result with all the bits of data. But it's not uncommon for users to put geoelevation data in a table and suddenly everything's way slower because it's doing them one at a time. And this has come up, you know, this happens with entity value users periodically. And so I think there had been a thread proposing that, you know, we should have some way to optimize inefficient code that users put in or at least provide some sort of warning about it. This is a pretty trivial example here where we're putting an up value on table that says if you call table of GEO elevation data then we will invert the argument. So it's geo elevation data of table.

Look, I think if we could do this automatically, it would just be a huge win. I think doing automatically with a warning message might be maybe even the best thing. We could do something like, you know, Right.

Except I mean, right, we can't do this automatically. I mean, in this trivial example, sure. But you know, if, right. If there's anything in between the table.

It'S computing something there. Okay, question. In a given input line, could we accumulate, could we know, oh, the person's done just a gazillion geo elevation data calls?

I mean, I think this is a problem for, for code assistants, not for the evaluator. Right. If we ran every possible analysis to try to optimize code. Right. We be a much larger system. Right.

We can't do that. But what I'm asking is this. Let's say that the server, the external server knows it's been, it's being hammered. Okay. And it knows that it is batchable.

Yeah.

I mean we tried to have a system like that for entity pre fetching, right. Where we would say there, you know, entity Value used to track and say, oh, you've made five separate calls to Country Flags. I am going to asynchronously queue off a job and download this and batch that. I guess, Tony, if you're still here, maybe you can get a better summary than.

Yeah, it's not functional, but I'm working on an overhaul of that system.

But which will asynchronously prefetch. Right.

So doing that asynchronously is not so easy. There are various design issues that have to be solved to do that asynchronously. So the first prototype will be synchronous. Actually.

How could that possibly work?

So if you first use a domain, then it might just prefetch it if it is small. And for the large domains, it might want to ask you first, while this is running, do you prefer prefetching the whole domain, which is of.

No, you can't do that. You can't ask. What are you going to do? You can't ask. What's that?

I was agreeing that you can't ask.

Right?

I mean.

Right.

The only thing running in a headless server, right?

Yeah, it's just you can't ask it. Not, not going there won't work. The one thing you could do is, which I think is the right thing to do here as well, is the possibility of a, a disappearing blue, you know, progress monitory type thing that says more or less. You're running a lot of separate geo elevation data things, by the way. You could do this, but it disappears at the end of the computation. By the time the computation is finished, you've, you've, you've swallowed the pill. It's done.

Yes. So my proposal goes in the direction, let's say if, if you're doing implicit entity class lookup locally and it's downloading a lot of data, there's already some blue box and that blue box could, for instance, receive a button that says, well, just download all of that domain. Yeah, so.

Yeah, I see. Oh, that's reasonable. I mean, so long as it's there. Look, look, look, look, look, look. Yeah, that's a good idea. Okay, so if it's being hammered and it can see in the case of this geo elevation thing, I mean. Well, this one isn't really a prefetch type situation. In this case, it could bring up a box that says more or less running lots of GEO elevation datas, by the way, you could, you know, put it the other way around. See what I'm saying? That would seem to me a useful thing. That's okay. Here's the basic principle. If you're sitting and waiting for something, then giving you advice about what you should be doing so that you don't have to wait the next time around is going to be a win. Do you see what I'm saying?

Agreed. Right.

And it's harmless because by the time you finished, it's like, look, I was only doing this once. I don't really care that I could optimize it. I've done it now. So I like that idea of watching, you know, basically. Basically, it's a general principle. When things are slow, if we could tell you how to make it faster, we should show that information. That's kind of a light entertainment while the thing is running.

Right. And that's, I mean. Right. That's the sort of thing like, you know, GEO elevation data could check. Well, have I. Have I been called five times with the same dollar line and then issue or something?

Yes, that's what I would imagine.

But what about having this particular value there which seems like relatively safe? Is this acceptable or is there any danger? So the particular value for GEO elevation data, only that one. Is that safe enough to be there in the system automatically?

I think so. This particular thing, I mean, I think it's a little bit nonlinear to have, you know, this case works, but as soon as you put an F around that geo relation data, it doesn't work. But I think if you combine those two things, I think it's reasonable.

Yeah, because it's an easy one.

It's.

It's in the next line.

Yeah. Okay. Just looking ahead and I know we're running out of time here. I just want to get a sense.

This one actually is very similar to the budget. It has some similarities.

It does. Well, it's. It's kind of a. Right. I like that description. What are these?

I'll just show you the actual values and their positions. And when you evaluate it, you basically do the out product of those values.

Okay, so this is. So there are two. I mean, you know, this is. Multi could have the meaning that you're explicitly giving this list. Or the multi is multiple possible. What happens if you have multiple ways to match, like with triple blanks and things?

Well, I have an example there.

Oh, there.

So, yeah.

But there's two different ideas here. One is kind of explicit spatial multiing and the other is in a sense semantic multiing.

Yeah, yeah. You can combine them together in some sense. I still kind of confused myself how it works. But there's an example of how you can use the values at certain position and the actual rules that act on a certain position that produce multiple values. Okay, Another thing, you can also have the actual evaluation also at multiple positions.

Right. I mean, okay, what does this do here? Where's the multi here?

Because you can apply your rule at all possible positions. Right?

Oh, so this is a multi. This is saying that multi can take a second argument which is a set of transformation rules.

Yeah.

Is that what you're saying?

Sure, yeah.

Even though most of the time multi is just a fed wrapper, like, you know, threaded or something like this.

Really?

Well, all of his examples have exempless of rules, I think.

No, they don't on the first one. Okay, so if I say F of multi here, what does it do?

Well, it will make a multi with an expression with f inside of it.

That's what I thought. Yeah.

But it also only works when the argument is not holded is not held.

Right.

If you do the hold of multi, then it's not going to do it.

Okay, but if I say this. Oh boy. Okay, now what happens if I do normal here? Does it.

Yeah, you have to make it careful. Unless there's special up value for that, it's gonna go inside of the multi.

I see. It probably does need a special value.

That's.

Yeah, there's an expression property. If you just ask for expression, you can extract it, but it's not going to be useful in this case because the actual placeholder is going to be returned. You, you might want to like write, evaluate, evaluate list and a bunch of properties.

Okay, okay, okay, I understand. But, but the, the main point about multiing is, I mean, just to remind ourselves, you know, the. In a sense, the story of multiing is partly told here, where we do have a bunch of examples now. Okay, so none of these. This is not a multiing story. This is simply a right. This isn't, this isn't where you're using multi. Multi is where you use.

Well, you can still use multi for that. Basically, if you define a function F, or you provide an explicit rule for multi, you make a multi object and then you can evaluate it iteratively, right. So that you could iterate it once it produces new expressions and all the new matches are, are pre computed where the new rules can be applied and then you can evaluate it again. Or you just ask some graph properties directly to construct this.

But usually in evaluation you are getting a foliation of this thing you've evaluated to a certain distance and you're slicing. I mean in the case of fixed point evaluation, you're going to the end of this. How does that relate to. You're saying that the multi could be just a few steps of evaluation. So you're foliating down through this like in these pictures. So you could have a multi that represents a slice here.

Yeah, you can truncate it if you want to. You can iterate it again, make a new foliation. But yeah, that's effectively the multi object itself represents one slice.

Right, but the multi object represents a slice. It's the state of space, so to speak. It's not the space time causal graph within the multi. Correct? Correct, yeah. How do you keep the trace, so to speak?

Well, I. I don't keep the trace. You can just like keep all the intermediate multis somehow or extract it when the graph is ovulated. But I don't. Multi is not.

But how do you make this multi way graph if you're not keeping the intermediates?

Well, just a function that just iterate this multi object many times, like five times.

I see. So in that case you are keeping the intermediates.

Yeah, in a way, yeah. Just some convenient functions that run the loop and store whatever you need.

By the way, just so I understand, given multi, how would you use that in the quantum framework?

Well, I wouldn't use it directly. Right. It's like it's not really analogous to any quantum computation directly like that. But maybe if you have some probabilistic representation of quantum computation, then you can essentially use multi as a storage for actual instances of species that you iterate over and rewrite over so that basically it acts as a instance of a probability distribution.

Yeah, right. So it's like in. In probabilistic programming or something. There's a similar. Except that multi here is an unweighted like graphs. It's an unweighted thing by default. But you could imagine decorating a multi to have weightings for every element. Do you agree?

Something? Yeah, something that by the way, those values in superposition they kind of can be copied freely because you can have the same placeholder multiplied multiple times even though each of them represents many values. So in a way you can use that maybe to indicate.

Yeah, discrete weightings. But I think it may also be useful to have just like in choice, in random choice, for example, you might want to be able to feed it a multi that is already awaited multi. I'm not sure we got more to figure out here. I think this is a much bigger comp.

Yeah, I haven't had time to dig into this.

No, we should not discuss this.

And for added bonus, Nick, string diagrams. Is that. Is that what that tensor representation, you know, is adjacent to or not?

Well, string diagrams and quantum circuits basically the same, right? Well, it's quantum circuits is one instance of those. One example. Or any kind of tension network.

Exactly. But so what I'd like to see is what we can do. You know, we've got various attempts at string diagrams. I'd like to see whether we can, you know, the relationship between multi and this and string diagrams and so on. It's kind of the direction. All right, we should wrap up here. You know, in a sense, by the time we've descended into combinators, we know it's time to wrap up.

All right.

Yes.

We'll circle back to some of these topics.

Yes, they're great topics. All right, thanks a lot, guys. See you all soon.