Live CEOing Ep 773: Language Design in the Wolfram Language [LogPDF, Multi, and More]

Hi everyone. Welcome to another Wolfram language design review. We are looking at various items, part of our incremental language development series. First up, logpdf. Yep.

So this was a question you had posed yourself, which is the machine learning group has their own log PDF function and should we move this to system? So I've kind of collected bullet points from the discussion both with the ML team and the STATS team, both that there's a. The generic version of logpdf is basically log of PDF, but there are distributions which sample in log space and there are also some internal utilities that already exist for log likelihood. And so it looks like, you know, beyond just having a trivial log of PDF of a distribution, there, there is some actual benefit to perhaps having a built in.

I think we should, I mean, you know, like we have a log gamma function because that's a thing where it's, you know, just. Yes, you can take the log of the gamma function, but it's just a better idea. I mean, you know. Now what about log likelihood? Is that a thing for which we should have. And we have a.

We do have log likelihood and we have likelihood likelihood is the products of PDFs and log likelihood is the log. So it's the sum of logs of PDFs.

Okay, so how does log PDF relate to log likelihood?

Log PDF would be a single element version of log likelihood. So what we have, we have derived formulas for the parametric distributions for log likelihood hood and we could use them to maybe implement the log PDF, you know, in the same fashion as the PDF is. So that means, you know, make it like a proper distribution function for every distribution.

Right? So in this case, log PDF of a normal distribution. Wait a minute, that's a multinomial, is that right?

No, because it's the function PDF, it's listable. So if you.

So I mean, can I get this. I could probably get this particular one by just saying log of PDF of that comma X probably. Is that right?

Yes.

Except actually it doesn't simplify. Why doesn't it simplify?

Because it's just the log of an expression. So either you do the function expand as above of that.

But even that hasn't simplified because it doesn't know that X is real and so on.

Right? So that's where the need to kind of implement this properly comes in. Because we know the assumptions on the data, we know the assumptions on the distributions.

Also, the one above is numerical, so.

It'S not for a generic X. And I have to say, one advantage. The reason why we have it in machine learning is that specifically for numerical.

Evaluation, the log PDF sometimes is more well behaved.

So you might have some numerical instability in the PDF.

All right, listen, let's have log. PDF. I see no argument against it and I see lots of arguments for it. Any objections? Nope. Okay, great.

Easy enough.

All right, next up, and let's see. The people who are here for log PDF can probably depart.

Yep.

So that'd be Gosha and Julio. You're welcome to step if you want.

Thank you.

Bye bye.

Okay, Tree traversal order. Why on earth is that smart quote the wrong way around, who knows? It's a bit odd. Looks like a bug to me. Okay.

So this is a missing variant of depth first for binary trees. And so I'm proposing here a generalization so that we can add it for any tree.

Okay. What does it mean?

In order means that you visit the parent before you visit its right child, but after you visit its left child.

I don't know why in order is the right name for it.

It's the standard name. There's pre order in order and post order. Pre order is parent. Yeah, Pre order is top down.

All right, so there's just a missing thing. So what, what's this minus two here?

So for binary trees, in order has a standard definition. For general trees, we have to decide where you want to insert the parent.

Well, we're not gonna. I mean, look, don't we have most of these tree traversal order things just have. It's just a string. Right. There's no. There's no parameterization of the string.

Right. This would be the first parameterized.

I don't think it's a good way to do it. I mean, I think that. Hold on one second.

If you've got in children, you can insert.

I understand.

In plus one positions.

I understand. I mean, you might want even crazier things like you might want. Well, I guess it's a traversal order. So you are going to do. You're only going to hit the node, the parent node once.

Right?

Yep.

So I'm suggesting in order means always after the first child.

Yeah, I understand. I'm just not wild about the. You know, the way of adding parameters to this. Don't we have. I mean, What's this? Does that mean vertical order?

So for base orders, we've got depth first, breadth first, and leaves first. Those can be combined with top down or bottom up, and with left right or right left. So in order would be a new vertical order. That only applies to depth first.

That's here. Okay, But to me this question of where the parent is inserted is very similar to this left, right, right, left stuff, isn't it?

It's most closely related with the vertical order.

No, but not the question of, of. I mean this is determining in which order parents and children are visited. But this is determining.

The order of children.

No, I. Well, I mean. Okay, somebody, a good design idea here. I, I really don't like. I really really don't like this. I mean in. And you can't use a list because you've already used a list for this case here. Right, right. I mean, look, to begin with, it could just only work for binary trees, which is probably the only case anybody's ever going to try to use it for.

Could you make a four element list where the parameters are the fourth element and then you support automatic for first and third?

No, because that would commit the parameter.

Yeah, it's not only being. Look, the thing that's clearly doable is just make in order work for binary trees and for non binary trees have it do the default thing of just putting the parent after the first element.

Three.

A pair with a name and an association with any parameters. Then there's no ambiguity with the three items there.

Say that again. I'm sorry, say that again.

So you can have, you can have the spec be a pair list with two things, the name and an association of parameters. Optionally.

I see. You mean. Well, the other thing to do, which is clearly the way to go is that tree traversal order can end up being an association. And you know, this could be called the, you know, base order arrow, whatever, vertical order arrow, whatever, horizontal order arrow, whatever. And then this other thing. Look, okay, okay, look, I think what we should do is just have the inorder case just say it is defined to put the parent after the first child node. Period. End of story. The ornate other case is going to wind up being, you know, I mean I can think of a gazillion. Okay, this thing here is in general the definition of a reference frame. Basically it's something, you know, I've written now quite extensively about this. It's, you know, there are these named cases, but there's an infinite number of other cases. You see what I'm saying?

Yes.

You might have something fully parameterized where.

Right.

Your insertion order and your, like you said, vertical order, horizontal order, all different.

So.

Right. I think we should just make an order be, you know, for binary. It will do exactly what you would expect for an N tree. It will put the parent node after the first. After the first child node Y. That seem reasonable?

Yes.

Why.

Why are you putting parent after first child?

Because that's cor. As you meant as opposed to after lost child or before first. No, you.

No, if it's before, that would be pre order. So. But yeah, if you want to be consistent with binary trees, you could either put it after the first child or before the last child.

Oh, so you're talking an infix type. Okay, sure.

Yeah.

Yeah.

Yes. So. So the only question is which of those do we want after the first child or before the last child?

Can I ask a dumb question? I'm going to. Anyway, whatever the answer. Is there some reason that we don't want to have it after each child other than the last?

Yeah, because it's a single. That I. That was what I was thinking as well. But it's a traversal order. You only visit every node once. Ok.

And I think it is interesting to consider traversal order potentially being able to revisit nodes. But it's not something we've tackled.

Right. How about the following thing? You know, maybe we need in order and then some word that means it's going to go after the first child and another word that means it's going to go before the last child. And then it's just a pure in order.

Maybe it's like in order left in order, right.

Thinking in order, penultimate.

I mean, you know, the long phrase I guess would be in order left child in order right child.

But that's. I'm not even sure that's right. I mean, I think it.

In order an ultimate.

You can use this along with left right and right left, can't you?

Yes.

So those would become misnomers if we put left and right.

Yeah.

Let's just have it. Okay. Put it after the first child, period. End of story. That's what we're going to support for in order for binary trees. It will do exactly what you would expect if you happen to have an NRA tree. It will not do something crazy.

Yep.

And you know, recognize that this is. These are simply named traversal orders. And there are an infinite number of other traversal orders which we didn't happen to name. And we're just trying to name useful ones.

Like an arbitrary function so that like an order actually ask for which child where the position to insert your parent. Something like that.

Yeah.

Then you can specify in order manually.

Yes, but the problem is what you want. The problem is most functions will fail. Okay. In the following sense that what you need an Order is an order. It's a sorting of nodes. I mean, I could imagine something where the tree traversal order is literally a list of what nodes to visit in what order. Am I making sense? I mean, these trees are always finite.

So I mean, you can always do that. Now with tree position, you can just ask for the list of positions in any of these traverse orders.

Well, I understand that, but I'm saying let's say you want to traverse the tree in some random arbitrary order. So I'm saying, how do you do that?

Yeah, we. We don't really have good parameterizations for different spaces right now. This in order with a parameter would kind of.

No, but we don't want to do that. We don't want to do that. What we want to do is you are given a collection of tree node specifications, tree positions. Okay. Your mission as an ordering function, as a traversal function is to put those positions in some order. See what I'm saying? Right. So is it the case, and I'm thinking it should be the case that there is a tree traversal order arrow and then it's simply a list of all the tree positions in the order you want to visit them. That's the general case.

Yeah.

Except in some sense, because it's a tree structure, unless we have hidden structure behind it, you're going to end up traversing the tree in some other order just to get the order that was requested. In other words, you know, you have to find the 47th node. So at the very top, do you go left or right to get to that? And you're going to be chasing it depends what operation.

Yeah. All right.

Nothing.

That's the wrong thing. Listen, will have to happen.

This question about the traversal order of trees, or in general dags, is the giant question of reference frames, causal orders. You know, it's a thing I spent a bunch of effort thinking about. It's, you know, we don't have in general relativity, for example, where we have exactly this kind of issue of finding, you know, families with space like hypersurfaces and so on. There are just particular named metrics that one typically uses. Just like here we have particular named traversal orders we use. And generalizing beyond that is really hard. So I think we should just not worry about this. I think we should put the in order thing in. And, you know, we made some serious effort to try and generalize this. And it has not been easy to do. It relates to the multi. Thing we're about to discuss. Go ahead.

I want to say in My own uses of tree traversal. In some of our kernel code, there are actually reasons to want to revisit intermediate nodes. You know, you drop down to the children, then go back to the parent, etc. And what I'm wondering is in the context of this tree traversal stuff, if we want a revisit option.

I don't know, I mean, do we have that in breadth first scan?

I suspect we do not. And I think when I was looking it was something I was missing. I don't know that for a fact though. And it's very possible. I just didn't catch the documentation.

This is kind of map at. At different levels, is it not? I mean, you could just have a big list of places where you want to apply a function.

Well, no, this is a bit different. This is when you want to traverse the whole tree. But you need to do some things at parent nodes both before and after visiting the kids, that sort of thing.

Yeah, I understand. I mean, this is. Look, the functions depth first scan and breadth first scan have not proved very useful. They're too complicated to use for the most part.

Yeah, well, maybe I'm adding too much complexity with that suggestion.

And then, and then we have this. Brad's directed acyclic evaluate, which needs a better name, but it's something I think we should look at.

Vertex weight fold was one that we discussed last time.

Wow. Which is a, which is another name for directed acyclic evaluate.

Yeah, I don't, I don't see an improvement on that.

Oh. Yeah, I mean, you know what it is, right? There's a function at every node and you are basically running the function at every node. So in other words, you think of the thing as an expression tree and you're evaluating it by, you know, actually applying that function to the child nodes. Does that make sense?

Yeah, I think of it as a generalization of tree fold. In trees you have this depth first order that tells you the dependencies. But in a dag, this reference frame stuff comes up.

Yeah, Yeah, your point is treefold is. I mean, perhaps what it really is is graph fold.

There's a fold graph function.

What the heck is that?

But directly, acyclic evaluator would be. It is a fault, but I would call it something like a path fault or graph path fault because it's really coding along paths with some custom functions that you can provide.

But I mean, wait a minute. What is false graph? I mean, where is it? Is it in the function repository?

Yeah, Fold graph. Is like a fold but it produces a graph.

Okay. As a. So it's. It's like, I mean, nest graph. It's. It's like nest graph.

Yeah, yeah.

Is this like there's nest list and nest graph and this is a straightforward thing for fold.

Yep.

Oh, you have lists. This is multi somehow.

Oh, this is yours, Nick, right? Yeah. Pulled with multiple inputs. I see. Okay, So this is different from graph fold, Which is take the graph and use it as the material to fold. I see. But the problem is it really is graph fold graph, is it not? Or is it. No, wait a minute. What does directed acyclic evaluate return? Does it return a scalar final result?

It returns a weighted graph.

But usually the same graph. Right. You just wait. Basically it just waits.

Yeah. Okay, but so there's an alternative.

It like flows the weights across the graph and then returns the graph.

So. Okay, but there's a direct different thing you could do, which is just evaluate. Just do what fold does and evaluate. Like on a tree, for example, tree fold. That's what it's doing, right? Tree fold is giving you a final result, is it not?

The thing is with fold and with tree fold that they have this relationship with the algebraic data structures for lists and trees. And that's not clear to me for dags.

What do you mean algebraic data structures? You mean.

I mean, like what, how do you like a graph? A DAG isn't usually expressed as a combination of like operators. It's just a big bag of lists. And so it makes me nervous.

Is it. Is it this kind of graph fold?

Yeah.

I like vertex weight fold. Gives you some idea of what's actually happening.

Well, it may not be the right function in terms of functionality. So what the heck is this doing? So this is taking. This is. This is treating. Let me do this on a simpler graph. Whoops, that's giving me a graph, isn't it? How do I get the corresponding tree?

You can do graph tree of that.

Okay, Okay, let me try to understand this.

So.

If I were to look at the input form a percent 8. And I get rid of these tree common nones, basically what's happening is I replace tree with F and I get this. Is that correct?

Yeah, actually do tree fold of tree comma percent 8. Fold not form.

I see. That's what I would.

The tree is like a fixed point.

I hope you put that in the properties and relations of that function.

I believe so.

Okay, I think we should have a separate discussion about Director. They said click evaluate and.

Yeah, we'll plan to have that one of the weeks that Brad's available.

Okay. All right. Okay. Should we move on? Okay. Let us dive into the grand world of multi. So where we left off with multi.

So we were talking last time a little bit about the multi as a container value in superpositions. And as we. It was kind of related to.

What's this? What's this? What's this?

Just ignore it.

Really? Why am I getting this error? Should I quicken restart?

No, it's not an error. I think just about missing hypergraph bucket.

Because why is it missing?

I thought, well, ideally should have been installed. I don't know why it can't be installed though.

What the heck is this?

Well, maybe you can install it, but it doesn't depend on it. Anyway, we can just move on.

Okay. Yeah.

Goofy, isn't there?

Okay, but. But in any case, we should debug why this is happening. But that's a separate thing from this.

Well, it's just how I made the packet. Maybe. Well, it's supposed to be installing the hypergraph when you load it, but I don't know why it's not doing it.

Okay, but we need to find out why it isn't. But not.

That's a bug in his hyper. In. In his code. In his packlet code.

Okay, fine.

And then fix it.

Okay. All right. So now what are we seeing here?

So we. Last time we looked just for all the sections briefly.

Right.

We can just start from the superposition and all the multiple ways of using multi. Because it's really multipurpose thing.

Well, so for example, if I wanted to implement the function multi way system using multi, how would I do that?

You create your multi and then there's like one property that is called edges. Maybe that's the most simple one. It creates a single level of edges from your initial condition to all the possible values it can produce. Whether there are explicit values or rules or whatever. You can then just take those edges and create new multi object and iterate.

I'm fairly confused. Okay, let's take it from the top. I don't understand. The arguments to multi I thought were just expressions that are sort of in superposition. Is that wrong?

Well, not really. You can either use a list of values or you can use an expression and then you provide a list of rules. But this expression also can contain arbitrary definitions. So it also be gonna be applied as a rules.

Wait a minute. What's the point there? There's an explicit version of multi where you have all the material and there's another One where you're generating on the fly. Is that the idea?

Yeah. Basically you specify what. How to generate things using rules instead of explicit values.

Okay. I don't understand what that F of multi of 2, 3, 4 means. I mean, that looks to me like just a single expression that doesn't do anything.

Yes.

So now if you combine it with the second multi, then it would be more useful.

All right, so if I take this.

It means f of 2 and f of 3 and f of 4 in superposition. So is that.

Yeah, that's what it means. So if I take this and I say like two times this or something. Okay, now how do I get an expression out of that?

A single expression or. Well, we can.

I want to.

No normal. No, normal doesn't work like that.

Okay, so how do I get the whole thing? Go ahead.

It's. It's list values, Right. You can list values.

The heck not that. Where's the F? What are the apps?

Try evaluate values. Evaluate Least don't remember. Let me check.

And it also doesn't look like nine values.

Evaluate list.

So, okay, so that's really a defining difference from listability. When multis convince combine they out of productize. Correct.

Yeah. So we combine multi with three values and with another with three values. Now we have a three times three in total, nine values. So that's why I said.

Okay, so if I say ab. If I say list multi, ab, multi xy. Oh, for God's freaking sake. It should be displaying better than this. But anyway, let's don't worry about that.

Well, I mean, when you have an infinite number of values, it's probably tough to display it.

Yeah.

So this is when you have two multis at the same level of an expression. How does this work if there's a multi at a lower level?

It doesn't matter. It can work on any level.

So if I say F of that multi comma g inside that multi.

And there's also an argument to evaluate list. If you can ask for a single or two values. So how many. So it kind of has some laziness built in because the outer product usually get very big very quickly. But if you want just the first one, you can ask just one.

Right. So now in general, I mean, there's a graph right from the. Well, let's see. Now each of these things has produced a space like slice, basically. Actually a branch like slice that. Right, Nick?

Yeah.

You can think of it like this way. Yep.

Okay. But now there are operations I can do. Let's see if I Can do a nest list. How do I get something that is like. And if I say nest list F multi.

You really have to be careful because this is going to be just absorbed to the nest list.

I don't understand this.

It's like you have to hold it or use an emulate it, but it's not going to always work.

No, this means you could have. This means nest list on X and nest list on.

Yeah.

Yes. But I mean, my question is, well, does it mean that or does it mean the thing that's going to outer product bigger and bigger at every step?

So I'm saying you see the placeholder, right? You. You just substitute in that particular for that particular posthoid all the values that you specified. It's just an expression.

Yes.

So what was your question?

The outer product here is just two elements. You're not getting like the, the penis is not nesting in this example. Multi is just treating nest list like any other expression.

Right, but it would get more complicated. I mean, the thing that I'm wondering about is if I say F of multi x, Y.

What's that hash supposed to do?

Supposed to absorb the next thing here.

That's the second argument.

Oh, I didn't mean to put it up. Okay, What does this do?

Right. The multi is super aggressive. And so you basically get. You're experiencing the joy of having zero evaluation. And so you must force evaluation at each step.

Why didn't this evaluate to a list?

Because you have multi inside. So it kind of. It was produced. Maybe it produced a list, but it was list of multis, so it got absorbed again.

How does evaluate list decide whether to return a list or a multi?

Oh, it doesn't decide. It always returns a list. But if you return the list of multis, then multi, the app value for multi, fires. Right? And now you get a new multi again.

So percent 8 isn't a multi just because there were no multis in that list.

Right.

Wow. Okay. But so, okay, imagine we have, I mean, basically function composition is. I mean from a category theory point of view, it's kind of like there are morphisms and then there are. Oh my God, what are those monoidal categories, right? Which are the. The. And so this multi, I mean, we've got.

Yeah, it's like a tensor product. So you have a horizontal composition too, Right?

So this is vertical composition. And then multi of fgh is horizontal composition. Is that right?

Well, in some sense, yeah.

But in general, a graph, by foliating a graph, we could turn it into a collection of Multis of compositions of multis. Is that true? If you have a dag, you can foliate it into compositions and multis?

I think so, yeah. I don't have like a very well developed intuition yet, but I think it's.

Like, like any resistance tree.

And then each branch is a mult.

Possibly, yes. Every time it branches, it's a multi. That's possible. I think that will be a good function to have is something that converts a graph or DAG into function compositions and multis. That would help us to make sure we understand what's going on. Now, you know, if you have something very mundane like a resistor network, presumably if I'm not mistaken, you can represent that by, you know, a ser. Combination of series and parallel circuits. Am I making sense? So I think.

We get a string diagram. That's, that's the next topic.

Okay, now let's look. What are these called? The things where you have. Who knows how this works? State space models and so on?

I do.

Okay, so is there a thing for doing parallel series Systems model.

Systems model. Parallel connect is a function.

Oh yeah, there's systems model, series connect. Systems model, parallel connect. So is this not analogous?

I suppose.

I mean, and then, and then this thing is giving some evaluation of those, you know, these transfer functions. I, I think these are all analogous.

Right.

It.

It grows the size of the state in the system when you connect things in parallel. They're all monoidal categories, basically.

Yeah.

And they can be made into a diagram. The string diagram.

Okay, we should look at that. What is feedback connect?

That connects the output of the first system.

I mean, I understand what it is, but what is that? What is that in the world of. I mean that's a closed time like curve in our physics picks a trace. Does it take a trace or does it make a close time like curve?

Well, I mean the close time like curve is a trace because when you have this feedback.

I see.

Is essentially the same as tracing your matrix. If it's. If you have some matrix representation of a diagram.

Yes. Okay, let's go on and look at string diagrams for a minute. And then. This is very weird. This thing with rules inside a multi is totally weird to me. Can you explain that?

Well, try evaluating it. Let's explore some of the properties. I actually have a newer version of the agenda. There are more examples there.

Okay, can you send it to me?

Yeah, Maybe not for the multi way system, but in general, more examples.

Nick, is the case of just a list of elements. Is that just equivalent to a list of rules with an empty list on the left hand side.

Well, I was thinking about. It's not quite equivalent. You can't really come up with a rule replacing some particular placeholder. Or maybe you can, but it's going to be much more awkward. So I kind of decided to leave it two of these different uses. So in a way, if you have a placeholder, the main difference I guess is that when you have a rule, it doesn't have to necessarily be applied at some particular position. So you can apply it at some part, but some parts of the expression are not going to change. But the placeholder always can have a value, always should be a have a value. So if you ask for evaluate list all the placeholders should be replaced with some values but not all the parts of the expression would be replaced with some rule because rule can be applied or whatever. It's complicated. It made sense to have the separate use cases.

Let's understand this. I don't get this at all. I mean, I don't understand. I thought multi had a list of expressions in it.

That's one expression.

But you have rules, so.

Right.

You generate all the list of expressions from the. That's my understanding. Right?

Yeah, right. I think that should be a different head. I think that's a sort of a multi rule or something head, you know.

Evaluate to multi or what?

No, I think it's lazy.

Can you have a number of elements in the list using this form of multi?

Yeah, probably. I mean to me there's a multi list that is just the raw list. Okay, that's just raw list of elements.

Yeah.

And then there is a generator. Multi generator, let's say which is a function and now that I'm a little bit confused, that function is applied to what an expression or list of expressions. And that function, you don't have to.

Have a function for generate, it can be an expression and then you can add a rule that just produces different out right hand sides depending what your expression is basically a rule, it's the same as the function.

Well, okay, fine. I mean we could say multi. Look, I mean this is, it's not very, you know that this is a function multi. Okay, this is. These are bad names, but this is a list multi, association, multi.

Well, I would.

Multi.

Well, I would say that it should be expert F for consistency.

Why for this one here.

Right. Well, at least what, what, what he has there is expert rules. Right.

So I know, but if we're going to do. Okay, so let's say rule multi. Is this so then That's a. I'm. I'm not a little bit troubled by argument.

Is it because map is F comma expert? Multi wants to be F comma expert.

Yeah, but be that as it may, these are generators of multis. So for example, in the case of our multi way systems, we had a ridiculous zoo of multi way functions that, you know, multi way function function, multi way system, multi way turing machine, blah blah blah. Am I making sense?

Yeah, yeah.

And this would avoid that. This is three cases. The explicit one, the function one. And again, these are all potentially lazy. Am I making sense? Should.

Yes. And should there also be an association Multi.

That's an association.

You're like naming the branches.

Yeah, possibly.

You might worry.

Although does that really need to be separate from the list multi?

Probably not. I mean it's that better name. But yeah, right. I mean it's almost. It's almost like too multi or something. I don't know. I don't know. But okay, but given these things, you then just, you know, how would you make multi way system with these. That's a rule multi. Correct.

Okay, so we just iterate it over. Ideally.

Well, I think, I think it's just nest list of rule multi or nest of rule multi.

Yeah, but you still have to somehow produce a list of edges, not just list of multis. Orig.

I understand, I understand, but I mean that. That's why I see what you're saying. So it's more like nest graph.

It's nest multi.

No, but the multi is a slice. Remember, the multi is a slice. The history is more than just the slice. I think we need to look at string diagrams.

This is just a reminder that we have it in quantum framework, for example.

So how is it represented there?

You just basically specify a list of your processes and you specify what are the inputs and what are the outputs. But by essentially naming them so that you can knit together the whole diagram. Your inputs has an order, your outputs has an order. They essentially named indices. You can connect it to a diagram.

So every port has to be unique across the whole diagram. Is that true?

Not, not necessarily. Because you also have this specific layer because we're only dealing with circuits there. So you can name it by an index of the wire that runs horizontally. So we have one, two, three wires, vertical wires.

I see.

Those are the names. You can just specify what. Whether this particular wire is an input to your processor or the output. And you just specify it by list of these operators that has some inputs and outputs in some order. Those would essentially be a Topological order of your operators, then you can make a diagram, turn it into a network, and so on.

And how does this relate to what we do for neural nets? It's basically the same. I mean it's a net chain thing where you've got ports going into other ports. We've got both net chain and net graph.

Yeah.

I mean what's the relationship between. Yeah, I'm confused here. What's the relationship between the net chain net graph world and this world here?

Well, they're very close. Right. Because the diagrams, what connects them together? They have. And diagrams contain some elements again composed either vertically or horizontally. That's your monoidal category again. Yes, Just in the quantum world you have some quantum gates which are some kind of tensors that you have to then multiply together to get your result. In the neural network case you have a bunch of layers that you again.

Should you multiply together to get your result?

Yeah.

Okay, but in our multi way system, so, so that multiply together, get the result is like directed acyclic evaluate, is it not? And it's also like the transfer functions.

Yeah. I think the neural network case is different because the tensor product, this horizontal composition is kind of different. It never has some layers taking like multiple inputs in a way.

Right.

Because unless there's some aggregation.

But that can be aggregation.

But in some sense.

Basic point is this. I think our traditional model for functions, you know, being applied to functions and so on, is this tree structured expression model. What, what? Everything we're seeing here is, is a generalization of that. It's graph structured expressions or at least DAG structured expressions. Am I wrong?

Well, they're not structured by graph, they're again structured by these two types of composition. Vertical, horizontal, you can turn it into a graph. And string diagrams also has some nice properties that you can reduce a bunch of wires and actually turn your compositions into just a graph.

Okay, but I mean I'm trying to.

Understand, wouldn't a more general form not required to be acyclic?

I'm not sure. I mean that's what. The feedback thing.

Yeah, because I was also working on something that I'm tentatively calling LLM graph, which is also related to this stuff. And one of the things I was exploring is the possibility for it to loop under certain conditions around certain points in the graph.

Let me understand that. So the LLM, I mean, for example a standard LLM. Are you thinking about how the next token is handled?

No. So it's. You can think of it as like picture each node as like an LLM function. Right. And you basically define a graph on these nodes where the output of some are fed into the inputs of others. And you can have named nodes. So you know, your string templates just pick up the outputs in the right spots. But one of the things I was looking at was like, what if you wanted to retry one of them multiple times until a certain condition is met? Something like that. You might end up with something that is an infinite loop.

But yes, I understand, if it's defined.

Well enough, it might. It probably wouldn't.

Right, but so what you're saying is, I mean, this is a flow of data story.

Yeah.

You have a flow of data through string templates.

Yeah. And I've been using Netgraph as sort of like the mental model for how to put it together. But that's the major difference. I'd like to have the ability to make it cyclic in some ways. I haven't figured out how that works yet though.

This is like all about representation. As soon as you have. For your diagrammatic process in general, as soon as you have representations for those special arc processes, I have an example. On the left you have this cup process, on the right you have then another cup. Right. As soon as you have these processes and representation for those. In case of linear algebra, there are some special states and effects. In case of maybe some computing, they even can represent some promise. Right.

Which thing is a cup? Is this a cup? What's a cup?

In the quantum case, it's just a bell state. It represents a state into superposition.

Okay. Yeah.

Okay, so you have a representation for that in some other diagrammatic process. If you have a representation for this kind of process, but for this cap thing, so that you actually can manipulate the whole diagram as strings, then you can do some nice simplifications of the diagram and reduce into much simple forms.

Where is there a string here?

All the wires are strings.

Okay, fine. So, okay, first question. There's got to be somewhere, some dictionary that somebody's made of all of the different meanings of string diagrams. If not, we should try to make that.

You want to collect all possible type of. Like that's I think the process theory, that's all about that because.

Yes, I understand, but I'm saying the actual uses that we've got now, you know, we've got Rick's LLM string diagrams, we've got control theory, we've got net graph, we've got quantum, we've got multi way systems. Okay. We've got game graphs, we've got A whole bunch of different things that are all the same idea over and over and over again. Am I making sense?

Is there a purely graph theoretical version of this?

Well, I mean, these are very labeled graphs. They're both labeled and evaluateable graphs, both of which are not, you know, communal garden graphs. Okay, let me make a meta comment here. I am sure that there is a nice, you know, sort of clean overall language design. This is a. I have to say, this reminds me, you know, this is a. Let's get it done now, because I tried to think about this in the mid-1980s of kind of graph based languages. And you know, one of my things of the 1990s, probably late 1980s, early 1990s, was, you know, we have infix operators, we have prefix operators. I had this idea of graphics operators, which is not quite this, but it's something close to this, I think. You see what I'm saying? So that would be a case where you are looking at the flow of data into and out of functions and the functions are on the edges. But that's not what we have here. Is that correct?

Functions are the vertices.

Yeah, that's what I think. Right. Which is also what we're doing in the case of expression trees, the functions of vertices. Correct.

But you could represent it either way if you chose to.

As a.

Which if you chose to represent it the other way. It's possible though, right?

Yeah, sure. But I mean, it's just like. But we have a whole model, I mean, you know, our whole language is based on tree based. Okay. We have a little bit of a different situation because in our trees it is not the nodes of the graph that are the heads. It is kind of a side node, if you understand what I'm saying. So when you convert a, you know, for example, if we convert an expression tree. Right. So this is expression tree of F, of X, of G, of A, B. Oops.

There's a whole zoo of ways to convert this expression into a tree. There's not just one tree representation.

Right. And this is the one. Right. This is what I mean. Okay. Yeah.

The older legacy, what was it? Tree Farm. I think that shows it in the form you were just talking about where the heads are like side notes.

Can we get that here? What is the option here?

Do sub expressions and you probably want heads. Goes to true.

Okay, okay. All right, well, look, the basic point is we have this very nice compact notation for expression trees, you know, F of X, blah, blah, blah. What's the thing we want that represents essentially, I mean, you know, there is a. This is. We need to understand this correspondence. This is a trivial string diagram. Is that a true statement? We could write that as a string diagram.

Well, as I said, you need just two types of composition to make the diagram. In general, that's like if you click on that diagram, process resource function on the bottom, like for these things. That's how I've been representing it. It's an old thing. Just look at, for some examples. You basically give it an expression kind of just specifying your inputs and outputs. But you can compose them either using composition or the tensor product composition. That's how we specify how they are composed. Sequentially or in parallel. Yes, that's how you can build up the whole diagram. And then the diagram can have different meanings, different representations. It can either be interpreted as a linear algebra thing, and those are things that tensors and you can contract them, or they can represent actual functions that run either sequentially or in parallel. Right. Or your.

But wait a minute, There's. There's a single way in this case, I mean, composition is not. We assume in our composition that, you know, F of G of X is just a one argument composition. In more generality, we could have something like, you know, F Of #A@STAR, you know, G of B hash and so on. Right?

Yeah. Well, if you specified actual which output of which function goes into which input, or whether there are multiple outputs, if you can all specify that, you can turn it into a diagram.

Sorry, can I say something here? So I think here we have a natural nesting of functions and their arguments. So there is some sort of natural direction in which things are nested. With tensors and linear algebra, one thinks that one have a function of a vector, a linear function of a vector. But because of how linear algebra works, one can also think that the vector is the outer function of the function. So there's this duality, and this duality makes things symmetrical. That's why we can use symmetric notations like u.v or uava.

Right.

So in linear algebra we have functions, but there is a symmetry added and that makes the notation symmetric. And that's why these diagrams do not need a direction. So with functions you have inputs and outputs, so arguments and outputs. So there is always a directed graph. With tensors, the connections are not directed because you don't need the direction. It's all symmetric. The contraction is symmetric. The functional application is symmetric because you can think of linear function of a vector or a vector of the linear function.

There's a way to make sense of direction there, because I have arrows there. They essentially represent the flow. Basically, if you conjugate this diagram, you can conjugate them. And if your actual tensors are complex, those would actually mean the complex conjugation. So there's some interpretation you can add to the direction of arrows too. Even for linear algebra, that would mean.

Well, and the quantum case. Right, which is a little bit. What is the complex. Is the quantum case like complex linear algebra or what, what is it like? Yeah, okay.

Yeah.

For which the arrow direction, but which is not something that comes up in general activity because the tensors are not complex or does that come up in spinner indices and things?

Yes, spinners are complex and, and, and there is a, an, an anti symmetric metric. So up and down are not symmetric. The metric is anti symmetric.

Right. So then there are arrows needed in that case, are there not? For spinner algebra, as opposed to linear algebra, I mean, ordinary linear algebra.

I'm not sure, because your spinners, you know, in the object, what, what is up, what what is down.

I see.

Just going to change the result of tensor contraction. It just changes the representation of the diagram so that there are some certain operations you can do with the diagram and the tensors. It just makes sense to add an arrow when you want to, for example, transpose the whole diagram or reverse it, or inverse it, or take the Hermitian conjugate so that the arrows kind of also flip around.

All right, we should wrap this meeting because I've got to go to something else. But, but maybe you guys, I mean it, it's clear there's a real thing to do here. We need a, whatever string diagrams, diagrammatic processes, you know, whatever we want to call it. But we need a generalization of essentially our tree structured expressions mechanism that deals with, you know, the, the deals with this case of parallel function, you know, these, this parallelism, the, the monoidal category case, so to speak. Maybe you guys could. Ian, Nick, Jose, perhaps, and anybody else who thinks they're relevant could independently meet and try and talk about the stuff. Does that seem reasonable?

Yeah.

Yep, yep, yep.

And try and I mean, you know what we're looking for. I mean, you know, look here, I like this idea of, you know, a, A, A kind of a. I mean that's like a multi that you know, out of product thing there, isn't it?

Maybe the outer products of inputs, because they basically can either take the tensor product of processes and the actual wires too. So maybe multi represents the wires, not the process.

Yes, I understand. Look, I mean, you know, this is going to wind up being some new notation, a bunch of new functions. I mean, just like we have list manipulation functions, we have graph functions, we're going to have some other collection of functions that represent, that manipulate these kinds of objects. And hopefully all these various things, whether it's control systems, you know, neural nets, etc. Can all basically flow. I mean, the concept is, what we want is a meta structure in the same way that our tree structured expressions are a metastructure where with specific heads they mean particular things. So in a sense what we want here is, you know, just like we have expressions, we have associations, whatever we have, you know, we want something that is a generalization of tree structured expressions, which we can make use of for these different applications by essentially putting in different heads. Does that make sense to people? Yeah, yeah, it's an interesting project. It's an important project because the fact that we're seeing all these different areas make use of the same basic ideas. And by the way, once we have this, certain things become sort of obviously parallelizable. And just like we have what I predict, just like we have parallel map, for example, which is a thing that parallelizes over lists, so we'll be able to have some kind of parallel directed acyclic evaluate or something that operates over these kinds of things and that parallelizes when it can and serializes when it has to type thing based on whether it's monoidal or an ordinary morphism type thing.

That's one of the things I was doing with the LLM graph too. Like I would do as much of it as possible asynchronously, right? And then figure out the dependencies.

Okay, we've got a lovely thing here to build. This is important. Ok, I'll talk to you all soon. Thanks a lot guys. See you later.