Live CEOing Ep 784: Design Review of Calculus & Algebra Features

Hello, everyone. Welcome to another Wolfram language design review, part of our collection for version 14.2. And today we're talking about calculus and algebra. First topic, array calculus. And I see and Roger has a background notebook here. Yeah, that characteristic of our long term project seems to have been started in 2005.

Right? Yeah. So one of the things here is to, and users have asked for this for a long time is to have symbolic vectors, symbolic matrices, and more generally, symbolic arrays. And when I say symbolic, I mean it's like one symbol, not a vector full of symbols or a matrix.

I understand. So how does this relate to tensors in general relativity? You know, Team Unu and things like that. Is that the same idea or is it a different idea?

Well, it's a more limited idea.

Yeah, it's a more.

Which is more limited.

Arrays are more limited than tensors. Right. Because tensors carry information about. Right. Is it covariant or contravariant, you know, what basis are you using, etc. Whereas arrays are just right arrays of numbers.

So in some sense, when machine learning.

People talk about tensors, they mean arrays.

Yes, yes, I know. Right. But in a sense, the wrapping that we're doing, like in the tabular stuff and so on, is a way of taking a raw array of something like numbers inside and then, you know, having connectors on the outside that in that case just describe no quantity. What's that?

No, no connectors.

No, no. But no, I'm saying for tabular, we're dealing with the fact that it is raw numbers inside, but they might be in units and things like this.

Yeah, but tableau has more types, it has dates, it has.

Oh, I know, I know, I know, I know, I know. So you're imagining. I'm just trying to contextualize here. Okay. You're imagining symbolic array is something which were it to be fully inflated, would just be a list of lists of numbers. Is that correct?

Yes, in our standard way of doing it. Yes.

But question. I mean, could it in fact be a list of. List of polynomials, let's say. Or is it really. No, it could be. It could be. Okay, but it is, it is something where the mu index has to be in the coordinate system is merely sort of structural part indexing. It can be nothing more.

Exactly, yes.

Okay, now.

You know, one, one thing that's really okay. So in terms of leveling up and get more effective language to specify problems is to use vector notation or matrix notation, et cetera. So imagine that you have a thousand by one thousand matrix that's symbolic, that's a million numbers. And if they were variables, symbolic variables, There are very few algorithms that we have today that works on a million variables. But when you work it, it's a one symbol, many things can actually be made to work.

Sorry, sorry, I got distracted for one second. Could you just replay those two sentences? I apologize.

Okay. I think it's, it's very effective to symbolically manipulate these things as, as, as one object.

Yeah.

Think of, think of just a thousand by thousand matrix that's, that's a million numbers.

Or if it's M and you have an M inverse and you put the two together, you know, it's just the identity matrix and you don't need to look at those million numbers.

Right. And so there are many reasons to want to do that, but symbolically and structurally analyze them. But one case where this comes up is that one important thing is to take derivatives. Okay. There are many reasons to want to use this vector or symbolic matrices and vectors, but one important use case that we're talking about today is to computing derivatives.

Sure.

And so you want to compute derivatives with respect to a matrix of variables. Yes, or a vector of variables, et cetera. And if you do that, so let's say that you have, let's say it's machine learning or statistics, then you often have vector and matrix variables in your code. You don't want to split those into scalars. You want them to stay vector matrix.

I got it, I got it, I got it. I got the motivation. I wanted something like this for 40 years. Okay, so I, I'm not, I'm easy to sell.

Okay, so then let's go and look. Scroll down in the notebook to the right a little bit and let's skip matrix D for right now. We'll come back to that in a second. And we're going to do the general array D case. So, can't remember now, but open the general array derivatives. Yeah, the general array derivatives. Look at the definition. And so if you look at that definition there, you might want to magnify it a little bit if people have lower resolution screens. This definition we actually introduced for D back in version 5. So D, if you have a component array, you can take derivatives in D with respect to a whole array of variables.

Yeah. Now we have some crazy stuff for that, which I don't like at all. What is that stuff?

What stuff?

The vec. The things, you know, indexed and all that kind of thing.

That's not it.

Okay, what is it?

No, he's talking about if you write, you can do D of F of X Y Z with respect to the list XY Y Z and that'll be the gradient. Yeah, right. Which is something which has worked since. I thought it was version six, but Roger Akis is claiming that since version five.

Yeah.

Okay, so wait a minute. This is D of F of X and Y with respect to. Is it like this?

Yep. That's obviously the same as grad of F of X Y comma list X Y.

Okay.

And you can have. Where you had the list X, Y, you can have any array of variables.

Yep.

And you can have any sort of integer after that if you want sort of second derivative.

If you take it.

If you do that with secondary. So you get the hessian right now.

So I can say this comma two.

Yeah. And you get the hessian.

Okay, okay.

Okay. The definition that I've listed here for what array derivative is, is completely consistent with that derivative. So let's see if I can talk without pointing. It's kind of a little bit hard to use words. Words are not as, you know.

Well, okay, but Roger, Roger, I think I understand the basic idea here. Can we talk about. I mean, can we look at the design?

Yeah. So then I have to bring up another problem which is the curse of listability.

Yes.

Okay. So the problem is that when we have listable functions, they interpret a symbol. Let's say that you have a as a symbol.

I understand. If you say A plus the thing. It's like what we use threaded to deal with. Threaded is dealing with relist abilating things.

Right. That, that's. Yes, that's changing the. The Right. That's making it list later rather than earlier. Whereas here we need to sort of turn it off entirely.

Right. So the issue is if you say A plus this, it gets that. And that's not what you want if A is a vector, basically.

Exactly. So you need to sort of somehow say that A is a vector.

Okay, so am I. Am I now doing that with non scalar of a plus Y?

So right now the design. We have a symbolic vector. You say equals symbolic vector.

I just want to type symbolic vector.

And. Yep. And symbolic vector has the non scalar attribute. So.

Yeah. So if you look at attributes of that and you. You could call it sort of a name or something, you know, like if you want to use it.

Well, I mean, I think we should. To give it a name that. That says. I mean it's. It's like a listability hold. Yeah, the analog for. I think we should Give it a name that is not mathematical, so to speak. Because really it is a thing that basically doesn't get ingested in the stability.

Yeah, but, but I. I mean.

Non listable.

But I think that you shouldn't say that. I mean that seems like very.

No, it's the attribute name.

Okay. That's the attribute name. That's fine.

Right. What we're saying is the attribute name is a structural attribute name. Symbolic vector is the kind of high level math oriented.

Yeah. Then we're in. Absolutely. Then we're completely in agreement.

Yeah. I don't think that non listable is quite right because.

Right.

It's not that this thing itself is not listable because you know, F by itself is not listable. This is like anti. Listable or something.

Yes. Well, let's think about what other attributes we have that prevent. I mean we have a lot of hold attributes.

Right. We have sequence hold. We have the various hold stars.

Right. But those are mostly. You are a thing and you are, you know, you're holding your children, so to speak. Although sequence hold. Yeah, they're all child holding type thing. Whereas what this is is.

This is more like protected in some sense. Right? It's. Yes, it's listability protected.

Thread protected.

What's that?

Thread protected is. So it's not threading into lists.

It's not bad. Except that I think that will end up wanting to use a name like that for something in multi. Multi.

Yes, exactly. I think that confused with computation.

Well, we want. I mean it's really. Right.

We have. All right, so all of our. All of we have constant flat. The whole family. Listable, locked numeric function.

Sorry, where am I putting this? Have I. Where did I put this before?

Do you have a symbolic tensors folder from when we. 10 years ago?

Calculus, algebra or something?

Yeah, that's what I would think. We've been doing a lot. Weird. I'm going to put it under math. Maybe I don't even have a folder for amazingly. Okay, all right, so we need a name for this. So this is the thing that is. Does not participate in listability.

Yep.

But we've got threaded.

Which changes how listability operates or broadcasting operates.

Well, what about unthreaded or something unthreading Un. I mean what it really is. Right.

But it really is as non threadable.

Well then why don't we say that? Why don't we just say that?

Okay, I. I don't have a deep objection to that. Does somebody else have that looks fine.

So non scalar becomes non Threadable.

Yeah, yeah, Yeah. I think it's much better to say what it's doing or not doing. Okay, all right, I get what that's doing. Okay. Symbolic vector. If I do this.

Yeah.

If I say symbolic vector and I say two times this symbolic vector plus symbolic vector. And I'm just going to, I'm going to ask a different thing in a moment. Okay. So then I, and then I say something like this. What happens if I say this dot itself, for example.

That has the potential to evaluate to something. I can't tell you exactly what it will do. Maybe full.

Yeah, certainly we don't have an expand right now.

Does full simplify do anything?

No, right now we don't have any operations that would do anything with.

But in principle we could. Right?

Yeah.

Yes.

Special expand.

Like we have some simplifiers, but there's a lot to come when it comes to sort of simplifying these symbolic array expressions.

Should this actually parse?

What do you mean?

No, I don't think so.

Okay.

Oh, you, you injected this, the typesetting type thing.

Yeah, he typed it in from by scratch. And, and the, the problem is, first of all, right, that A is a symbol, not a string. And you know, and there are additional arguments.

Why does it matter that it's a string? Why couldn't it be just any expression inside the symbol?

Well, it can be any expression, but I, I think in general, you probably don't want it to be a symbol in there. Tell me why in the typical. Because you know that. Then suppose you say V equals symbolic vector of, you know, V, then presumably you want to be right. It's the old problem of, you know, what, what is the vector?

Yeah, all right, fine. Okay. Okay. All right. So now if I take this there, I don't even know what it means. Can I have a symbolic vector function A of X or something? Or what do I do there?

Yes, yes.

Yeah.

So I mean, if I say of X here, of X there. Okay. Now I say D of that comma X. Didn't do anything. Yeah.

Actually needs some dimensions of the vectors so that it can check that they are consistent. So. So if you have A comma N, B comma N, then it would know that it's.

Right. So this is a vector of indefinite.

Does it really need to know that to do. I mean, some operations it can do even without knowing the length? Concert.

Right. But how does it know that they have the same length?

Well, how does it know that, you know, any of these things which we assume are compatible have the Same length, so to speak.

Well, it doesn't.

It.

Right. It sits there unevaluated.

No, I know that, but aren't there cases where we basically make assumptions about variables being meaning the same thing or meaning whatever? I mean, I'm surprised that we can't do anything with arbitrary length symbolic vectors.

We, we are doing things with arbitrary length.

You're right, but the problem is you added them together. It didn't know that A and B was multiplied.

No problem. Okay, I don't like that notation for A to the N. That really doesn't.

I don't either. So I think we should come back to that. There's a whole.

Yeah, the whole typesetting should be considered very rough. You know, with the infrastructure issues we've had the last three weeks. That's much further behind than where I want it to be.

Okay, all right, fine.

So in general, a lot of functions like linear, solve and dot. Well, dot actually has a nice form, but almost everything else doesn't have a nice typeset or didn't have a nice typesetting form. But now we want to deal with a lot of things as formulas that we used to only think as programs, and all of a sudden it's relevant.

Yeah, maybe. I don't know. I mean, you're not going to introduce some weird note, iconic hieroglyph for LU decomposition or something, are you?

No.

So what, what, what kinds of things are you talking about?

Identity matrix. Those things. And if you see them in formulas one's matrix or zeros matrix, when you see them in formulas, they have very definite type settings. So you get nice formulas, I guess. You don't see a race. You're a scalar guy.

No, I just think that, I think that maybe there's an obvious thing to do. Maybe there's an obvious way to do that so that it can be generalized from. Okay, but in any case. All right, I, I know, right?

You, you can type in, you know, one's array of three of list three, comma four, and you'll see something. Right? So it's a one with three. Yeah, I, I wanted to use double struck. Like we use exponential E and imaginary I. But our double struct zero looks really terrible. So if we make a suggestion.

Okay, it's an array. Why not put a box around it? Why don't have. We live in modern times. Why not do something like this? Why not do something where it's, let's say, just, just for the sake of argument, it's a. That doesn't look very good. But, but, but you get the idea where it's something that has.

Well, that's very, very non standard. I mean.

Yeah, well the standard was made in 1960s, right. When computers were coming in. But when computers printed.

Well, well I think it predates that by, by, by several decades.

I mean if you read any modern book on machine learning statistics, they use, they use matrices, they don't just use scalars and.

Right. They, they use bold face variables. Sometimes they use double lines. That's a not super common standard, but I chose to use for matrices specifically I chose to use left right vector to complement right vector for vectors. That's again changeable, easily changeable. So if you did symbolic matrix of a, you'll see a left right vector.

I've never seen this myself because I haven't had it working.

Okay.

Version for three weeks.

Look this, I mean then I don't understand why we don't have a one with a hatted thingy dinghy above it. See what I'm saying?

Right. Well I, I. The reason I chose to. So actually so if for symbolic array we use the, the rank as the, as the little hat. So if you want me to add that to ones array and identity array I can do that.

Well, okay, listen, listen, listen, listen, listen. I don't think it's the main thing you want to talk about. We've got to talk about notation but let's just do the zoomed out review of what this is all about.

So the first thing is there, there is a generalized chain rule.

Yeah.

And that's what I would scroll down to in the notebook on the.

SO array chain rules.

Yeah.

Well, yeah, okay, well it's just the chain rule, right.

Well but you get an actual chain rule. If you don't use this definition you will not, you know, or you will get a chain rule that goes to the different direction F and G. Here.

Are those now symbolic vector.

Yeah, don't have to be. The chain rule is true anyway.

No, I understand that but I'm asking.

So what you need, what's new is that the way that you compose these things. See what I'm saying? There's a different, our dot product only contracts one dimension. But if you compose, if you let's say have a matrix argument, you need to interact with both of those dimensions, the row and column dimensions. So you need a different dot product. A K dot effectively.

Right.

I don't really understand that.

I mean it's basically tensor, right? It's tensor times followed by tensor contract. Right? You're respectively dotting the last k indices of the first tensor with the first k indices of the second tensor.

So it's ky dot so to speak. Yes, rank k dot. Yeah, yeah.

If you do that, you get this generalized chain rule that works for any.

Okay, fine.

Any rank array.

Okay, let's just, let's just zoom out for a second. Okay. We have the concept for tensor that we have for.

But according to tensor means something else than arrays.

I, I understand. Why is this taking forever? This is crashing. Oh, whatever. Okay, what is the. I'm just trying to understand. We've got a thing called symbolic tensors right now. So I'm just, I'm just trying to avoid what we've done way too many times, which is, you know, we have one thing, we have another thing. You know, we've got symmetrized array, etc. Okay, so we've got this notion of a tensor here. So what does this operate on? Is this operating on explicit tensors? Is Jose here, by the way?

Jose is not here. I can speak to this question though, or. Right, so the, these tensor operations operate on symbolic tensors declared via assumptions.

Oh God, what a mess. So you're coming up with a parallel way to do this, is that correct?

It's. It's somewhat different, but yes, very bad.

No, these tensors are supposed to be physics tenses with, you know, map.

Are we going to deprecate these in favor of these or what are we going to do?

These are arrays as used in applications like statistics and machine learning, signal processing. So these are not used in those domains at all.

Okay, what's this? What's this? This is the nonsense that I was talking about. This is the horrible nonsense that I was talking about.

That's the thing you insisted on, right? We had a symbolic tensor object and you didn't like it, which is why and insisted we use this assumptions mechanism. Right? So. So yes, that is exactly what happened.

Well, I think for other reasons, but.

And, and Right, so this operates both on explicit, both on explicit arrays and these arrays declared via these things. And I think it should also operate on these symbolic tensors. I, I don't know. Or these new symbolic arrays.

Ok, so we have to pull these things together.

No, no, that, that's doable. Okay? They. Because that mechanism, that assumptions mechanism does not solve the listability problem. Listability curse at all. You cannot mix lists and those. You get in deep trouble.

Right? You have to use all explicit lists or all. Or all symbolic lists because otherwise, you.

Know, I get it, I get it.

Or you have to do inactive plus and things like that.

Yeah, okay. All right. Okay, so I understand. So let's try and pull these different design threads together.

No, but that's not going to be a problem because the, the symbolic arrays is strictly more powerful than these things that evaluate assumptions.

Well, okay, but, but I understand that. But like for example, if I say, you know, symbolic. Oops. Okay, that should work. Right? Right.

I mean, you don't even need to do that.

Yeah, you don't. Right.

I mean, it could work. Didn't think about it.

Oh yeah, but you've got to think about it. This is how we make a mess, okay? This is how we end up with multiple incompatible things. This is what we're trying to avoid. This is what happens when people just build random stupid libraries. They end up with seven different ways to do things. That's what we try to avoid. So let's please not ignore. We've got this design. We either have got to deprecate that design or we've got to interoperate with it.

This is not a problem. To interoperate the new mechanism. Strictly more powerful.

Okay, fine.

It's not. It's not strictly more powerful. It is more powerful in some ways.

Okay, so we should be able to test. So what we've got is a type declaration here. We've got yet another type declaration in the compiler, I think. Is that true or does it take these type declarations?

I mean, these are different from what we have because. Right. The compiler wants, you know, machiny sort of types, and these are mathematical sorts of types.

I don't know whether it enter operates. Okay, so tensor rank operating on a symbolic array will give something, right?

Yes, that is one of the things Jose and I made sure.

Okay, so if I say of this tensor rank. Great. And tensor dimensions, if I say dimensions of that.

Well, dimensions is a structural operation. You need to say tensor dimensions.

All right, I'm just asking. Okay, so that said that for very meaningless reasons.

Yes, but if you do tensor dimensions, you'll get what you expect.

Okay, okay, so let me just understand this. Like the symmetric thing over here, is that going to be supported in symbolic array here? Maybe not yet, but one day.

Yes, it's already. I mean, I don't know how extensive the support is, but the supported in.

The design, it's not supported anywhere in the computation because the derivative doesn't care.

Well, I mean, but if I say.

We should talk about that separately, it does.

But if I say something like this, three, three symmetric, I would.

How would I specify it symmetric of all or symmetric of 1, 2, depending on which notation you want to use.

Like this. Yep. Okay. Okay. And then I say transpose. That.

That should probably be tensor transpose, because again, transpose works on general heads.

Okay, so that's working right now. But it should correct, you guys. Everybody agrees that should work, right?

Right.

S should transpose.

Well, it should return its argument. Right, because you've declared it to be symmetric. So. Yes.

Right, but we need to have transpose in formulas. That's how you write array matrix formulas.

Well, but are we going to have transpose or tensor transpose? And how are you going to know which one to use?

Transpose should work. I mean, I'm just saying simplifiers don't necessarily work. Now.

Derivative, the operation of transpose would work, but I don't think the head transpose should work well, because.

Roger, what do you think?

Because transpose works on general heads, transpose.

Works for these symbolic. You know, we, we have lots of support.

We're overloading transpose for this case so that it does not do what you think it does, so to speak, Just like we overload length or dimensions for sparse array. I would argue we should overload dimensions for the symbolic array case. Okay. Okay, let's see.

I would say a problem with doing that is that, you know, right now dimensions can be predicted to spit out positive integers. And if we overload it, that will.

I see. It could have an N in the symbolic array.

Correct?

Definitely.

Whereas tensor dimensions can already return N because you can make assumptions about it being an N by M array.

I'm skeptical about this. Okay.

There might be code that expect that if dimension works, it can extract parts.

How is somebody going to know it's transpose works, dimensions doesn't. Who's ever going to know? It's not right. It's not right. We can't do it this way.

That looks safe. I think. So dimensions on any one of the new objects should just treat them like concrete vectors, matrices, arrays, and return the dimensions of that list. Yes.

I don't understand that. M and N, what does it do? Look, I mean, this is the bad case, right? This thing is N comma three. Okay. It just does that.

Right. And, and the, I mean, the reason it makes sense to. Right. The reason it makes sense to overload, you know, sparse array. Is that right? You can then use those sorts of things to take parts of it. You can't take. Right. You know, the part one, two part of the symbolic array. I mean, the reason why we created tensor dimensions in version 9 was precisely to deal with this issue.

Yeah, okay, okay. But I don't think it's explainable that. Okay, so let's go. You've got now some new thing. Array dot. What the heck is that?

That is the k dot. This is the rank K inner product. It is a generalized inner product.

That's fine, that's fine. Okay. Something like zeros array and ones. Array is a symbolic array, right?

Yes.

If you have concrete numbers, it becomes concrete.

If you use normal.

Oh, oh, no.

Okay.

You want to have it in?

Well, I haven't been able to play with it because we have had no builds for a long time.

And Chronicler Array is a problematic name because it's not the actual Kronecker Tensor. I'm not sure what a good name is.

But what is it? What is it?

It's value is 0 or 1. It's 1 if certain indices are equal.

Okay, how does it relate to Kronecker Delta?

So Kronecker Delta is. If both indices are equal, then it gives you one.

So Chronicler Array. I think Chronicler Array might make more sense as a two argument Ones array. It's an array where you give conditions on which subsets of indices want to be equal. So if you impose no condition, it's one's array. If it is. If you. If you have rank two and you declare that. That the two indices need to be equal, then you get the actual Chronicler Delta. But beyond that it's. It's not. It's not what? It's beyond, you know, in the general one's array.

But it's zero if in the case where it doesn't satisfy the. The equality constraints defined in second argument.

Correct.

Yeah, it's a terrible name, but I mean.

Right. And. Yes, right. It's neither an outer pro. In the general case, it's neither an outer product of Kronecker Deltas or what people call the Generalized Chronicler Delta, which is, you know, an anti. Symmetrized outer product. Yeah, it is a product.

It's a straight product of Chronicler Delta.

Well, I suppose, depending on how you. I don't think that. No, I don't think that's true. Identity Array is an outer product of Kronecker Deltas.

Kronecker Delta would be like you have the J1, J2. So it would be chronic, er delta of J1 1, J12 times chronicler delta J2 1, J2 2 and so on.

Times. It's not times. That's not an outer product.

No, no, it's not. Outer product. It's times product. Like product product.

Yes. Yeah. So I guess it's a sort of inner chronicler array or something. You know, that's still Horrible name. I don't know what to call this, which is why I suggested a second argument to Ones Array. But maybe we need to think harder about a good name for this.

I'm not happy with one's array not evaluating with that name.

So if, if you do, it will come up from the practical point of view. If you, if it comes up in, in taking derivative and. And you have derivative of.

Right. So let's call it matrix. Let's call it symbolic ones array. Okay, so part of this family.

So then do the. Then there'll be symbolic zeros array as well.

Absolutely, yes. It really can't be one's array.

I kind of agree with that. In any case, we've got type settings, so it'll get hidden away eventually in the.

And then symbolic identity array too.

Yes, yeah.

Okay.

Yes. Yeah. I mean, array dot will work explicitly on explicit. On explicit lists of lists, right?

Correct.

And how does it relate to tensor contract?

So I should.

Tensor contracts.

Yeah. So basically if, if, if you. So tensor dot, sorry, array dot of a comma B comma k is first you take the. It's A, do the tensor product of A and B, and then you contract the last k slots of a with the first k sl b respectively. Right. So in order, right. The kth most goes into the first, the k minus one of a goes into the second, etc.

I'm sorry, sorry, sorry, I got distracted. Could you say that again, please?

Right, so when you so do Write down array.of, you know, a B, K. And so that's going to be tensor contract of a tensor B. So that's, you know, escape T star outer B. Yep. And then you're going to contract.

Okay. First the last k indices of A and the first k indices of B.

Yes, right. In particular, in the order where the R minus K index of A goes into the first index of B, the K minus R minus K plus 1 goes into the 2, etc. Etc.

It's actually on the array documentation page.

It. Okay, Just a second. Hold on one second. Okay, sorry. All right, look, we're not going to get through what we need to get through in this meeting, so I'm afraid we're kind of running out of time. So what. What else do you want to show me here?

Well, this is the big. This is the killer app right now for these symbolic vectors. Matrices and arrays is differentiation. We will do gradually more and more features that works on these.

Wait, what's this dot doing here? That's the ordinary dot.

Yes.

So in the cases where it's easy to understand dot like two vectors.

What's this doing here? Is that a symbolic vector?

Yeah, no, that, that's a. Oh, yes. I guess with nn, that's the unique.

Vector used in a symbolic way.

Because identity matrix of N. Oops, that.

Will just sit there.

Okay, but is that going to be treated as a symbolic. Yes, so that will be treated that. If I say. But if you do some example tensor dimensions of this.

Oh yeah, that's worked for ages, I think. Okay, and if you. And Right, but if you do. Right, and. But if you do identity matrix of three, then. Right. You obviously get a list of lists. But if you do what we're now going to call symbolic identity array of list comma three, that will stay symbolic unless you hit it with normal.

Sorry, symbolic what?

So type in identity array of three, which we just decided will be named.

Some symbolic array of three.

Right. So you need list, list three. List three. Because it's a list. It's not var args.

But it should. If it's an identity array, it should have a default for a single argument like that.

Maybe.

I think it should be an identity matrix in that case.

So this is an identity matrix.

I know. No, it's not an identity matrix.

That is an identity matrix. Do normal of it. That is.

Well, okay, then it absolutely should work to just put a naked three there. Okay. Okay.

I mean you could evaluate to the form with, with the, with the list there.

Right. So right, if you. Now if you do list 3 comma 4, that's going to be a rank 4 array.

Okay. Okay.

So it's, it's the identity.

And but so if I do identity.

Matrix of list 3 4, that's a, that's a square. That's a rectangular matrix. But if you do identity array of list three four, that is the outer product of two square.

That's nice. That's nice. But then identity. Symbolic identity array of three should just be a three by three, surely.

Okay. I mean that sort of precludes the possibility of, you know, you put in a variable there that you'll later fill in with the list.

I think it's more useful to be able to just say of N there. I don't think you'll ever remember to put the list in. I think it's better to just say N then.

Okay.

Okay. What else are we going to achieve right now? Assuming that this interoperates properly with the old arrays stuff, I think this is going to be okay. We need a name for that Chronicler array thing.

Maybe the ones array with second argument would be good enough I guess.

No, I'm not sure. I think it's a complicated concept. What is it called? Sort of.

Mathematically I couldn't find any, any, any good name for it so has nothing to do with Chronicler, it's just times of Chronicler Delta.

Okay. But what it is is it's a collection of. It's. Yeah, I, I kind of have to go, so.

All right, well we can pick.

Let's see what else. We have a lot of other stuff here.

Yes. None of it. The rest of it is as big as this. Okay. Conceptually the one thing we started on is.

Oh, you do get binary nand. How nice does that work?

Yes, yes it was merged already and it's in dev. It should work over there I think.

Good useful to me. Well are we in fact going to make the change?

Yes, it's already made and ready for.

Well that's still. That's something that we'll do discuss tomorrow.

Yeah okay. Oh yeah, that's part of the other media.

Sorry. Yeah, it looks like there's not a lot of controversial stuff in the rest of this.

So one, one amazing thing that you get there is a generalized array Taylor series that you get out of this. Isn't that amazing?

Yes, it is. What's it useful? I mean but, but the concept that you can use a symbolic matrix as a function head is not obvious to me. See what I'm saying?

Yeah.

So I mean the fact you can take this and use that as a head.

Yeah it's not.

How, how is that any different from team you knew of, you know, T comma r.

Yes. I mean no, I understand that but I don't think that's obvious to people because you know, you can't take them. I mean if I take a matrix, an ordinary matrix, if I say, you know, identity matrix 3 of X, I've got a garbage thing. See what I'm saying? So I would not immediately expect it to work.

On the other hand, if I take.

A symbol A, sure it works but the fact is that.

But then it's assumed to be sort of a scalar valued function.

Right, but how do you get people to understand that a symbolic matrix which otherwise I would expect to just be a thing. It's the end of the road type thing, not a thing that you can use at higher order.

Yeah, that's a good point. How do you do that with sim? How do you assume that a symbol can be used higher order?

People don't. It's one of the things they have to learn. It's something that is surprising to people.

Right.

People know it because they know that there's a function F. But many of them think, oh, a function has to be called F or G. Nick is commenting that identity matrix of zero should be a null list, isn't it? Oh.

A null list is not matrix Q. So I don't, I disagree with that statement.

We'll address this in a different way, but we should bring this up again. But let's not. This is not for today. Okay? Symbolic matrix. This problem of how can it be a function?

I mean, I, I think it just needs to be examples.

I don't think that's the right approach. Let me suggest a different approach. Okay, let's imagine that we have a thing called matrix object. Not sure that's quite the right name, but something like that.

Well, that's definitely not the right name because I can think of at least 17 different meanings for that.

I mean symbolic matrix is a nice name, but why it just suggest it's the leaf of the tree matrix, so to speak. Get what I'm saying? So how do we avoid that? I don't think examples are enough.

I think with typesetting coming in, you wouldn't see it like when you see it in full form now, it seems very complicated.

Although people would still need to be able to type it in. Right. To start their problems.

Yes. So, but I mean, it doesn't feel like a. I mean I use matrix valued functions all the time. It's, it's, you know, and you do whenever you use statistics, machine learning and stuff, but they're just like capital letters, you know, like that's. And you say somewhere that it's, that it is a matrix.

Well, it isn't a matrix. The thing. It isn't a matrix. It's. It's a disembodied function thingy that when applied to X will give you a matrix.

Yeah.

Which is not the same thing as saying it's a matrix. When you say symbolic matrix of A, you're kind of saying it's a matrix. Not. It returns a matrix when fed an argument. See what I'm saying?

I mean, I, I agree. I mean I, I understand up to a point, but. Right. I can right now write, you know, F of X blank equals, you know, list, list, you know, Bunch of expressions in X. Right. And, and so, you know, I have an F and an X and it produces a matrix. Right. I, I, I, I'm, I just don't quite see, you know, why calling, you know, calling it symbolic matrix. Right. Matrices can contain functions and you need to express the, defend the, the express the dependence on independent variables.

Yes, but they, they don't usually, we don't usually do the thing of doing a through basically, or an operate, you know what I'm saying? Because what we've got is essentially M of, let's say, AB acting on X. Right? And that is supposed to turn into M. I think it's. What is it? Is it oper. I remember how this works. Or is it through? This was an operation I thought would be important, but it's just like nobody cares about. There we go. There it is through. See what I'm saying? I thought that would be an important operation. But that's the operation you're implicitly assuming happens, right?

Yes, yes, yes and no. Right. I mean. Right. It's right. The, the, the, the symbolic matrix is a single object. That object has some dependence. Right? It's, it's right. You know, a, a tensor field, right? It depends on variables and it produces a different tensor at every point of the manifold, right?

Yes.

So, you know, some symbolic matrix is declaring that you're, you know, I mean, Right. We don't have tangent and cotangent basis here, but you're, you're declaring that your value space is, you know, some product of cns, and that value can depend on some variables.

All right, look. Okay, we've opened this topic. We're going to have to discuss it further.

Okay?

I mean, we can make some of the cosmetic changes we already discussed.

Yep.

I wish there was a way of signaling that the symbolic matrix isn't. I wish there was a way of being able to say that it could be a function somehow in some very simple way, As opposed to calling it matrix symbol. Do you see what I'm saying?

Like add an argument to it and is that, would that be too much?

No, no, no, no, no, no, no, no. If its name was matrix symbol.

Yes.

You mean it be like symbol but matrix?

Well, yes, but if it was, it was called matrix symbol, it would be unsurprising that you could say this. Okay. Do you see what I'm saying?

Sure.

I mean, I don't. It's okay if we have vector symbol, matrix symbol.

Right? But I mean, that's a, that's a possibility. I mean, it's A, it's a, it's a little bit typeish, if you understand what I'm saying. Right.

I would, I mean I, the, I would, I would only, the only downside I see to this is. Right, you know. Right. We have a nice family of symbolic star things that are all easy to find in this way.

In this auto completion. You'd find it with fuzzy auto completion. Just type, start typing. Symbolic, you'll find it. That's not an argument to me. I mean the, the argument to me, I hadn't really internalized this point about symbolic matrix. Just seems like it's a frigging matrix. See what I'm saying? As opposed to it as symbolic construct that can be used as a symbol like thing, but it always represents a matrix. Okay.

And, and do we make the same name change to the other guy? So one array symbol and that's.

Yeah, all of them.

In that case, I don't know about the one array thing. I'm not quite sure how that works. Okay.

I mean one array, I don't know. But the, the other, the vector, the matrix array.

Yeah, obviously. Obviously.

Yeah. Those three, it's obvious they have to go together. I, I just think.

Right. But I mean, Roger and others. And others. Okay. I, you know, looking at this, I really do prefer this because it really, then, then is completely unsurprising to me that I can say matrix symbol of a, of X.

I, I don't disagree with you.

I think it's, Yes, I, I, I agree it has that advantage.

Okay, what would we then call one's.

Yeah, so I don't think one's a race symbol makes any sense.

No sense.

So it would either.

Well, I, So in other languages like NumPy, the Python NumPy, or MATLAB, it's just ones or zeros.

No, because they, it's because it's a freaking Fortran library that just does linear algebra. So it's a big, you know, so it's, but it doesn't, it evaluates in that case. Right?

Yes, yes.

Yeah. So it's not really relevant. I mean, what, what you're defining as a symbolic array of ones.

Why don't we just skip the array business and call it symbolic ones and symbolic zeros?

Possible.

And then it's still not super long.

Anybody's ever going to understand that.

Yeah, I'm not sure. I, I, I'm not sure I see this as an improvement.

Well, okay, but I, I don't think.

I mean it's, it's going to.

Why is it symbolic? What aspect of it is symbolic.

It's symbolic in that the. The dimensions can be symbolic, but it represents an.

What if it isn't symbolic dimensions? What if it's just 2, 3?

Then it represents it. It represents the 2 by 3 matrix of all ones. And, and the reason we don't want it to necessarily explode is that then every single time you use. Right. Every single derivative will blow up. Where you have a concrete dimension will have these mat.

Let me just talk about something here. So matrix symbol is something that represents a. Okay, hold on. What this is. It's a matrix symbol whose value is specified to be. Not an arbitrary un. You know, symbolic thing, but it's something that is. It's an array of ones that if you normaled it would just be an array of ones. I mean, I'm wondering if we called it ones array and zeros array whether that would be sort of symbolic enough.

I think so. I mean I think it looks kind of also easier to remember and use.

Is that what you called it before? Is that what you had?

Yes.

Yeah.

Well then, then matrix symbol and ones array. Then ones array would be a thing that is a symbolic ones array in effect wouldn't evaluate immediately.

Correct.

As you have it. I mean, so exactly what you have right now.

Yes, exactly. And you. We. Yes.

What about identity array? That might be okay too. I don't know what the chronicle one does but. But, but then, you know, this really helps me this. Yeah, Matrix symbol, array symbol really makes a lot more sense to me. Oh, that's an interesting idea from Nick. I think this isn't a winner, but it's an interesting idea. Array symbol of ones. A named array. I don't think it's quite right, but it's an interesting idea. I mean. Or array symbol of purely one there. Because what else is it going to mean? You see what I'm saying?

I find that this highly unnatural. Right. I mean. Right. Just as symbol of ones creates a symbol named ones, I would expect array symbol to create an array whose name is ones, not one which.

No, no, I'm not proposing this one. This one is not a winner. Right. I'm asking about the one below this one here.

Right. So what would you have for identity? And it would be weird because this thing would not make Right none of the. Well, I guess the second argument would make sense, but the third and fourth wouldn't necessarily. The third is the. Is the domain and the fourth is the symmetry of the array symbol of the array symbol.

By the way, I really prefer array symbol to symbolic Array. Array symbol. Makes sense.

That is okay. But the other idea is not, I mean, it's just really not.

Which other idea?

Array symbol of one.

No, it won't work. It won't work. It won't work. But I do like the fact that you say array symbol of A and it is an element of arrays. That seems to make sense to me. See what I'm saying? Yep.

Yeah, I think those look nice. They also highlight the matrix array of vector a bit more than the symbolic part, so.

Yes. Which is also nice.

I mean, so it's kind of, it's convenient for us, but for people to look for. It might be easier to look for.

Yeah, whatever. That's because they assume that they're alphabetically ordered. But I mean, that's.

Yes. Yeah, but that's okay. I think it's, I think this. Fine.

The new names, they are alphabetically ordered, just in reverse alphabetical.

Okay. But we still need to come up with ones and zeros and identity and the name for the Chronicle.

Oh, I, I, oh, I thought we were settling on one's array. We're saying we're not.

No, no, sorry. One's array is okay. Yeah, that's okay. Identity array is okay. All right. Yes, I think that's okay.

So it's really just chronic or array that we need to figure out.

Right. And then we've got to figure out typesetting of all these things.

Yep.

I have to say that I think that the, I mean, the bold face Ish convention that's used for matrices is quite nice.

Yeah. So here's what I thought we would try to do is to make a system that gave users some choice and that could be sort of named styles maybe.

I mean, we've wanted to do that for 40 years or something. I wanted to do that.

Oh.

Since 1979 with, you know, give users a choice of how to represent formulas and things. Styling. Was it X +1, 1 +X? Things like that? I don't think we necessarily start here with that type of issue.

I, Yeah, I just think that, well.

In a sense, especially this is sort of more contained than that. Right. Because we can. Right. You know. Right. We have these like eight symbols. Right. And I can, I mean, I've had some preliminary discussions with John about how we could do this.

And you know, with something form. Right. It will be, you know. Well, maybe, but no traditional form or something. It would get kind of weird.

No, the, the idea would be that it would be, you know, a front end option that you could set and if you want some helper function?

I don't believe in this. I don't believe in it. I think we should do the work to try and get a good default and people will be okay with. I think, I mean, I think traditional form could have various. And we could have various things initially in the function repository which are ways of formatting these, you know, matrix valued objects and things using some particular convention and then we could see what, you know, how that, how that feels. But I think, you know, as we have done for many, many things, we should just come up with a canonical way of representing this. And it will be good. I'm sure. Roger's done a literature survey of what people have ever done in the past. Like I've seen things that use subscripted double brackets. I think for, for dimensions. I don't know if that's, you know, I don't know what makes sense there. That doesn't really work for our language where subscripted double brackets are part extraction.

Yeah. I think that for, for most things, if you look at vector matrix formulas. Array formulas are more rare. Okay. But those two, you almost never see dimensions explicitly on symbols.

Yeah. Well, I personally think if we're going to do that, my opinion would be the dimensions should go. Okay, here's what I think we should do.

But, but, but I think that there could be a nice debugging mode to have dimensions. But I, I don't, I just think.

We should do if I think, I think we should put the dimensions up above like that.

But I don't think we should force people to have them. It's just not going to be what people do in these formulas ever.

No, I understand. It's just like a symbol doesn't say what it's. I mean, I, I think that might make some sense. Okay. I think if it can be made to look right. Typesetting wise, I think it might be make some sense.

Okay, well there's plenty of discussion to be had here, so.

Yeah, I mean the problem is that I think. Yeah, anyway, that's my suggestion. Maybe a small three there.

They could be tooltip also.

Yes, there could be. That's not bad.

I think two tip looks like a really good idea. It kind of isn't.

So array. Right. If you do array with, you know, symbolic array of, you know. Yes, but type it in now.

Yeah. Okay.

Symbolic array of, you know, whatever name and then list 2, 3, 4 or whatever.

Right.

So then hover over it.

The hell is that three? What does that three thing do is that you type it is that your. Oh, you did a three. You did a top. A three on top.

And Right. Because that's, that's the, because that's the rank.

Yeah, it's not bad. It's really not bad. Yeah, I think it's quite nice. And the bold face A is actually looks quite decent there. The problem is that some people use bold face. Okay. In, in antiquity, so to speak, Europeans tended to use boldface for vectors and Americans tended to use that arrow superscript arrow over script arrow.

Yep.

But there is sort of a handful of conventions that people use. I don't know how strong opinions they have now. The numbers I've never seen. But in a pedantic mode or debugging mode it might be useful. But that's not a formula you want to see, typically.

Well, I'm not totally sure about that. I'm really not sure.

Yeah. And also. Right. And if it's not displayed, it also means you can't edit it. So.

I, I'm not convinced. I think this is not bad and I think having something that's a little bit exotic for more exotic rank 3 tensors and things is not a bad thing. There's a notation for vectors and you know, let's say we went with the, I mean the two way arrow thing for vectors I think is fairly obscure.

You mean for matrices?

Matrices, I meant. Yes, yes, I think that's fairly obscure. Yeah.

It's, it's not super common. I mean it's not unknown, but it's, it's not very common.

Well, let's inventory everything that people have used and then try and you know, take it from there. And then let's get another meeting about this. Let's include. I, I mean remember, remember we live in modern times where you can do things like put gray backgrounds around things, put boxes around things and so on. You see what I'm saying? We're not constrained to have things that can be done with lead type. Right. Nobody's responding to that.

I, I agree and we've been talking.

Yeah. Yes. But I also think that, you know, it can't look too exotic. We don't want it to look too exotic. I know that if you, and if you make it sort of, you know, really heavy like a grayed out box, and now you have, you know, 50 of these in your result, then that's okay.

So I've got an idea. Okay. Another convention is underlines for vectors. Right. Oh, for God's sake. I thought that would do an underlining. Well, anyway.

Maybe underscript.

Well, I. I think it's a style. I think underlined is a style.

Right. Okay. Doesn't look very good there. But. But you get the idea.

Yeah.

Okay. My idea is this. There is a convention for vectors that says put one underline underneath it. People know that convention. Yes. No. Maybe.

It's pretty rare, but yes, I've seen.

Yes.

Okay. So one possibility will be an underline and an overline to represent a matrix.

I think it's more common among people who do that to use a double underline for matrices.

Okay, well, anyway, let's get an inventory of these notations. Okay. Okay. I. I think we just have to have a notation where when people see it, people say, oh, I know that what that is. It doesn't have to be. Oh, I use that when I hand write it. You see what I'm saying? Do you understand what I'm saying?

I understand what you're saying. I'm. I'm not convinced there exists such a notation in whatever I think there is, given. Given the multitude of. Of conventions.

Right, I understand. You just got to have it something where if. If you saw an A with a double arrow onto, you know, with an arrow both ways, you'd probably figure out what it was. Although you'd say that's kind of stupid.

Do you sign a star? Is that element wise? Conjugation? Is that transpose? Is that conjugate? Transpose? Is it adjoint?

I have no idea. It could be any of those things.

Right, exactly, which is why I'm saying there does not exist.

Yes, but. Okay, let's just inventory what's there. Let's have another meeting and let's go over this. Okay.

Okay. Which is what I've been saying for the last half hour.

That.

Okay, very good. Okay. This is nice stuff. I really like matrix. Symbol Array. Symbol. That really helps me on this thing. Okay, see you guys soon. Good stuff. Bye.

Bye.