Live CEOing Ep 826: Language Design in Wolfram Language [Tabular]

Foreign.

Okay. Hello, everyone. Welcome to another Wolfram Language Design review about the subject of tabular. So, yes.

So the agenda that we have today starts by looking at these two pages. Row indices and column indices, which are two functions that we use a lot and we have never looked at in detail. Is this large enough?

That's fine.

Okay, so these are functions that take tabular object and then a row spec. And here there are several forms of row specs and return the list of indices corresponding to those rows. And we have the similar function for columns.

I don't understand. You mean indices column. And so if I pick a row like row two, it has certain columns that are non missing and maybe others that are missing or something. Is that what you're talking about?

No, this is more like find the rows with some property. It's a bit like position, only we don't call it position because it returns a list of integers and not a list of list of integers. And so, for example, we can do selection by value. Find cases, find the permutation that would order the.

Okay, so hold on, show me an example.

Yes.

I think the big extension is predicates or patterns. So based on values, not just for.

Example, these are those rows which satisfy some criterion.

Exactly. So that's a tabular object. And.

So why isn't that just select itself? I don't understand.

Well, select itself would return a subtabular, while this returns the indices of those columns. Because you may want to do mapping or extraction, whatever. So this is some sort of position function.

Yeah, but position takes a pattern, whereas this is taking a predicate. Right. We don't have an analog of position that takes a predicate. Right.

Could have an arrow pattern and then we'll work too.

Say it again. What?

We could have row indices of a tabular and then a cases operator form with a pattern.

No, I understand, but why isn't. I mean, okay, if I just want to know, in a list, where are all the things that are greater than 7? I can do that with a pattern in position, but we don't have a function that just does that. Right? Right.

Correct.

So perhaps we should.

Jose. I mean, I for one never really liked the fact that this stuff with raw indices is always done in two steps. Like you get the indices out, then you have to plug in the indices. And I understand that if you work within memory databases, it really doesn't matter because no one is going to change the data from under you. But I feel like that if you grow to larger databases, this is probably going to bite you back. And I feel that column indices doesn't have this problem.

What is column indices doing?

Column indices is the parallel function to select columns. So there is this other function. And let me put it bigger too. And so we have a tabular and a column spec. And so again we can do selection cases, sorting operations.

Look, I mean, I just don't understand why select itself doesn't do this for rows. Why is select on a tabular? I mean, other than the fact that we don't have a function that is an indices function or something, or select indices or indices select or something like that, which could be like general function.

Yes. So I guess we were trying to have operator forms of various cases so that this is very powerful. So we can have various forms of.

Yeah, but it's one of these things we have to look in the documentation to know what works, isn't it?

Yes, I guess so. Another thing that we were doing here is to offer various output forms. For example, do you want a party specification or a list of indices or a boolean bit vector, which would be usable in other places.

Right.

And so we realize, and there is this example here. If we take the IDs tabular object. So this would be a selection giving those rows in which the species is setosa and then we can extract that. Right. But we can do the same thing as you are saying directly with select, or we can use a quality like specification. So yeah, this is just an alternative method because you may be interested in doing operations with. With these indices that you.

So for instance, let's say when you know those locations or indices, you might want to highlight those rows. Let's say that you have some criteria, you find a subset, you might want to highlight them.

You mean visually?

Yeah, visually.

Okay, what would be the function that does that?

Well, you have to set an option like say row styles to those things or row highlight.

Right.

And then you can do other kinds of processing on those things too. You might want to modify your tabular in just those places.

Okay, let me understand what the select or any other function that's put in the second argument. What does it get as its argument for rows?

It would be the whole row as an association.

Okay, so any function that I put there, I can put any function I want.

Yep.

I don't understand. I just completely don't understand. How is it returning? The select has to say it's there or it's not. It's a hack, right? It's not actually. The select is returning when you say select. Right. Look at that. Example that you have there, right? The example there.

Yep.

Okay. I don't understand this. This is a hack. Because if I actually run Select.

Yes.

What am I running the select on? It's a list that. That select is being run on a list of row. Row associations. Correct?

Exactly. Yes.

Okay. So whatever I have. So I'm running it on a list of row associations and then pacilationally the select is returning certain ones of those rows and something is magically seeing, oh, it didn't throw out that particular row, so I will return its index. Am I right?

I'm. I'm not sure I understand. So select, that's the obvious thing, right? Is selecting the rows in which the.

No, it's not. No, what it's doing is it's taking the list of associations, right?

Yes.

And it returns another list of associations that has only some of the associations in it. Correct?

Yes, exactly correct.

Now how does it go from that to the list of indices?

Well, it's giving us the positions in this.

No, it's not. It's not giving the positions. Let's say there were two identical rows, okay. Two rows that were absolutely identical. And for some reason the select is using outside metadata. And, and, you know, so it, it can't know which position. I mean, you're hacking it backwards, right? You're looking at the actual associations that survived the select, and then you're doing a position with respect to that original list. It's defined.

It's actually different, Stephen. It's. But you're right, it's a bit of a hack. But the way select works is by choosing which indices match the select and, and then using that part. You know what, it's a hack.

You've got to go into the insides. It's going to work with select. If I use a different kind of thing, it's not going to work. Right. It's a hack. You've hacked select specifically, Right?

Yes.

Okay. That's not going to work. It's not going to work. It's the same mistake we made with Data Set. It's not going to work.

But this is an operator form. It's just an operator form that allows us to pass a function.

No, it's not. It's not. It's not.

Not.

Rob just admitted the issue. The issue is select can take a list of associations and it will give back another list that contains only some of those associations. Okay. That's a well defined operation that select does. But that's not what we're having it do here. Here we're hackerating into the insides of select to figure out which indices it's picking. You see what I'm saying?

Yeah.

It's not a well defined top level operation. If I put another function in there, like cases, for example, where I, I match patterns for these, you know, for these associations, you know, it might or might not work, right? It depends on whether it's been hacker rated or not. You see what I'm saying?

Honestly? Not. I mean, to me this is just.

I mean, I think we could use a name, doesn't it?

No, no, no, no, no, no, no. Understand why this is a hack. Okay?

Yeah.

Select. When you run select on the list of associations, what do you get back?

Another list of associations, right?

How do you get the indices from that?

You can't.

So what's the problem in saying how do you get the select positions? 1, 2, 3, 4, 5.

But you can't, you can't, you can't.

You can't get the positions right.

You have to do it as a post processing step with position or you have to hack the innards of select.

You, you hack the innards of select. You're.

No, I understand it, but I want Jose to understand this point. Right? Jose, do you get that?

So I understand that the rob has to manipulate internally the these to find the. The. But. But to me that's an implementation issue.

No, no, no, no, no, no, no, it isn't because it means there's not a well defined semantics for what world indices does. If row indices of tabular comma F applies F to a list of associations and returns the indices of the associations that survive, that's well defined. But it's a hack, right? Anything other than arbitrary F there is a hack. See what I'm saying? Right?

So if we drop select.

Would it make sense then? Well, then it would be something like no, but then we can't use anything. Do you want something other than select there or do you only want select there? Did you understand what I'm asking?

Function.

Okay, Is there any other function other than select that wants to go in the second argument of row indices?

Cases. If you want to use a pattern, you know, from a pattern language.

Well, that's cases, that's cases. You can't use a naked pattern. It's got to be with cases, right?

That's why we have a headdresser to them that says, hey, I'm taking predicates, I taking patterns.

I understand, I understand. It's not going to work. It's, it's okay. So let's Unroll what the issue is. Right. First issue is neither select. Okay. Cases has position, which is similar to this index finder. Yes. It takes a pattern, right?

Yes.

Select has no indices analog. Right. Am I making sense? Okay, it could have one. It probably should have one. Okay.

Yeah.

Okay, but you just said it was a hack.

No, no, no, no, no, no, no, no, no, no, no, no, no. I'm saying a top level function called select indices or some such other thing that all it does is to find these indices using internal code, using the code that Rob is using is hacking into. In the case of this row indices function. Am I making sense?

Well, sure.

Can I, can I put it another way? Well, basically what you're saying is that the thing that moves from predicates, it's is match Q and the thing that moves from predicates to. Sorry, from. From patterns to predicate. Sorry, from predicates to patterns is condition, not cases and select. You operate directly on the predicate you use. Condition.

Yeah, yeah, okay. I mean that, that would be. Right. That's right. So in a sense, one of the things would be to say that this function row indices fundamentally takes a select spec as its second argument, but it doesn't have the word select in there. But if you were to want a pattern match, you would use match Q of A and that would be your, you know, that would be your predicate, which is now a pattern. Am I making sense?

Sure. Yes. Yeah, sure.

Okay, so then, then the alternative would be that we've got this thing called, you know, I don't know, row select, let's say, okay, type, type that in as an example.

Yep.

Okay. Sorry, I'm not seeing it. Are you typing it or did you type it? Did you type it? Jose, are you typing it or not? I think he's having some Internet problems.

Okay.

Because.

Yeah, I, I, I did.

We're not, we're not seeing yet this. Okay. In any case, to me this seems like. Okay, so row select with match queue. I mean, we could have row cases as well, but, but again, row select isn't really the right thing because it's going to return indices. So I think there are two different things we need to think about. No, there's no cases there. Get rid of the cases. Get rid of that. Cases. There's no cases. It's match kiss, match queue. Okay. So that raises the need for a new function. Did you, Sorry, I'm just the, I.

I think there's some delay in the connection.

Yeah. Okay. All right, so there's a separate Thing which is select indices or some such or what should the name be for an ordinary function having nothing to do with tabular, just a standard top level function that gives you the indices that match a certain. A certain that. What? Do we have another way to do that?

Yeah, you can do position with.

No, no, no, no, no, no, no. Without condition. Without condition. I'm asking, is there a single function that you give it a predicate and it will give you the positions that match that predicate without using condition in the pattern.

No, I mean pattern test equivalent condition.

No, no, no, but that's. No, I'm not, I'm not saying that. I'm saying that, okay, if you type it in, type in position, list comma. Okay, now is there an operator form of condition? No, no, no, no, no, no, no, no, no. Yeah, that's right. That's one possibility. Okay, but is there an operator form where you could just say position of blah comma, condition. See, there could be an operator form. Do you understand what I'm saying? There could be an operator form of condition that has a blank in it, right?

No, I mean, I think we have to do this.

What does condition. Can you look up the documentation page for condition, please?

Yeah.

By the way, that's not condition. Condition is a semicolon column.

Yes, yes, I was.

What are you doing? We do just want this standard existing. We don't need the source page. Okay, so it's a two argumentation.

Yes, I don't have the documentation we can go to.

Okay, so one thing we could imagine is a one argument version of condition. You see what I'm saying? Do you understand? You there? Hello? Hello? Anybody there?

I'm thinking about it.

I'm just trying to understand it. Yes.

Think about it like accept, except has, you know, I think if I'm not mistaken, has an implicit blank in effect. So the idea would be maybe somebody else should be sharing here. If Jose's.

No, but wait a second. That would be a one argument version of pattern test, not of condition because it takes a function, not an expression. You can't do a one argument version of condition because you don't have any.

That's fine.

No, no, that's not going to work, as Carlo points out, because the criterion doesn't need to refer to a variable and it can't do that. So what we're talking about really is pattern test.

Yeah.

Yes, sorry, my connection is like 10 seconds too late apparently.

Okay, can we look at the documentation page for patent test, please? Maybe somebody else should share if the yes. Yeah. So it's a two argument function. That's interesting. I mean, we could easily have a thing which is a one argument form that is just blank, question mark, test. Correct.

Something like this.

Where is it I when we need him? I'm sure ITAI is the right person to find a counter example or that would be dangerous.

Right, that's what I'm talking about right there. Right, so this is the way to get indices for something. Now we might want a single function that does that. Okay, and then so let's understand the full extent of row indices and what one plans for it. Because row indices with, you know, with a select spec, which you believe is the most common specification. Is that correct or is there some other specification?

No, I think we believe that the, the criterion function is the most common.

Yeah, Furnaces. That's correct. It's going to be select.

So. Okay, so are there any other specifications you would like other than select? Are there any other ones you think are worthwhile.

Patterns? Certainly. And we were considering ordering like.

So the one, the ones which we haven't, we haven't really fully designed yet are selection based on primary keys. That's the one other thing.

Okay, so that's. You just blew yourself right out of select there. Right, okay, so if you had any idea you were going to use select, there's some kind of thing that's going to have a supplied hackery, you know, thing with. With other keys. Forget it, it's just not going to work. Yeah, okay, what's the sort by idea there?

Returning the permutation. It's like ordering. This is like ordering.

Isn't that ordering, really?

Yeah, we don't, we don't need that because. Because we've now got ordering.

Right. So now there's the question of how do you return the list of indices? Is it a list? Is it a bit vector? What is it? Am I correct?

Yes, that's the part here, the third argument.

Okay, all right.

So really don't think it's a good idea to have this thing return on its own. But what do you mean?

I'm sorry? What do you mean return on its own? What are you talking about?

I think it would be the same problem as entity list. The fact that if this thing is something that has to do a back and forth to, you know, an external thing, you know, you have to do two rounds.

You're saying this should be an inert object, Is that what you're saying?

Not necessarily. Maybe an inert object, maybe just a way to specify part. But I don't ever.

So what do you mean?

Or something that gets.

Are you suggesting that row indices does not return?

Yeah, I think that's useless.

How would you actually get an answer?

But you get an answer when you plug it into part.

What?

Yeah, no. Or not part, but square brackets, right?

No, no.

This is the same mistake as entity list, that it has to do two back and forths and people will end up writing very inefficient code and mapping.

We should figure out how to fix that rather than giving them some complicated conceptual thing.

Yeah, that's the way the evaluator works. Either you hack evaluator and you have.

Special evaluation or you cache things which we should be capable of, which is not. Which we should be capable of doing.

Because Steven, if you're planning on caching things like you're doing something where you get the result out of this part thing, where you only select 10.

Let's not have this argument, okay? It's not a useful argument. I mean, if you're suggesting this function, which already is a crazy function, right, Is. Is now going to be a function that actually doesn't return in a thing.

This function is implemented quite fast. I mean, it's used.

No, no, no. Carlo is making a different point. The point he's making is that he doesn't want to actually get back the indices. He wants to wait until some whole giant computation is built up and then he makes one database call to. To get the whole thing resolved, right? That's what he's doing.

Because basically.

So that's the wrong way to do it, in my opinion. Right? So what we should be doing and what we should be doing with the other database stuff, if that's what you want, you build up a symbolic database query. Rather than messing with all these other functions you build, you make sure the whole thing is built up as a thing where every time you do it, it comes back, it returns. In other words, you start off with a symbolic database representation, okay? Then you do an operation. For example, you do row indices on that thing. What you get back is another database handle. Do you understand what I'm saying? You don't leave row indices unevaluated. You just. If what you're dealing with is an external database, you keep on doing operations on that external database and it keeps on returning external database objects. Do you see what I'm saying? That have been modified.

The result of raw indices is a lazy tabular, right?

So what I'm saying is. Listen to what I'm saying, because it's also what applies to the other database operations that we're doing? We need to go back and change this. Right? So there are two possibilities. Either you have a tabular, it's a local tabular, everything's happy. It's. It's an immediate tabular, so to speak. Okay. And in that case, every function returns. It does its thing, it gives a result, whatever else. Okay, plan B is from the beginning, you say, I am dealing with a remote object that is just an object which is going to need a function that resolves that object. Am I making sense? And then every operation you do, like row indices or whatever else, it just returns, you know, it's called the database handle. Whatever it is you apply row indices, it comes back with another database handle.

Yeah, but so far, the design we looked at like two, three weeks ago, the database handle that was coming back is a tabular object, just lazy. That tells you I have these many columns of these types of.

Then we have no discussion. Then there's no discussion. Then there's also no, no, no, no, no, no, no. Then row indices comes back. If you've given it a lazy tabular object, it brings back a lazy tabular object. If you've given it a tabular object that is resolved, it resolves it. Right. That's all there is to say.

Okay, but. But then, then the, the. It should maybe return a lazy data column, not a lazy tabular object, because it has the.

I don't know. But okay, whatever it's doing, it will always return something that has had the operation done. Now, what do I mean by done? It could be that it is queuing it up to be done in an external database, but it has to not come back looking like a dead, dead duck. Just saying. You told me to do this. I just sat here and did nothing. That's not adequate. Am I making sense? It's got to come back as a new object that says, okay, I understand. You told me to do this. Right. Doesn't have to have actually done it, but it's always got to be. You apply a function, you get back a result. That result might not be a resolved result, but it cannot be something where the function just sits there unevaluated. Am I making sense?

Yeah, yeah, yeah.

Okay. And that's what. We also need to go back and redo our other database stuff so we never have the situation of functions just sitting there unevaluated. That it always has to be. You gave it an object, it processed the object, it gives you back another object. Does that make sense?

Not sure about that second part, because I think I thought the whole point was to have a symbolic representation of things.

But.

Yes, but, but not one that doesn't evaluate. Right. That was a mistake to build up these big things where they look to people as if they did nothing because people give up at that point. Am I making sense? Roger, do you understand what I'm saying? Yeah. Okay. It's something we've got to fix. Okay, let's go back to this.

When it comes back to this, what we're talking about right now, think that they're long term. I think it's super valuable to have an expanded part syntax that does look at values, not just, you know, positions, you know, like the difference between a for loop and a while loop. It's data, it's value dependent.

Yeah, yeah, yeah, right. Actually, that's not true. That was true. I was just talking to somebody about this. I think Dennis Ritchie, creator of c for his PhD thesis, wrote about the difference between primitive recursion and general recursion and said it's like the difference between for loops and while loops. Because at the time a for loop had to have a fixed.

Yeah, yeah, yeah, I know. Yes, yes.

So it's actually not true.

You're right, you're right. It's slightly wrong, but to a first order approximation, that's. That was the big expansion when it happened.

Okay, all right, so. So go ahead. What are we going to say?

Right, so I think that that's something in, in general to expand in the language. And I think that that the part syntax, if we allow that, will be incredibly much more powerful. But it's also then useful to have a thing that can do something explicit, get an explicit list of things, because that's also an explanatory model. But there are also other use cases than just picking things out. You might want to use that as a specification like, like what rows to highlight or what something else you want.

Okay, so what you're saying is. Let's just zoom out for a second. We've got two action items, right? One, we need a function in the language called something like select indices, but I don't like that name. Okay, which gives you the indices which are selected by a predicate, right?

Select position.

We were avoiding position because it's. It's just indices, it's just integers.

I, I think it's some other. It's kind of, it's like almost like indices test, but it's not that either. Indices satisfying. It's not that either, but it's something in that Family. Okay, so there's that function indices, the indices testing function. Now the next question is what? Can we just use the indices testing function on a tabular or does tabular need its own version of that function because it's going to have primary key extravaganzas and things like that?

Well, I think there's two answers that. One is there's a primary key extravagation and there's also choosing row indices versus column indices. Like select doesn't won't choose columns for you because select only goes down one level. So we need to think about how to get to that second level.

Right.

Do we, do we still need to select columns now that we don't have column map anymore? Like how often do we really use.

That except, well, the column indices and the internal code is used all the time to see whether somebody's specification is legitimate columns and which ones there are. So. Yeah, but again, imagine writing functions without having that.

But again, that's. On the other hand, this is how you implemented things on the other hand, in the databases, word column indices don't exist, Columns are always named.

I don't think we should. I mean, look, this is going to hook up to a bunch of data sources, not just relational databases.

No, I understand that, but I'm saying, I'm saying that whether eventually you normalize to numbers or eventually you normalize to strings is pretty much a design choice that you make at the beginning. And so this is a normalization function for that. I'm just not sure that things like asking, for example, for things that apply to the whole column as a function. I don't think those are that useful given that the data is row oriented, because it's very unlikely that you can apply the same function to all the columns that are going to be different types and different things and so on and so forth.

Well, but you might want to apply a function, select columns that are of, say, real type and apply some function to all those elements.

So that's a case where the selection happens on the metadata, not on the data.

That's true. That's correct.

Which is a point I was trying to make like about a month ago that might be useful, but something that explicitly works.

I mean, I won't disagree with you, Carlo. I mean, maybe, you know, we could have just a normalization that would say whether somebody's given legitimate column names or something. And yes, that could be used as.

Well.

Instead of indexes.

But here we have the third argument to specify a form, so we could. I have a third argument that specifies in which form.

Yeah, but I don't mind. Again, I don't mind the function that normalizes, say, from string to integers. What I'm criticizing here is this section with select cases sort by. I don't think you're likely to do that on the. Like. There isn't this symmetry anymore in the functions we have and so on between rows and indices. This is not a labeled array. Rows and columns do very different things. You're not going to sort the columns.

Why not? You may want to find all the columns that contain an A or contain some numbers or something.

Or as we've seen, underscore. Mm for millimeters.

All right.

But is that something you do on the name or that you have to transverse the whole data to look at that?

No, no, the names.

Yeah, the names. I'm okay with having something that. But I would argue that then the important thing is to do something richer on the metadata rather than ever letting people think that you can do this on, like, select all the columns where the sum of all the values is larger than 100, and then you pass it on all the columns and half of them are strings and it's gonna splat. I don't know if that's useful.

But that's why we need.

But maybe it's useful when you have extended keys.

Right, because we're talking about matching from string names too. But that assumes the flat structure on the keys.

No, you haven't extended keys. But extended keys are still metadata. Absolutely. Yes, yes, yes, but.

Yeah, right, so. But we don't have a. We don't have a functional way of. I mean, the, the key name metadata. Yes, that's separate. But. But the type. We don't have a way of separating that metadata from the actual column data. Maybe we need to. I don't know, but that's.

That.

That would take some thought.

Okay, let's. Let's see where we're going here. So the other question is the return value from. I mean, you're saying. Okay, can you tell me the ways in which, you know, index select or whatever it is? Yes.

So this is a list of indices, a list of integers I know.

Most obviously, I'm asking how does it differ? Index select applied to a tabular versus index select applied to, you know, everything else, so to speak. How does it give me other ways in which those things differ?

Oh, index select up there. It uses the applies to the integer data. Go up, go up, scroll up.

No, no, no, I'm not asking that question. I'm asking the Question. I'm asking the question. Assume that we have a function, a stat, a top level function that's a select that returns indices. Oh, okay.

Oh, but there is a thing up here called index select, which is different. I'm sorry, I misunderstood.

Well, I don't know what that is. I don't know what it is because I asked you guys multiple times, I said, if we have a function, select indices or whatever, whatever the thing is called, that is a top level function that simply returns the indices of selections. Yeah, okay. What I'm asking is what special features do we have to have within that to operate well on tabulars? Do you understand my question?

Yeah. So basically, basically for rows, none. We have to have no special features. For columns, we need to have some, some kind of special features. But Carlos arguing we shouldn't have it for columns anyway, so maybe that's not.

Okay, so fine, so then, then, but, but among the special features, what was that discussion about primary keys? First question? And what's the discussion about different return indices? Because different return values. Because we could imagine different return values for our select indices function, it might be useful there to have, you know, these things like a, like a, you know, even, even something like a byte array.

Well, this is a boolean. This is a bit vector, which is a very compact.

Okay, fine, fine, fine. But I mean, but that's a function that, that could be a third argument to select indices or whatever that says what the return form is. And that would be applicable even if it's a list of, you know, a list of integers or something. Right, that it's selecting.

Right, exactly. And this is a party specification, so it may return all or a span instead of a huge list.

I see, I see, I see.

So it might be much more. I think the select indices as a language function would be super useful because I get sick of having to do flatten of position of blah.

Yeah, well, fine.

Which is, which is how position always gets used by me anyway.

Fine. And then it's select indices and match Q if you want to do a position like thing for the indices.

Exactly. I think, I think that's. I mean, maybe it's not the right name, but it's not the right name function.

Yeah, it's not the right name. We need to get a name for that. Okay, fine. So that's a really useful thing. All right, so let's. So we've made that progress here. Now what's the thing you were saying about primary keys? What's that deal?

Well, yeah, if we have primary keys. How do we select rows by primary keys?

What does that mean?

What is, why are they different than other than others?

Well, I think the main difference, I mean, I don't think it matters for column indices. The main difference for primary keys is that if you select by any predicate, you can get 0 to n rows back. When you select the primary key, you know for a fact that you either get missing or you get one. That's the difference. You know, you're kind of guaranteed to get either of you.

And why do you care?

Well, selection by a primary key should give a tabular row object if it or, or nothing. Right.

Or are missing. Yeah. Whereas, I mean, it's the thing we were talking about the other day. There are some situations where you get a tablet roll out or select situations where you get a whole tabular out. It is like you operate differently on it later there, you know, and, and it, it, it has ramifications in terms also of the stuff we were talking about with missings. Like if you operate on a tablet, then you're operating column wise, so the missings get deleted. If you operate row wise, the missings are sticky. So it's a whole different mode of operation. Right. Even if the resulting tabular might be empty or contain only one row, in general, it could be n rows long.

But in principle we could do selection of rows by their primary key. Right. So we could have this, this sort of thing.

What is a primary key? I'm sorry, I don't understand this. What is it?

Okay, the primary key is like a key in an association. In a key in an association. It's like you're guaranteed that you either get one value out or you get missing key not found, whatever it is.

Right, as opposed to what?

It's injective. It means it can't point to several things, can point to at most one.

I see, I see, I see, I see, I see.

It's what we sort of. We haven't figured out the name. We called key columns. But that's, that's not a great name because it's too confusing with column keys, but.

Right, okay. All right. What else did you want to discuss in this meeting?

Some time ago you had requested that we look at the lists of element types. I don't know whether you want to have this discussion now or we will look at it on another day. So this is the list of.

Types.

We have and then examples of each case. And at some point we had to look at this very quickly and we had a discussion on the name of categorical types.

Yes. I think categorical types is not a good name. I think this, this is more like choices or maybe enumeration, but let me see how it's used.

So this is, I think, enumeration, if you're thinking old programming languages, right?

Yeah, yeah, yeah. So in, in the compiler. What? These types are also supported in the compiler in many cases. Is that true?

Yes, yes, many of these cases are.

Okay, so. So let me understand. Which ones are not supported in the compiler.

Perhaps this 3.

The quantity is not the date. The date and time types are not at this stage. I mean, I think we could, we could do that.

Yeah, I get it. Okay, so what about the, the categorical stuff?

The category ones are not.

Okay, so then let's, let's come to an agreement on what we're going to do with those. I think enumeration. I don't know whether enumeration is the right term.

In the interpreter types. We call it choices in the documentation.

Okay, just. I think choices is not a terrible name.

I think it's pretty terrible. I mean, like if, if. If the categories are male, female, or the categories are small, medium, large choices.

Seems pretty reasonable to me. But we have random choice. The function random choice.

And, and, and Steven, another thing is that I don't see here, but we might want. Is.

You can. Might as well then call them factors.

Any subset.

That's what they're called in statistics.

Saying that, Roger. I mean, choices is a. Is a common English name for things. Neither enumeration nor categories is a common English name. Categorical is certainly not a common English word. Yeah, but I mean, there are a limited set of choices for this object, and that's why we call it random choice for that function.

Yeah, yeah, yeah, I see. I. I think that's not. I mean, what would you call the ordered case then?

I mean, ordered choices.

What about any subset? Are we going to have some kind of. Like each row is one or more of the choices or zero or more of the choices, rather than.

Well, we have any subsets in an interpreter type, don't we?

Yeah, yeah, but it's.

Okay, so listen, listen. This table needs to be redone with several columns, okay? We need the compiler's version of this. We need the interpreter type version of this. And perish the thought, if there is a similar thing for data set, which I don't think is exposed to the user, right?

No, it's not.

No.

Okay. The other types are in read list, for example. Some of these types exist in read list, correct?

Yes, I think number.

Yes, they do.

Yeah, I think say that.

Yeah.

And so we need that. We need a correspondence with those as well. And I should run along here, but. To be continued. Okay. I think we made some good progress here.

Okay.

All right. See you guys soon.

See you.

Bye.

Bye for now.

Okay.