Live CEOing Ep 830: Language Design in Wolfram Language [Tabular]

Stephen Wolfram leads a design review of tabular data handling in Wolfram Language, covering how to represent tables, connect to data sources, clean messy data, transform columns, and handle joins and pivots. The team flags remote/lazy tables, outlier transformation, and type-conversion helpers as priorities for follow-up work.
Chapters
Okay. Hello, everyone. Welcome to another Wolfram language design review. We are talking about tabular again. And we're trying to look at the top level guide page describing tabular processing. So can you scroll it down a bit just so we can see? Actually, you might as well make it as tall as your display. Is. Tabular communication. It's an interesting concept. Okay, so which things here have we already reviewed?
So we've done a little bit. We primarily worked on tabular, object, little bit on data sources, cleaning a little bit, and transformations where we spent the bulk of our time. Object and transformations where we spent the bulk of our time. We've had. We've dipped into visualization a little bit. We haven't really talked about modeling. We haven't really talked about communication.
All right, so is modeling.
Did we.
I mean, modeling. We've got this whole issue about variable names and things, right?
Yes.
Are we. Are we tackling that today or are we not tackling that today?
No, it has to be when Julia is back. And. I mean, I think. I think today what we can do is as we drip into these different pieces, we'll see. I think we can identify gaps or things that we need to return to. And I think that would be very valuable to drill into any particular topic. Usually needs prep and some time both. Okay, so if we pick object that. Okay, so the object basically is sort of representation options, also sub representations.
You might want to make it larger. But okay, so.
And very basic operations like getting stuff. Getting.
So, so how do we represent remote data here?
It's not covered here. It's not covered here because we don't have a working design for that. So we can have remote data. If you go to data sources, we can have remote data in the following cess.
You had transparent remote data. I thought that's our goal is to have transparent remote data wrapped in a tabular, but it's in fact running on a remote SQL.
No, I mean, that's one thing, but that's not. I don't think that that's an mvp.
What does transparent mean?
It means that you do operations like, you know, transform missing, just as you would not knowing whether the data that you have is in a remote SQL database or is local in memory.
I eagerly. Like, you have to wait. Every time you hit shift enter, you have to wait like 10 seconds for the data to be processed and transferred.
No, our primary goal is in memory processing. Secondary goal is to have out of core or out of memory processing. And we don't have that working yet. So it's not here. We don't know exactly.
All right, okay. But I thought that's okay, but that's the primary thing Carlo is working on, is that correct?
Yeah, but it's not transparent. We never said transparent. We said lazy. And then there's a function that brings it into memory. Like, it doesn't feel exactly the same as, like, you always want to perform all the operations together at the same time.
Okay, but we've been through this many times. And when do we get to see that design?
Well, we looked at it already, like a month ago. Like, I am making improvements to it. Like, some of it is broken right now. So you can't look at it at the moment on, on today's prototype. But, like, today I worked on this with Rob for two hours and a half. It's, it's like, it's, it's okay.
So do you feel you have a design that I've approved for this?
Yeah. We'll have to look at it again, I think.
Okay. But I mean, it's not something where I'm going to say it's all the wrong way around. Correct.
I don't know, Stephen.
I cannot predict that, Stephen, given the number of times you contradict yourself, that is an impossible guarantee to me. Lovely.
But I, I, I'll just put in my two cents. I, I don't, you know, I think, I think Carlo and I have a good way of working together to make this happen and make it look just like a normal tabular object. But.
Okay, fine, that's what I'm looking for. And that's what I'm looking.
Two things. Just to manage your expectations, we will need to go back on the design because there's a couple of things that we haven't fully gelled, like whether turning an external tabular into tabular that sits in memory is 2 tabular from tabular or yet another function. Second thing, you will have to review the boxes because it will not look like a table. It will look like a summary box before evaluation.
That's fine. I'm happy with that.
Review those two things.
I'm very happy with that. You're making a symbolic thing. It is a handle. There are functions that actually make it, make it, you know, bring it into memory. Yeah, yeah.
So there are a couple of things to review, but you've looked at it and you mostly.
We need a name for this operation of going from the remote tabular to the, you know, suck it all into memory. Yeah, it's kind of like tabular cache, like local cache. But it's not.
Didn't we spend a bunch of time naming that, like a few weeks ago?
What did we come up with?
I don't remember.
I also don't. I think we didn't.
There wasn't one thing that we said, oh, my God, this is perfect. But there were a number of things like tabular fetch.
Right. All right.
Because. Because caching is not the right thing.
No, it's not the right thing. Yeah, it could be import tabular. I think that's what we discussed last time.
Yeah, it could be.
I think import tabular is a decent idea. I think that will allow us to have kind of various kinds of options, as we do with two tabular and so on. Okay. I. I think it's a little problematic, but we can. I don't think we have to dig into this today.
Yeah, I don't think we need to dig in. But I already have, like, import of data connection object, and that returns a tabular if it points to something that, you know, represents.
Yeah. So that's just a repackaging of that capability then. But you've already got a tabular summary box as opposed to a data connection object. What we're saying is the data connection object has been moved into a tabular summary box, and now we have to take that tabular summary box and hydrate it basically locally. Does that make sense?
No.
Let's go through that again.
Data connection object and tabular will be in some sense interchangeable in some situations.
I understand that, but I think the data connection object. Okay, so data connection object, you kind of expect you could say tabular of the data connection object, and you will get what I would assume will be one of these summary box, like, you know, remote tabulars.
Yes. Well, if it points to a file.
That you can bring into memory, it's just going to bring it into memory.
I understand, I understand. But most of the time it won't do that. Most of the time it's going to be too big to bring into memory and it's going to be simply, you know, pointing to that. So you have an operation, Sean, that, that I. That it's going to just bring it into memory. Right. Which is an import operation. And that's great. And that's exactly what we need. The only question is how to package that when the thing that is staring, you know, that's in front of your nose is the summary box thing.
I don't know.
You keep talking about the summary box. I mean, it seems like the. The real question is just we have two. Given that you have, say an SQLite, SQLite, you know, file file on your computer, we have two ways of accessing it with tabular is my understanding. One is you just, you know, import it all into memory and it's in memory and you've got a tabular, you know, whatever. And the other is that you have a tabular that's backed by SQLite in some way. And so if you have a data connection object that refers to this thing, then when you import it, you can get either one of those two things. Well, right now, data connection object is, is just a way to specify the location to something. And right now everything is designed for incore. As far as out of core goes there, there wasn't really any design for that yet.
All right, fine. All right, let's move on. Roger, you're in charge. What are we looking at next?
Okay, so next after that is data cleaning. And that's to essentially get both the right structure in the right semantics so that you can do further processing, ask questions, get answers.
By the way, for goodness sake, we don't change the format of, I mean, this basic operations on columns. You know, we don't have lines like that in guide pages. Right. The sub line underneath basic data cleaning.
Okay, we can remove that.
We have.
Yes, yes, it is.
Really? Where?
Well, we introduced it, you know, just a few years ago.
I've not seen it. I've never done that. Where do we have that?
Well, you can do a search in the guide page system and we'll find where. Where we have it. I don't know exactly.
I'm not a big fan. I'm not a big fan. I think it's an excuse for not saying what you mean more clearly.
I mean, well, we can remove it. That's not a problem. I mean, it'll be a little bit more mysterious, but we can remove it.
Well, that's why it shouldn't be basic data cleaning. If it's basic operations on columns, why doesn't it say that you've got a page about data cleaning? If that's what that section is, then call it that. See what I'm saying?
Yes.
And I don't know what restructuring data cleaning is. I mean, are you restructuring the operation of data cleaning?
No, we restructure the data structure.
Okay, fine.
So what's values in some cases will be names of columns, in another case that's what pivoting does and vice versa.
Well, so then, you know, operations on columns, you know, restructuring Data restructuring will be the next one. These, these subheadings are not helping you. They're just, they're just being an excuse to have a lousy overall heading. You see what I'm saying? I mean, and again, you know that, that value cleaning makes further analysis possible. More robust that. You don't need to say that there. That's what we have our blurb at the top to say. Okay.
In any case, like here, column types is. All the others are functional. Column types is an option.
Yeah. I don't think we have the quite the right solution to change types. I think it's complicated to do right now, but I think it's an important data cleaning operation.
I also think that like somebody who comes from the Wolfram language and tries to understand what pivot does without ever finding the word transpose there will be utterly confused.
Oh yeah. I mean, we haven't talked about these. We have not reviewed these pivot things yet.
We've looked at them, but we should go back. Absolutely.
I've never heard of pivot longer.
Yeah, we're closer to being able to review that. Stephen, Maybe next week.
Okay, fine. Okay.
Yeah, so I think. So here's an example of something that I do think is important. Like if something comes in as strings and it states or something comes in as strings and should be categories. Those things, I agree. Big impact.
So those are like semantic import. But your point is? They're already in there. How do you react? You know, how do you change them? Right. And I'm not sure why that isn't, you know, transform columns. Right.
Well, I think you're onto something in that, for example, transforming string to dates is a very explicit operation where you might want to say what the date format is. So yes, that's something that in other situation we would do with say, interpreter or date object and whatnot. Whereas transforming strings into categories doesn't change the data at all.
Here it is.
It doesn't. It changes the representation of the data only.
Yeah, I'm. I'm totally confused here. Transform columns is not about separating and combining columns. It's about applying functions.
Oh, no. In this, on the data cleaning page. That's what it is about. That's the only. Because we had separate functions for this, but now they're based in transform columns. So that's it.
Yeah, okay, I understand. But. But I think it needs to clarify because this operation that we're just talking about now could trivially be a transform columns operation.
Then we'll list transform columns here and what the new task that it does.
I think so, yeah. Because I mean, transform columns, I would have thought. I mean, what you'd want to do is list whatever function you actually use to, you know, transform all those strings into dates. What is that function? Is it import string? Is it date object? What is it? See what I'm saying? I mean, in other words, what I would imagine it would look like. Form column. Okay. All right, so transform columns of, you know, whatever it is, you know, my date arrow, you know, from date string of hash my date. Is that right? I mean, is that correct thing?
Yeah.
Okay. And are we happy with that way of, of changing data types, basically? Or is it more. Is it better to have something which is in bulk able to do it for a whole column? Maybe that's able to do it for a whole column. I don't know.
Well, I think that's, that's largely an implementation detail you shouldn't be concerned about.
Okay, but are we happy with that approach or do we want to have something instead where we.
It doesn't always work. Like, there are some situations where, like for example, you're turning strings into integers you can use from digits or whatever, that. That is fine. But if you want to transform integer 32 to integer 64, maybe we should use cast. But you really didn't like that the other time we talked about this?
I don't think so. I think what we should do, perhaps we should have a thing that is some kind of setting in transform columns that allows you to just say, I mean, do we have already a syntax? Why isn't it, you know, right. You know, rule goes to typed of, you know, value comma, you know, new type. Possibly it should be. But an alternative would be something even simpler than that. Transform columns, you know, quotes my date arrow and then an interpreter type as a string. Or is there another meaning for a string to string transformation? There in transform columns, what does column name, arrow, string mean?
So far, nothing. But it's kind of a like interpreter types are not at all well suited for this because they always expect strings on one end and on the other end on the output end. They don't, they don't necessarily return integer 64 or 32, you can always produce.
I understand, but it's something like interpreter types it. Or is it like import types? You tell me, what's it like? What, what existing system do we have?
It's okay, so first of all, you need two types, right? You need an ingo and. And sometimes there's more, because sometimes, like going from to date is not as simple as that because you Need a date format sometimes.
Right.
Or from string to real, you might have commas for.
Listen, if it's that complicated, it's just going to be a function, it's just going to be, you apply whatever transformation.
And I think that's fine. But there are some situations because tabular is more strongly typed than the normal evaluator, where we don't have a function and the only equivalent we have is in the compiler and that's cast.
We could, I mean, sort of bizarre way to use these types is, is the, the, the, the, the numeric tabular types are very much associated with numeric array. So we could use numeric array of blah comma type for those types. That doesn't, it's not a very, it's kind of a hack around avoiding using cast.
Not a fan. I mean, is there, look, I think for the operation of going from an integer 32 to an integer 64, I think that's a low level operation and cost doesn't sound too scary for that.
Yep, yep, I agree. I, yeah, I think that's typically what they're going to.
Right, but let's not try and use cost to go from a string, you know, to a data object.
No, I, I agree with you.
I, I, I, I kind of agree with that. Yeah, right.
But what you're telling me is it's all so complicated, it just should be a function. You see what I'm saying?
I don't think it's complicated. I think it just feels much more natural to use N to go from integers to reals than anything else.
Whatever, it doesn't matter. We're not disagreeing about the output outcome, so there's no need to have some specialized, you know, you know, column retran, you know, column casting outer function. Right. It's just transform columns and there's no particular shortcut syntax. It's just transform columns, hash, you know, column name, blah arrow n of hash column name. Is that correct?
Right, correct.
Okay. Because another point is that we think it will be as common to want to combine three neighboring columns and so on as to do an in place transformation of one particular particular column. Is that correct?
Yeah.
Why do you think that?
Well, because I think dates might be sprayed across three, three columns, things like this.
Well, that's combining columns.
Yeah, I understand that, I understand, I know, I got it. But the whole point is there's no reason we don't have to talk about anything specific to, you know, this type conversion section here, which probably should just talk about transform columns again. Do you see what I'm saying?
Yeah.
And I don't agree. If that first section is Basic Operations on Columns, I think you should say transform, separate or combine columns as the description of transform columns. Right. Because I think it's, it's. And I think you should rename that section Operation Basic Operations on Columns. Get rid of its. It's very interesting how putting in a subtitle makes the main title worse. It really is very interesting.
There is one thing that is also interesting to me is that, for example, in the type stuff and the, the value data cleaning, obviously you would use transform columns, but then you would use, I don't know, transform columns and n. So it's like it's somehow the guide type of.
Yeah, the guide page should list some of those functions, some of the common functions.
It's like you always want to use those two in combination.
Like a suggestion about the guide pages. Okay. Once you get to. I don't know what type inference is saying here, but, but you know, that should be a sub guide page which lists functions which are mostly not tabular functions, but are just the functions you're likely to want to use if you're doing those kinds of type conversions. Does that make sense?
Yeah.
Okay.
Then there's transform missing, which we haven't quite finished. And this one, outliers, which we haven't really started on, which we might not ship.
Right. I mean, I transform outliers. I mean, wait a minute. If, If I have -99 means missing.
Well, then that's part of missing.
Okay, so for outliers, that's mostly a bird sat on the anemometer. So it said the wind was zero. Is that right? Or is that supposed to turn into missing? Should turn into missing because we don't know what the wind was because the bird was sitting on the anemometer. Right. So explain to me the use case for outliers. For the transform outliers.
So the use case for outliers is that when you have extreme value data, which it often is something that probably went wrong or, you know, in the measurement process or something like that, and what it does is, has an inordinate impact on some types of summaries. Like mean, if you have, you know, all your values is between what do.
You want to transform it into? Do you want to transform it into a missing or do you want to. Some value?
Yeah. So missing is one way to do it, and one is to project it into sort of some number of sigmas out.
Those are examples. I have a question.
Okay, so we have something called windsorized mean which does that.
Right, I remember that. Yeah. Okay, but so hold on.
And trimmed mean is making it missing. I mean it's removing it.
Right, right. Okay, so the basic point here is for transform missing, basically the data point is going to get thrown away. Transform outliers. The data point's going to. It's going to get thrown away and then the operation that's operating on it is going to just say that was a missing data point, but it knows nothing about the data point other than that was missing.
Right, but transform missing also has a bunch of imputation stuff you can fill in.
No, I understand that, but.
But the transform missing is if I say, I mean, I think like transform missing I think is different from what you're saying. Because. Because if I say mean of a tabular column and the tabular column has some missing stuff in it. Right. I, I think that's if I want to get the mean of that column and ignore all the missing stuff. I don't think transform missing does that. Right. Transform missing is if I want to. To get rid of the missings in some way.
Right, well, to replace the missing with something that's not missing.
Well, right.
Make it so there's no missing.
Yes.
Right.
But I feel like transform outliers really wants to be two function because one of them transforms the outliers into missings and the other does an imputation in a way that would be impossible if it went through missings and forgot what it was. Yes, one is like outlier to missing and the other is outlier to normal values.
Yes, I agree.
Actually one other question is first of all, is there a difference between an outlier and an anomaly?
No. Well, I mean maybe it should be anomaly because it outlier for categorical stuff is probably, you know, it's not.
Doesn't make sense.
Okay, fine, but then, then. So okay, so if it's anomaly, then we. So we have a couple of anomaly functions, right, for like replacing anomalies and stuff like that. Are any of those relevant?
We have anomaly detect. What else do we have? And we have synthesized missing data. What else do we have?
I thought there was one other.
But let's look.
I mean we have a bunch of built in robust descriptive statistics that part of their operation is to do an outlier transformation. That's why they're called robust.
Yeah, it looks like there's anonymously detect, there's delete anomalies, find anomalies.
Yeah, I see that there down at the bottom under future data cleaning on this very page.
Right.
So I just wonder if those. If those should just. I mean, do we need new functions or is it just that those functions take a tabular or something, or tabular.
Columns or something like that? I think Julio has said he doesn't really like the design of those functions as they are, and so he wanted to make new functions without him. I can't really speak for him.
The other problem is that in general, the problem is that you always kind of want to go through transform columns. You don't want to like. All of those functions on anomalies are things that take one column and return one column.
So they might.
One of the things that these functions can do is to basically give you a menu of obvious things to do so you don't have to program it up yourself every time. It's a. It's a touring, complete system. We can program anything. But how easy to use should it be?
I'm sorry, so what, what's your point there?
I mean, my point is that. That we can program anything.
Sure.
You know.
Right. So what are the common cases?
Yeah, so that's what should be in. In these things. Now, I don't agree.
Named common cases.
Exactly. Now, this one is. Not that this needs a revisit, but, but.
Okay, can we go on? I. I'm still trying to. I still don't really get the flow of this meeting. So, I mean, what we're identifying here is we need to have a discussion about transform outliers and we need to have a discussion about pivot functions.
Yeah. So this is to identify. Otherwise we just keep talking about the same things. You know, we forget.
So I think we're. We've identified that. We've identified some problems with the format of the guide page. I've suggested some sub guide pages. Do you agree with that concept?
Well, I'm going to try to make do without it because it already has seven sub guide pages.
No, no, no, no, no, no, no, no. I'm making a specific suggestion. Go to that page that you were on. I was making a specific suggestion. When you go to something like type conversion. Right. I strongly think they're going to be 30 different functions that are relevant to that. And to show people here are the functions that might be relevant is super useful for dates, for entities, for whatever else. You're not going to include them on this page. They would gum up this page.
Well, I think we start on this page and if it turns out to be that many. Yes. Then we build a sub guide page.
Okay. I think it will need a sub guide page, but. Okay, go, go on. I think it's convenient. It's nice to have a sub guide page. It helps people.
Yes, but I mean we already have seven.
Doesn't matter.
Okay.
I mean guide pages are cheap.
Okay, this is the transformation. So these are, these are transfer. So the goal of this page is they're transformations where you ask questions and try to get answers. So if the other one, like how to get data, how to clean data and the basic object. This is I think the highest level operations of transformations.
I mean like what the hell does union do on tabular objects?
It does join and then delete duplicates pretty much. And sort.
Delete duplicates of what kind? Delete duplicates of rows or what? Rows, Rows, yeah. Okay. So presumably there is a, there is a join. Is that correct?
Oh yeah, yes.
Where is it on this, I mean, and combining tabular objects. I would have expected Join.
Yeah, column wise, combine, include joins.
I think it only includes join across is not, not vertical joint.
Right.
But join is a no brainer and it already works. It doesn't need further documentation or anything like what Rob has already implemented it.
Well, I understand, but on this page where you're talking about union and complement and so on, it seems like you should be mentioning. Oh, it's right there. Sor story.
It should not be in disorder, I think.
But yeah, no, I, I agree. I just didn't know what worked and what not this morning, so. Or last week.
Okay. I think we need to actually be trying to. Wait a minute. Number categorize. I don't know what any of those are. Those are new functions I've never seen before.
Well, you have seen them because we've talked about them, but we haven't discussed them at length. So those are for many cases when a workflow is to segment your data effectively, there could be sort of number ranges, like if it's age, maybe you have young, middle and old or something like that. And that's often a way to set up your data, augment it in some way that you can see whether those.
I want to come back to a comment Carlo made when we were talking about data types because he was saying something about strings and putting them in categorical buckets. Right?
Yes.
So I have a question. I mean, what would not be good is to have some invisible categorical buckets. How do you tell? Okay, so let's say young, middle, old, whatever you're. What, what do you actually do? So you've got, you know, a bunch of 4445 years old, whatever they are. Right. And then. Sorry, and then, and then your, your. Is it that you would make another column in your tabular or you would transform that column?
Yes, another column.
Yeah, that's the typical case.
You have another column making a new column or transforming the old one.
Just.
It's just about whether the new one is the same name as the old.
Sure, sure, I understand you're changing the.
Data from the actual numbers to categories.
Right. But. But my question is this.
So that is also a way to create subsets of data. Okay. When you add, when you categorize something, that's a way to create subsets.
Well, I know, but then we've got all these grouping things that group based on the value of some column. And I guess what I'm asking is if you've got a big list of strings, you know, small, medium, large, let's say, and something knows that that column is only small, medium, large. It's something internal to the tabular data structure that knows, by the way, this column is only going to have three possible strings in it. Correct?
Yes. And sometimes it can also know that they're ordered. Like if they come from numbers and you do intervals, they're going to be ordered.
Yeah, I get it.
Yeah.
So, okay, so, but my question is, when you do normal of that tabular, any of that information is gone, correct?
Right.
Yes. So we don't have yet a sort of a way for these categorical stuff to live in the system, like dates can. I think we should eventually.
Right, so one of the points there. So what this would look like. I mean, we do have categorical data in various kinds of visualization, do we not?
Yeah.
And we have there. How do you specify small, medium, large being ordered for visualization purposes?
Categorical scale, Ordinal scale. Ordinal scale. Okay. We have two nominal and ordinal scale. So ordinal is ordered.
Okay.
Give the list and that order which you give them is to order.
And that's the, that's the order they will be rendered in the visualization.
Yes.
But so I mean, imagine that we had a wrapper that was called, you know, I don't know, categorical object.
Yes.
And that it knew about small, medium, large being an ordinal scale thingy.
Yes. I think it would be kind of like a date object. Right. So the date object contains a bunch of extra information about which calendar you use. And yeah, you know, date objects do.
Satisfy greater than signs and so on. I mean, they. You can do greater than you can.
Do quality Testing and now we can even do descriptive statistics on them.
Right.
Right.
So.
So if we had that object, they would sort. Right. They would do other things. You know, we'll gradually sort of build out what they do.
I understand, but I mean in the case of a, from tabular or a normal of a tabular, let's assume that this column has been made. I mean in the little iconography at the top of the column, we presumably will indicate whether it is a grouped categorical or a whatever they're calling bucketed, you know, collection of things or just preform strings.
Yeah, there's the string icon and the category icon are different.
Okay, but it's a little bit disappointing then. Okay, so let's imagine that I want to create a tabular just in pure Wolfram language code. And how do I go? So let's say I've got a small, medium, large column. Okay. And I'm going to do from to tabular. How do I tell it that I want that column to be small, medium, large ordinal categorical objects.
So right now I think the robust implement, I mean that you can say ordinal scale and nominal scale.
Yeah, no. Okay, so what does that look like?
You do it with this, you use column types, goes to categorical ordered of small, medium, large.
Oh, we changed the names, but initially it was the same name we used.
In visualization, but we changed the names and then with that. It'll verify those categories.
And basically that's an approach where you're basically using a template for the whole column and saying categorical order, blah, blah, blah. An alternative would be that every single element has a symbolic wrapper saying what it is.
Right.
And so we need to understand the trade offs of those things because clearly if it's every single element, we can, you know, use them elsewhere in the system and expect to round trip them back to tabular.
Right.
On the other hand, if it's purely an annotation in the tabular, once it comes out with normal, all that information is gone.
Right. But I think that this is a little bit more of a build out to do that system wide kind of support.
So I understand, but we should know where we're going so that we're, but.
I think we should go there. So I, I mean, I think that is the right direction to, to make them proper. Kind of like a date object is proper.
Okay, so then we'll, we'll have a change then between this version where we will end up with a, A, an, an, you know, a metadata, you know, a Categorical in metadata only, so to speak, versus a, A kind of locally categorical thing.
Right, right. I think it's. Yes.
I think we should know what the rough design of the local categorical thing is, even though we're not going to do it immediately. Okay. Because presumably the main difference would be in terms of the annotation. Well, if you extract that element, if you do an operation on that element using transform columns or something, you're going to need to know whether it's. Well, maybe you don't, maybe you don't. Maybe it's reasonably transparent.
Maybe most operations you do need to know, Stephen, this is something I spent not huge amounts of time, but not small amount of time is figuring out which. Which operates in transform columns can just operate on the categories and which actually have to work with the data itself.
I understand, I understand that. I understand. You need to know implementationally, I'm saying from a user's point of view.
Right.
If I want to just say, you know, if X is greater than 7, do this or something.
Yep.
And I mean, X is greater than seven is never going to work. The small, medium, large, usually, but in the, in the future, if X is greater than medium will work. But. Okay, here's the point. Anything that doesn't work now, that starts working when we change. This is all good. Yeah, yeah.
I think that there's a variety of things that this, that injection into system can do. One of the things we could do is to curate a bunch of categorical scales, like bond scales and other things.
Yeah, Yes, I think that would be nice.
I think we also need. We also need this categorical local to be able to do sort correctly, because order has. Order would know about that.
But this is another thing we need to have for another meeting. I unfortunately have to leave, but you guys can continue. But I'm sorry, I've got to go. I'll see you guys later.
Bye.
Bye.
More from Guy

Cheryl Strauss Einhorn | The Human Edge: Smarter Decisions in the Age of AI | Talks at Google

Rachel Glennerster | Innovation and the Center for Global Development | Talks at Google

Rebecca Winthrop | Rethinking the Purpose of Education in the Age of AI | Talks at Google

Dr. Robert Wachter | A Giant Leap: How AI Is Transforming Healthcare... | Talks at Google






