Live CEOing Ep 768: Language Design in the Wolfram Language [InterpolatingFunction Methods and More]

Okay.

Hi, everyone. Welcome to another Wolfram Language Design review, part of our Language Completeness and Consistency series. And looks like we have a few assorted topics for today. I'm surprised we don't have more. But anyway, let's get started here. We may not be able to do this one because our key person is without power somewhere in Montana. Is it worth looking at this or is it.

Yeah, I think we can get started on this. Right. So, you know, we may get lost in some of the details, but. Right. So interpolating functions have the. These undocumented method things which, you know, the question is how to expose them.

Or that is why doesn't an information play.

What's that?

Well, use the information function.

Yes. Which is. Well, yes and no, because some of these are, you know, they take. Right. They take parameters. And information isn't really set up for.

That, is it not?

And, and so some of these are actual. Some of these are actual operations.

So, like these string arguments to interpolating function were introduced as a, As a kind of rogue project years ago. I never really believed in them. I always thought they would bite us. And guess what? They're biting us. So, you know, what we've done in other functions is to break this stuff out into the information thing.

And if that's. That's not. That's not the. And that's not the entirety of the story. I mean, I know you like to opine, but maybe if you actually pay attention, you can learn something.

Please, go ahead.

Right. So for example, if you look at the, you know, at the vector evaluate thing. Right. These are. Right. And, and we used the sort of funny string applied to argument thing because interpolating function is a. Is has listable evaluation. Right. So on some level, what we've done with data structure is sort of, you know, serves as a good model.

I don't like it very much. I mean, it was the best of the. It was the best of a bad collection, what we did with data structure.

Okay. And.

So what. What's the. The point is we have operations on interpolating functions.

Yep.

And we sort of rogue them into these, These string arguments.

Yep.

Now maybe there are actual proper official functions that operate on interpolating functions just like things operate on sparse arrays or lots of other kinds of things. I don't really know. And there's some things that are more like properties, which are more like information plays. And there's some things that maybe should be actual functions operating on interpolating functions. Sandra on our livestream comments asks about Extrapolation order, extrapolation handler. It's common that you just want the value of the first and last point rather than a crazy continuation of third order. Not quite sure what that means because I think, I mean, if it's the first and last point that are actually in the data, why aren't you picking them out of the data?

He may be referring to extrapolation order 0 so that you can evaluate the function freely at points outside the domain and you still get those points.

You mean interpolating function.

Yes. So in the same way that we have an interpolation order, he may be requesting an extrapolation order.

Right. So if you, you know, evaluated that function at, you know, 55 that it would get, I mean, I'm not sure.

We just give an error how you.

Do that, how you would do that in two dimensions. Like in, if you had a one dimensional interpolating function between 1 and 4 and you evaluated at 5, that would still give you the same output as you would have gotten at 4.

Well, we can still can still extrapolate into dimensions.

Right.

And I, I don't remember which order we are using.

Okay, so if I do this and I say that of 11. Yeah. It just says it's outside the range, but then it says it gives an.

Error message and then doesn't, but it gives a value. I think he's, he's requesting that we have a way to control that value. In this case, he might want 10. If we had extrapolation order goes to 0, it would return 10.

I understand, but the other question is, should we have a function called extrapolation that can take an interpolating function, for example, and extrapolate it without giving error messages and so on? I mean, I think it's right to have an error message there. I think otherwise one just gets stupid wrong answers.

Yes. Perhaps if we had an extrapolation order that would indicate that you are interested in extrapolation and then we do what whatever that order says and not emit a message.

But we could also have an extrapolation that can be applied in addition to interpolation. We could have this which immediately, you know, does, you know, create something that is extrapolatable. I mean, I do agree that it's a somewhat common operation.

Right.

Although, you know, any, you know, my, my experience has been if you have, for any sort of real data, I mean. Right. That that thing tends to diverge very quickly.

Yeah, yeah.

I think time series has some control on this.

Let's look at these things here.

Yep.

Which Ones. So there are a bunch of these that would be reasonable information type things. And there are some that are not. Which ones are not. Which ones are more like computation.

I think. Well, the ones that are called, that have evaluate in the name, obviously.

And what is evaluate is evaluate not just the default operation of interpolating. What interpolating?

That is basically just default operation.

Okay, what's that one?

That one I am not sure about.

Okay. But many of these are really information style things.

Yes. And you know, which is, had we gone on to, you know, the rest of the agenda, you would have seen that, you know, what should information give?

Okay, fine, fine, fine. Yeah, I mean, I don't understand why. Why not just give everything and information and have it, have it work like information does for other functions where you can call out particular pieces of information if you want them.

Yeah.

Right. Okay, I do agree that kind of, you know, just like the duration operation for things like time series. Right. That's a useful function. What, what is the analog of duration for giving the interval of a time series? Or do we not have that with this? Look, my feeling is what we should do, and probably we need to come back with with Rob here is, you know, figure out which things are information things and figure out two or three probably new functions that are applied to interpolating functions that do the things that are actual computational operations make sense. That's my suggestion for where to go next. Disagreement. Okay, all right, let's move on then.

Okay, so this is something very simple. It's the idea in the same way that you have select and cases, and then we have any true, all true, non true, which are aligned to select. These would be aligned to cases.

How are all true, any true and untrue aligned to select?

Because they take a function, a criterion function like selectas. So these would be.

Taken, I would say that they're more analogous to and an or, right? I mean, they.

Well, or union. Okay, what I mean.

Those are predicates. So I will start over because I disagree with that characterization. Right. We have any true, all true and non true. And you know, which are sort of predicates that tell us, you know, apply some sort of test function and you know, they're guaranteed to short circuit, etc. Etc.

Okay, so I mean, if we say something like this, Oops, wait a minute. Any true?

No, I think you want the list of table of 10. So range 10 comma prime Q.

Yes.

So now you could use all true and then it would give false because not all numbers are. So the Idea is to have the equivalent triple. But the second argument is a pattern rather than a criterion function.

Right, right. So you know, any match is basically, you know, just member queue.

Yeah.

And so all match and none match are sort of, you know, variation, variations thereof, which I've written up there.

But so I mean, for example, this.

Right, that would be another way to write.

Yes. So in a sense this comes from the fact that according to Rob, doing select is faster in manipulating large arrays and therefore if we have a predicate that can analyze quickly rows, it would be good. And so this is a way with the operator form of saving a few columns or instead of any true of match Q something, it will be any match of something. It's just that it's syntactic sugar for, for what you have in the screen.

I understand. How common is this?

Well, for arrays in which all columns have the same type of object, like all numbers, imagine that you want to select all rows in which everything is a rational number. Say I'm just inventing right now.

Yeah, I, I, I'm not, I mean, I'm not convinced this is that common an operation. You know, yesterday we, we had a little thing about keep elements.

Yep.

Right. And none of us could really remember, I couldn't remember what keeper. I couldn't even remember that we had a function called keep elements.

Delete elements.

Yeah, Keep elements is the one we wanted to add. It Delete elements.

Okay, fine. Delete elements.

Yeah, exactly.

I just think these things that are not. Yeah. I mean I find myself using any true and all true and so on. I use those with some frequency.

Yes.

Would one use these?

So, yes. And in fact recently Rob has in improved their efficiency quite a lot in that triple. So this was the idea of doing the same thing but for patterns rather than criterion functions. And again, we can survive clearly with any match Q of something. But this saves. It's syntactic.

Sure. I'm just trying to think, maybe I'm just not thinking straight, but I'm not thinking of, you know, I'm trying to think of code where I would have used this.

So it would be like.

Well, I think any matches the wrong one because that's just member queue and so, and surely you use member queue.

Yeah, right.

I think I could be wrong, but I think that member queue is going to unpack and any match would not.

And it's not obvious that member queue takes patterns, so I bet that most users didn't know that Memorq takes patterns. It feels like a very literal oriented function.

Yeah. But okay, but that's. We can't have a new function just because people don't know what the name means.

That's fine. I agree. It's more for the other two and. Yeah, and then to complete the triple.

I mean, for all. For something like all match. I'm certain I've written code that was essentially. That would have come down to all match of something, comma, underscore list, to make sure that I have like a list of lists of.

All right, I'm agreeing. I agree. That's useful. And I'm thinking of code I've written. I've written plenty of code which has apply and to a bunch of things.

I think this is something that we would write with cases. Blank, blank, blank, the pattern blank, blank, blank, this sort of thing. But this is much more readable.

All right. Okay, let's do it.

And.

And you. You think it's superior to basically duplicate member queue as opposed to coming up with names for all match and none match?

Yes, I think. I think that duplicating member queue is. I mean, I don't know whether the operator form is.

Now the operator form. The difference would be that. I mean, the difference would be that you can only use, you know, a single level as opposed to multiple levels. Although, you know, I think the number of times that I've used member queue with more than one level is, you know, can probably be counted on one hand.

I've used it in arrays and things.

Well, you've used it on arrays, but have you used it at multiple levels of the array?

I've used it at a particular level of the array.

Exactly. Which you can also do with these guys. I see you can't do it at multiple levels, but as I was saying, I think the number of times where I've done it at multiple levels can probably.

At the same time. This is usually. I see at level at that specified level. I think it's okay. I think any match is okay. I think it's a little bit weird that it's essentially a synonym for member queue.

And I'm trying to think of similar cases. I think, for example, we introduce contains all, even though it's essentially equivalent to subset Q. Yes, for same reason. To have that member of the full family of the contains family.

All right, let's do it. Moving on. Oh, boy.

So this is a suggestion which, Nick, can you tell us, was this a WFR or where did this come from?

Yeah, there was a WFR that is at least somewhat popular for nega binary digits. So that's integer digits to base minus 2. But there is a, you know, general form where you can get integer digits in any integer base that's negative and it's deterministic. And so rather than obviously having a spin off function that's just integer digits in a different. In negative space.

Sure, sure, sure. Here's my question. You know, there's a unique way to represent. There's a unique sequence of digits that represents any positive integer, right. Or a given base. What's the statement about negative bases? Because for Fibonacci base, for example, there isn't a unique sequence of integers that represent a unique sequence of digits. In other words, for what?

There isn't for.

For a golden ratio base.

I see, right.

There's a. There is a canonical version of that, which I think we have. I mean, if I say. Real digits of PI comma 5 comma golden ratio, is that the other way around? I'm sure we had this. Oh, base B. Sorry. Yeah, there we go. Okay. There is not a canonical version of this. I mean, this is one of the possible sequences of digits in golden ratio based that reproduces PI. Am I making sense?

Yes, I think, at least for integers, I think this is, you know, I think this is unique. You know, the reason I'm not so thrilled about this is. Right. Well, if we added the integer digits, then, you know, there's like at least eight other functions that we would need to add it to.

Yeah, Digit count and all that kind of stuff.

And I'm not sure that this really just. I mean, this sounds like a great wfr. I'm not sure that it.

Well, I think in a perfect world we would add this to the system. Why not? I'm not sure it's our highest priority thing to do, to put it mildly. I think it is desirable, but not high priority is my point of view. But I still want to know about uniqueness.

I'm quickly reading about it. Apparently the main advantage is that you can represent both positive and negative numbers without the minus sign.

Yeah, right, right.

But my little table here.

But the complication of operations is very large. It seems to be unique.

Yes, okay, but for example, here we're just dropping the minus sign. So if this was in base minus 2, which it should presumably be up here.

Right. So if you look at. Right. So if you look at minus 5 through minus 1, I have the nega binary there. And so you see, right. You get two digits at a time. Right. Because you need to get back to the other side. So you go from 0 to 11 to 10. Right. Because the even slots are negative and. Sorry, the even.

Yeah, the even powers are positive and the odd powers are negative. Right, But. So what you're saying is that if I say from digits of this thing here, comma minus 2, I get minus 65. What do you know? What's that? What do you mean?

Huh, I didn't know that that worked.

Why wouldn't that work?

Yeah, you can put an X there.

Yeah, right. Exactly.

Okay. Guess I never tried it with anything other than.

You know.

Right. Okay. So fine. So. So then, okay, I'm taking it that this is unique. Therefore my comment is, You know, this would be desirable, but I can't say that it's very urgent. And I would say putting it into integer digits and then leaving it not in the other functions for the time being would be an okay solution. I mean, it would be nice to have it in everything, but I would say this is a low priority operation.

Okay.

I don't know if it's possible to have it so that it just calls the top level code in the case where there's a negative number there might make it easy to deal with this. Also we might be able to compile that top level code. See what I'm saying? I, I think the, My, my statement would be we should have this. If it's easy to do, we should do it. If it isn't easy to do, we could, for example, have it be top level for a while. It will be a good example of something which should be compilable.

Okay, Right. So.

Oh boy. Yep. Go ahead.

Right, so.

Right.

So this has to do with, you know, position largest and smallest. And I guess also take largest and smallest with association. So you know, position largest does this sort of funky thing where if you ask for one, it gives you the end. And position largest only works at level one. Just like take largest only works at level one.

Yes.

And so you know, if you give it. If you ask for one, it gives you an index. If you ask for multiple, it gives you. Well, I guess it's actually giving you a position in edge case. It's just giving you a singleton position versus a list of positions. Right. But then if you, you know, if you do that for.

I don't understand why those positions are wrapped in lists if it only works at level one.

Well, then there would. There would need to be no list in the singleton case either.

Right, I understand, but I don't understand why this sub list. Yes, I agree. I don't understand why position logist Works this way at all.

This was a very long conversation and of whose entire details I. I can't recall, but it had.

Oh, that's right. That's right. It's like communist.

It.

The problem is that there can be a tie. That's.

Oh, right.

Right.

Yeah. So if, if, if, if we do. Yeah, if we change to 15 to 20.

Yeah, yep. Okay, so now what. What's the question here? So you're saying if we say position largest of the association a goes to 10, b goes to 15. Oh, boy.

Yeah, I mean, this was partially. I had an example where I have an association where I have a bunch of keys and values and I want to figure out what.

So that is the example right after the next bullet point. Right. So if you compare position largest of the masses here with position of foo comma max, then. Yeah, we don't support that. We do that. We support that in take largest.

Where.

You'D also need a second argument.

So if I said here, I'm confused. Why isn't this the heck. What is index there?

So index is the position.

Right.

It's just that we didn't want to call it position to avoid that list you were referring to before.

Okay, but I don't understand what this. Oh, index is. I see the two there is the index, right?

Yep.

So what. What are the keys doing here?

Well, I guess the decision was made to preserve the structure. To preserve the head. Right. Like just as take. Preserves the head.

Yes. And. And it's reversed because it's in decreasing order.

Yeah. Okay, but so the idea would be that position largest. If I said position largest here, arrow key, that that would return B or list B.

Well, that's not my suggestion. My suggestion is that position largest right now is just plain wrong.

And what it does where.

So go down beneath the second bullet point and I'm comparing position largest to position, and I'm saying what position largest is doing is wrong.

I agree.

Position gives the key object. And I think it's good for position largest to be consistent with position associations.

This here. I don't know what this was giving here.

It's giving basically what you would expect.

Okay, okay, so now you say position of that comma max of that.

Right. Which is the input right underneath.

Oh, sorry. So what are you saying? You're saying that position logist should give that as output, not the 5. I agree with you.

Yep. And then, You know, then the question is, do we modify take largest. I mean, it would be weird if index was a key object. So then we could Either. I mean, we could just live with it.

Index is essentially the position. You know, we could say arrow key there and it would return B comma A up at line 131.

Right. And the question is, you know, and then the question is, what would key give in the non association case? Should it just be like not present or. I think it's better for it to be there with, you know, missing of not applicable in the not not association case.

Well, except that remember, keys of a. If we have a list of rules, qies just gives the less downside of the rules.

Right.

I mean, I don't know whether we think that key should degenerate to a position spec. Probably not. I'd be perfectly happy with the missing not applicable.

Yeah, me too.

Yeah. And the. Does Ticklar just even understand the list of rules? Yeah, Tickl just doesn't even understand the list of rules. Has to be a list of values of or an association in which case missing of applicable. It's the correct thing. If you ask for the key for the key of the largest thing and you gave it a list, would it be too much to change index to give the key?

I think so. I think index. Index sounds like a number.

Yeah. It should give the number which you can still use in an association with a raw part number.

Yep. Or with extract.

Yes. All right. I think we got through our agenda.

Yeah, there's. Since. Well, I guess we only have two. Two minutes. But if, if you will allow me, I wanted to sort of bring up one other thing. So if you open the HELP browser and go to ref slash program slash Wolfram, You need the ref in front of it.

Ref colon or ref slash slash. Okay.

Right. So you remember what you may or may not remember. Remember that when we created these things about a decade ago, there was some discussion about where examples go versus where all the options go. And Alex Newman and I were like, well, we should list all the options before the examples. And you're like, no, no, no, we have to do examples first. And so recently there was a, an email thread and Joel was like, oh, I looked at the ref page for how to get license information so I can script whether, you know, an activation.

I'm okay with putting the options first if that's what you're going to ask.

Yep.

Now I don't know whether we should try and do what we do.

Yay.

Well, the question Were you here for this whole time for that? No. I mean, look, maybe we should do the thing that we do on. On, you know, function pages of Having that closed section at the top, Because that's sort of the best of both worlds. There's a lot of detail here. And. And maybe. Maybe these sections should be closed up by default. I don't know. I mean, these sections being closed with attractors. I don't know if I can edit this, can I? Oh, yeah, look at that. What do you know? You know, if we close these and put little attractors here, I think this. And some don't even need attractors. That would seem pretty reasonable to me. We just didn't have the notion of. Of closed sections with the tractors at the time when we designed this.

Okay.

Does that seem reasonable?

Yeah, that. That seems reasonable to me.

But I agree the examples here are a lot less compelling than they are for a function.

Mm.

Okay, cool. Anything else?

That's it.

Let me ask this question. I mean, it feels like these agendas have been getting a little bit thin, but I feel like there's a ton of stuff we could be talking about. I mean, there's a kind of ton of serious chestnuts that we have not been roasting for a long time. You know, there are all kinds of. I don't know what I mean, from things that I've been concerned with, things like the multi mechanism, to, you know, ports and graphs, to all kinds of things like this. Another one that I really wanted to have, and it was Brett here. I don't recall. Yes. Okay. I want to have this graphics summit meeting where we talk about these things like spacing between graphics and graphics arrays and so on. Unless you think you solved that.

I mean, we've had several instances of that graphics summit, and I. Right.

So, Brett, do you think you've solved that problem? In other words, am I going to get any stupid graphics grid spacings anymore?

Oh, probably still.

But you think you fixed one round of problems or not?

It's gone through a few rounds of tightening up. And like when we beefed up absolute options a few versions ago, this was.

Still happening within the last few months. I don't know it in 14. Oh.

Well, I have not looked at it closely recently.

I think it will also be the case that our work on label graphics box will provide Brett with more and better tools.

Okay. When is that coming to closure?

I am hoping that we'll be able to deliver the box in this upcoming release, but it. I can't guarantee that it will be in time for Brett to make effective use of it, though, in that release.

Okay. Okay.

Still, I guess I'm off one backlog of things Which Itai and I just talked about in the last week, actually, which I know you'd want to see here, is conflations of symbols that are used as option names and as function names. Oh, gosh, those are. We have a collection of, I don't know, maybe 15 or so of those that. That should be reviewed.

Yep. Okay. Well, I'm just saying, I. I mean, I'm just trying to understand is there a giant pile of, you know, backed up stuff here or what? Because it feels to me like there's a lot of stuff should be being discussed here that I haven't gone and categorized, but I'm kind of wondering where it is.

Well, I mean, the question is, have things advanced to the point where it's useful to have another discussion? I feel like the ports thing was sort of spun off to other meetings. So I haven't. And I'm, you know, that's something we could maybe revisit. And, you know, I think we have a few other things coming in, like equal bracket and whatnot that will undoubtedly, you know.

Yes. Well, I'd love to see multi get some time.

What is multi? I do not know what.

Multi.

Multi sets or.

No, this function called multi for doing multi computation.

I.

That is not something that has ever been discussed in this meeting.

Maybe it was an ILD then.

Yeah, I mean, there wasn't there. There was a multi computation meeting series at one point, right?

There was, but I don't know what happened to it.

Okay, I can. I can check on that.

Offline. Yeah, that. That's. So maybe that's what happened, is that that meeting series for some reason died. Okay.

Yeah, I will see what's going on with that with Nick and if we need to bring that through. ILD if there's not a regular series.

All right, very good. Thanks a lot, guys. See you later.

See you.

Bye.