Live CEOing Ep 774: Language Design in the Wolfram Language [HistoricalCountries]

Wait, wait. Cesar, you're going to leave this, right?

If you want.

Okay, let's. Let's start off here. So we're Wolfram Language Design review about a topic we have talked about several times before, which is the representation of historical countries, kind of an encapsulation of human history in computational terms. All right, so what we have here, we. We've talked about this several times before and hopefully we are towards the end of figuring out how this is supposed to work. So we have geovariance. So the UN. Go ahead.

Yes. These are the three geovarians that we ended up with. We have union area, which is the polygon representing all the territories of a given historical country. The union of all the polygons.

So wait a minute. That's union overall time.

Yes. Previously we had a collection of all the polygons, but now using the geoserver, we can make the operation to return a single polygon.

Okay, so for example, if I say here something like British Empire. Let's see if the sun really never set over it. What the heck is this? This has been happening to me. Is that. Oh, that looks like it's part of the predictive interface. That may be some other thing. Somebody needs to reset that. That's a failure. How come?

That's NDD class of countries.

Well, should work as opposed to. So I mean this. I don't see why this doesn't work. We should make this work. I don't see why.

We have been thinking on a way to hook up country and historical countries.

I think that if you have a geovariant that says union area, it better work for classes of current countries. See what I'm saying? That's a simple thing. Should be. Should be easy to make it work. Okay, now why is this jammed here? Or is it taking forever to. Let's see. Well, I don't know what's happening.

Yeah, the thing with. With that British Empire.

Yes.

No, don't. Don't matter.

Well, now it's saying. Downloading 14 of 28 values. So what's happening? Is it not cached? Obviously you should make sure that these things are cached.

Yeah, I will check on that.

Okay, but. So I don't know what we should do here. But. But okay, so this. So I understand the concept of. Of union area. So what other.

We can. With the wrapper dated, we can restrict the time period in which we will do the union. So it doesn't work on. It doesn't unite all the polygons in all time, but once in a Certain interval of years.

What's this? Well, okay, so there's bugginess here.

Yes.

Okay, I'm sorry, go ahead. So, time series here. I don't understand what that's returning. I mean, in other words, down here you're returning a time series of polygons. Polygons. But here, what was happening here? How do we manage to display. Oh, I see, I see, I see. It's. It's that time series indexed into that year. What if I miss the years that. For which we. If I say 202bc missing value. But is that the right thing? I mean, do we have to hit it on the nose?

This is an issue we discussed of using interpolation. Yes.

So I think that what we should do probably is. I'm thinking, I mean, one possibility is to return the nearest date and generate a message. I think returning missing is not useful. Now if. If the date that you specify is outside the. The range when that historical country even existed, then yes, return a missing. But if it's just a data lack for us that we didn't have 201bc.

Ah, no, no, no, no, no. I didn't get you that it uses. If it's on a.

This is such crap. Look, look at this. What. What the hell's going on here?

Yes, I think I know what the problem.

Just make sure it works because it doesn't right now. And, and make sure. Whatever. It shouldn't take that long and it shouldn't. Oh, gosh, you didn't give me this. And. Where do we put. Where did we put this historical data? Is that where it is?

Yes.

Okay. You know what? It's always good to put the date in. Okay, all right, so I'm sorry, right.

Now, the way the time series work, if you ask for a year before the starting of the time series, it will give you the first volume. But if you ask for a year after the finish of the. After the ending of the time series, it will return missing.

Right now, I don't think that's right. I think it should return missing if it's before the beginning. I mean, you know, if I say here 800 BC. And by the way, that minus for displaying the year. So. So there's so many bugs here. Come on, guys, you've got to do better that. That's no good giving an absolute time thing like this, that you shouldn't generate that message. Am I making sense? You can generate a message.

Okay, yeah, we can report that to the time series, because that's a time series.

This is A. This is a message generated in the time series. Because what you're trying to do is to extrapolate and that's why you get the. You get the information that you're trying to.

Yeah, I know, but also the input value is. Is really bad here. You should know that. That's. You should give that in the correct format.

That could be improved upon. Yes.

And also know the format. Use the format. Yeah, okay, but, but you. You were just being. I was just being told that if I ask for something outside of the range, that I'm going to get missing. Oh, here you're doing extrapolation. If it's before, you're doing extrapolation. And if I say something like 2300 A.D. when the Roman Empire presumably did not exist for some reason, it's giving missing not available in that case, but not in this case. So I think it should give missing not available in both cases. Am I making sense? I don't understand the argument for doing it the other way. Do you understand what I'm saying?

Yes.

Okay, so if there is an argument, give it. If there isn't, fix it. Okay. Is there an argument?

No.

Okay, then fix it. Okay, so here you should not be generating a message from interpolating function. You should be saying something that is more about, you know, historical countries and so on here. Am I making sense? Because you must have a harness on this anyway that's dealing with. I mean, hopefully this isn't just a raw time series here.

Or.

Or is it, If I look at it is.

So it's a. It's a regular time series. Which content is.

By the way, these negative numbers for dates. I really don't think we should be doing that. Whether we use BC or bce, I don't know, but I don't think we should write negative numbers like that. I just think it looks kind of silly. Comments on that, please.

The problem with the negative is that also might be confused with a dash.

Well, I understand, but it just looks silly. So why are we doing that?

That's why it's problematic, I think, in the full date.

Yeah, but I'm asking.

We can do it for these calendars, right? There are other calendars with negative numbers and there is not a convention.

That's fine. That's fine. But also this January 1 thing looks like nonsense to me. Why isn't that granularity of years?

That's what hasn't been inserted, I guess, in the time series.

Well, we should fix that. We need to fix that.

Yep.

Okay, so this here, the Fact that you're using a time series and getting raw information from it. I mean, if that's the implementation thing for right now, I guess we have to live with it for right now of getting an interpolating function message. I don't like it. I don't think that's what time series should do. In fact, Gosha, that's just wrong. You should not generate a message for interpolating function when you, when somebody is seeing a time series. You see what I'm saying? That message should be associated with time series. Am I making sense? Do I have to repeat?

What do you mean associated with the time series?

I don't want to see the word interpolating function here. The fact that time series is implemented using interpolating function is a detail for us, what I'm saying. Yeah, that shouldn't have been done that way. It's sloppy. Okay, right. That should be something about time series. Saying time requested lies outside the range in the time series extrapolation will be used or whatever else it is. But that should be associated with time series. It's really sloppy to have let it fall through to the implementation error message. So. Okay, you'll fix that?

Yes.

Okay, so now here. So I think when that's fixed, that's less problematic in terms of the mechanism here. Am I making sense? Do you understand? I don't think we need to worry. I think that message will be enough for dealing with geovariant, you know, with the historical country, geovarian thing. Did you understand? Francisco? Anybody else? Do you understand?

Yep, yep. Makes sense.

Okay, for God's sake, please respond because otherwise I just don't have any idea whether you understand what I'm talking about or not. Okay, Okay, fine. Okay, Okay. So this pattern of saying time series follows the pattern of things like word frequency, data. Yes.

Yes, we follow that.

Yeah.

So this is with the same idea we created this variant.

Yeah.

So if I just said dated without giving the geovariant wrapper, that would presumably work as well. Right. In other words, if I say here, What the heck is this? Okay.

So if you call entity value of the dated and polygon. Yes, you get the same time series.

No, I'm asking a different question. If I say geolist plot of dated Roman Empire. Oh, gosh. Why didn't that work? Make that work.

That outside of bounds. Because the. You would be needing the Roman Republic.

Oh.

You can try on the year 200.

Oh, yeah, yeah, yeah, yeah. Okay, fine. Five, five, five. My, my apologies. Bad history. Now, although I'm Surprised I didn't get a message. Brilliant. Why didn't I get a message when it was outside of the bounds? Because I was getting a message here. When it's outside bounds, why didn't I get a message here? I think I should get a message there.

Yes, the message comes from. From the time series itself from the object. But we could give a message for the entity.

Usually we don't give messages for entity values that return missing by itself.

Right. I think this is a very obscure case. I think you should give a message in this case. I think people will just be completely confused without this. Am I making sense somehow?

Right, so it's a kind of edge case because yolisplot usually ignore missings. So that's why it doesn't give any message.

I understand, I understand. Geolist plot doesn't give a message, but I wonder if the dated should give a message. I mean I know dated is normally inert, so at what point does it know there's a problem?

So the question is. Right, so because the only plot tries an entity value of dated and gets a missing from the polygon and then ignores the missing.

Okay, so that entity value call should probably generate a message.

I'm not sure if we do that ever.

Well, maybe we should start. I think this case is really obscure to people. People are just going to think it's broken. See what I'm saying? I mean at the very least the missing should not just say not available. The missing should say something a little bit more clear about what happened.

I agree. So we need to think where to intercept this to properly give some information. Yeah, but it's not that straightforward.

Yeah, I understand. That's lovely. That's super cool. Now okay, you say dateless step plot. If I said date list plot and I said something like interpolation order 2 for example. Surprised that didn't give an autocomplete of different possible values. Now what's that? What's that?

Complain about the missing values.

Yeah, but it's got to give a more elegant thing than this garbage. Look, it either doesn't complain about missings or does is it complaining about missings. Does stateless plot complain about missings? I plots usually don't.

Right. It looks like it's complaining about the missings because you can see that this with missing somebody.

Yeah.

Okay, so when you go investigate that, I mean, I don't know. I don't know what happened there. The result was very nice, but I don't know what happened there. Okay. Geovariant maximum area okay.

Yeah, that one. Well, it's pretty self explanatory. The only thing is that.

Yeah.

When given the dated wrapper, it returns the maximum expansion in the respect.

Yeah, that's lovely.

Yep.

Okay. Geo identify.

Yes, there is a bunch of stuff that we discussed here. When there is an historical domain and an historical entity, it will try to match them in both time and space.

Yes.

Using the dated wrapper around the domain will return data droppers with the dates of intersection. Using an historical domain against a non historical.

Hold on, hold on, let me, let me just ingest what's happening here.

So.

If there's an explicit dated, then it returns. So if I said. So this is a space time point, is there any way that the second argument identified can be a spacetime region?

Yes.

So for example, let's say that I say. Okay, let's say I say this and then I say dated. Well, actually I just say Roman Republic. So this should generate a giant list.

Yes, this might time out.

Okay, well give me a smaller place. I mean give me a smaller one. I mean again, I would love to see a progress monitor here. Well, okay, but let me, let me give. Gosh, timed out. Okay, could you give me a smaller. A place with a smaller.

You could also restrict data.

Okay, fine, fine, fine. Okay, so let's say Roman Republic. And let's say dated Roman Republic minus 250 to minus 200. Why the hell are we got these stupid things here? Well, I thought we got rid of this, that calendar icon. Do you understand what I'm saying?

We got rid of that in date object, but apparently not in date interval yet.

Okay, so let's get rid of it.

Yep.

Yes, in that case it will.

That's a very nice result. Okay, go ahead. Sorry. In that case it will match also.

In space and time.

Okay, Two commas.

Roman.

Two comments, Two comments. The dated should canonicalize to just be the simple form not to have a date interval. There I'm making sense. Doesn't seem any, any harm is done by that and it's easier to read. Do you understand what I'm saying?

By simpler form you mean the pair of of years?

Yes, that is correct. Right.

So what happens if we have dates that are. Or finer granularity?

I'll give those. Then dated. Give those two of those dates. Dated allows that. I'm just saying don't use date interval, use the list pair.

Okay, so if, if they are years we use there.

Well, what about if the. I mean if they're January 13th of, you know, 19, 22, why don't we just say January 13th of 1922 in the LISPA.

With.

With lists of numbers or. Or with a pair of dates with.

A pair of date objects.

Fine.

Yeah. Dated should dated surely allows date objects in it.

Yes, at least one. Yes, it does.

Yeah.

I don't know.

Well, let's make sure it does.

Yep. Yep.

Okay. And I recommend that as the canonical form so that we can just use naked integers for pure years. Yep. Okay. This is really a nice result here. That's very impressive. Okay, so now geo identify historical country. The city of Rome. That's cool. I got to figure out what it says for. Autocomplete is not there, by the way. Probably will get automatically fixed. But I'm just pointing it out. And Joe, identify the first argument. Autocomplete isn't there. Do you understand? Can somebody respond to me, please? I got it. Thank you. That's cute. That's really quite cute. So. And if I were to say something like this, I mean, I actually don't know in the US whether there's any. Whether one should have. Well, okay.

Yeah, we don't have.

No historical country is known for that time.

No.

We don't have indigenous. You know, I mean, I don't know whether that's properly mapped. But. But the.

Yeah, we don't have that data.

We might take a look and see if there is any of that data available. Be interesting. This is really nice. It's really nice. So now I'm about to learn some history here. So that's going to be anything that intersected Peru at any time in history.

Yes.

Cool.

And if you use the dated wrapper around historical country, it will give you the years in which.

Can I just use dated like that or do I have to say all.

Oh, oh.

This is really nice. It's really nice. It's very. I mean, I wish it wasn't so slow. Lovely. Why is this font all messed up? Does it look messed up on your version here? Doing Command 7. What the heck is happening to that?

Yes, it might be because some of these examples are actually from the documentation we've been working for mathematics.

Okay, so. So I don't understand this. Why doesn't it just work for Confucius on his own? In other words, I don't see why. I mean, there's one thing. Okay, let's look at this.

So the problem is that this is not a GEO entity.

So.

So we don't have any. Let's say what we want.

These are place terrible like these error Messages are absolutely terrible. So why don't you. In this case. Well, I mean, hold on, let. Let me just try something else. Okay.

This is something interesting because as of now, the only two historical domains are military conflict and historical country. So every other entity is taken as an only geographic.

Right, but that's not right because we do have information, for example, for buildings and so on. We do have information. I mean, if I said here, you know, something like lake. Is that a lake? I can't remember. Yes, right, that obviously a lake is a lake. First approximation, it's been there forever.

Yes. Our next steps.

What's that?

We're trying to. We're going to try to expand the domains, the historical domains to other geographical domains that we have, like building historical site.

This is disappointing again, more caching probably is needed or something. Do you see what I'm saying? Why did that time out? That just seems wrong to me. Right, but, but okay, but. So we should have a list of. That's lovely. We should have a list of. We discussed this in one of our previous meetings. A list of entity types that support creation dates and so on. Now a person, for example, that's a tricky issue because the person, you know, Hannibal wandered around all over the place. Right.

Right.

For persons, usually we have the place where they were born and also death place. Right. And also the dates for. For both event. So the problem is that there are entities that might have many events associated with them.

Right. So I think what we should do for a person that's mobile. Okay. A building is not mobile. Right. So I think for a person we should just generate a more sensible error message just to specify, you know, birth or death or something. Do you see what I'm saying?

I see, yeah.

You know, now I also make another comment. I, I make another comment here. It's possible that we should add an A property for a person that just says birth, for example, and that returns the pair of place and date. Do you understand what I'm saying?

Yes.

Is Alan here by any chance? Probably not, no.

He's off today.

Okay. All right. So I mean, that would be something for him. I would, I would recommend that. I think given that we've got all this nice functionality here, this would be, this would be useful. That makes sense.

Yes. And I think the opposition accepts a date.

But that's an absolute time date.

So.

So it's not the same as the database we need for.

Well, we probably need to do that. We probably need to say geoposition. Well, I don't know. Do it. Does Geoposition work with dated.

In which sense dated inside. No, but we can have at least lat. Long height date and we can have a date.

I know that, but I'm. I'm saying. I'm saying if I want to know the geoposition. I mean, if I say geoposition of Paris here, for example. Actually, if I say geoposition of France. Okay. It's going to give me a geoposition. If I say geoposition of dated. Roman Republic minus 100. What is that?

I think it's the center coordinates for that date.

Okay, great. Then this just worked. Check that it's not something crazy internally.

This is a call to your entity. Sorry. To entity value. So.

That works correctly. That I'm happy. Yep. Okay. Geo entities.

It has more or less the same characteristics as geo identified, only that it identifies entities within a given entity. Like for instance, military conflicts in the Roman Republic. Stuff like that.

It's very few. I'm surprised there's a few. Why are there so few? Is that because we don't have enough data?

There is. These are not conflicts that had the Roman Republic as an actor, but that happened in the territories of the Roman Republic.

Oh, I see. I see.

They fight more outside than inside.

Yeah.

Yes. Okay, fine. That makes sense. Those are people attacking the. So if I. If I do a timeline plot. Let's see if we can see the. The Roman Republic fall apart. Why is this so slow? Again, could you please look into the possibility of generating. Okay, so that's. That makes sense now. Boy, that's interesting. I mean, those were times when. When they were, you know, under attack, so to speak. How cool. So if I say date histogram, Why didn't that work? Okay. Something to. To debug. Okay. Date histogram I think is not accepting historical entities. Makes sense. Please respond. Is somebody going to get this or do I have to worry about whether this is being tracked properly?

No, we'll communicate this to Brett. I think it's just that there is not a connection between the entities and their dates yet.

Right.

We had to do that for something like duration, for example. We had to add it manually.

Yes.

Military conflict also work with date intervals in order to support complex set. Might last a bit longer. I don't know if that works with date histogram.

Okay.

Yeah.

That's something to do to discuss later.

Yeah.

Thank you. Okay. What does it mean by nearest in both space and time?

It intersects in time, but such as nearness by especially only.

So I see. I see. Selecting in time and then doing the geo nearest not space time. Nearest.

Yeah, we don't have the arrows, we.

Don'T have date near it, so to speak. Yeah, okay, that seems sensible. Okay, so geo withinq is the same intersection of time and space in that case.

Yeah.

So in a sense it's date within Q. Geonearest and date within Q in a sense up here, right?

Yes.

Oh boy. Geodistance. So wait a minute, we've got these space time regions and this is asking as we scan through time, what. What was the closest they ever got at any time? Is that correct?

Exactly.

Which is you get by having union area of the two things. Ah, but that doesn't work because union area might give you the things at two different times. In other words, the. The proboscis going into, you know, where the Roman Republic took over the Azores or something might have happened at a different time from the time when the Han Dynasty took over, I don't know what, Japan or something. Am I making sense?

Yes, that's why the. When there are two historical entities, it will calculate the distance within their polygons or shapes at the same time.

Yeah, that's great.

And check the minimum. If you want the other stuff, you can use the geovarian tuning area.

Right. What's this?

When there is a non historical entity, it only calculates it as a geographical position.

All right, listen, it's taken a long time to get here, but this functionality is lovely. It's great. Let's just make it faster and make sure if we can at least get a progress dialog up, that will be really good. And there are a few rough edges to be. To be smoothed. But in general, I just think this is spectacular. I mean, you know, the, the kind of computational history that you can do is really, really, really nice. And it's kind of a. Honestly, it's a. It's a reason for us to try and get more historical information. Because, I mean, what. Well, Alan isn't here, so we can't really talk about that. But I mean, this is really nice. Okay, any further comments from. From Johan Jose? Anybody?

No, go ahead. Okay, so.

So I.

We're just going to mention the idea that I think we have discussed in the past in the same way that entities in geoposition arguments, immediately called geopositions, think we should find the collection of entities and a single property such that when we have date arguments, we can extract that property immediately from these historical entities. And this would apply to visualization functions and other.

Yes, so wait a minute. That's one way around. But I mean, in, in, I Mean there are things like the discovery of a chemical element in a particular place in time. It's a little bit funky to put, I mean, I don't know if geolisplot of tungsten and molybdenum or something. Molybdenum and gadolinium do we want to show. If you say geolisplot of those two things, it's a little bit funky to show their discovery places.

We have a position of say a continent, which is a bit strange too. But this works because the position property works systematically and that's why it's so easy to propagate it. So yeah, it requires a bit of thinking and curation to assign, let's call it date property to everything.

But.

Otherwise how can we make visualization functions work with.

Sorry, sorry, sorry. I spaced out. I got distracted by something else. Could you repeat that please?

I was thinking about your idea just a few minutes ago of making visualization functions work with entities by extracting a property.

Yes.

And I was thinking that this can be done if we agree on a common name for a property that means a date.

I agree.

We have the position property for all the entities.

Yeah. Right. Right. So let's do it. Right.

So it might be difficult to decide what's the date of one of the things you were mentioning before.

Well then in that case, the quotes date property just should be omitted. But if there is a date, I mean, for example, if you say Eiffel Tower date, arguably that should be a, a date interval from when it was built to now.

I see.

I think that's a good way to do it. I mean, to have a date property and then for each thing it can decide. Right. If it was a date of construction, a date of destruction, etc, it can, it can pick that up. I think it's a good way to do it. Just like we have a position argument, we could have a date argument.

A date property position is warranted to be a geoposition, not a polygon, for example.

Right.

That's why it's so easy to manipulate it everywhere because it's homogeneous.

Okay, but so then we might want a, a, We might need a name for that. Could allow intervalness. I mean, most of the things, I mean, I don't know, nuclear explosion, for example, that, you know, that has a place and time, a very definite microsecond. An earthquake has an epicenter and a particular time. Yep. You know, even a volcanic eruption might have an interval. Yeah, I don't know. I mean, I, I, I don't know if we say a date property, you know, should we allow date intervals or should we. Should we nail it down more? I have to go. But this is really lovely, guys. Mean, it's taken us a very, very long time to get here. How long has it been? It's been like five years or something, I don't know.

Yeah, but we've been releasing a different things in the.

In the meantime.

I know, I know, I know. But this is really nice. It's really, really nice. And just. There's just so many interesting things that can be done with it. Anyway, Cool. All right, we should run along here. Thank you all very much. See you soon.