Live CEOing Ep 732: Design Review of Word and Language Data

Okay. Hi, everyone. All right, so hold on one second, I'm trying to get somebody to work here. Okay, well, welcome to a Wolfram language design review about words and language, which means human language is still a thing. Hold on one second, I need to make something work here. Okay. All right, so.

I think in the, in the vein of clearing out old chestnuts and a lot of things that are in varying states of completion and review from a lot of different word and language related domains, there's a lot of stuff here. So there is the overhaul of word data, splitting entities into word and word sense entities. There are also phrase and phrase sense entities. And we need to talk about kind of how we handle that. There's the question that still has been outstanding for a while about we have a lot of different word lists and dictionaries that fulfill different roles in the language in different places that are not all synchronized with each other and some questions about how we want that to actually be realized. There's also some stuff that's in dev mode just in language data. Some new properties, new pods and the scanner side on Alpha side, there's a new domain language family which is mostly. And Christian crew can say more about this. But mostly taking things that used to be entity classes in language data and representing them as entities in their own right because they themselves can have different properties associated with them. And there's a whole hierarchy of things there. Also some of the Design questions about 2 grams word frequency and the kind of phonetic phoneme data stuff that we've talked. Talked about at different times in the past, but still there's a lack of clarity about what we're trying to accomplish in the end there. And a lot of this also kind of got started before we entered the LLM landscape. And so there are also some questions about whether there are things we should be trying to use there or how we account for the rise of LLM related stuff, given that that's something where they're strong.

Okay, shall we dive in?

Yeah, I think so. So one of the things here, you know, the, the surf word. The word and word sense entities have been set up in the surf so they're. They will behave like regular entities for the first time.

Yep.

You know, ultimately the thing that we would like to do is get rid of the old WDX word data packlet and have prefetched data, have actual entities, have the data at least there finally be synchronized with what's in alpha. Because the alpha word data stuff has had more surgery and additions and tweaks over the years. So some of this, I think is just useful to look at and talk about the design, I mean, the names, how the individual word senses look. Does this make sense for us in language?

I think the answer is yes, this looks good. I mean, I think, you know, I have a minor objection. That should be slightly lighter gray because I mostly want to see green. But what is that green there?

That green is a green that doesn't have a sense tag. That's one of the things that we've talked about that possibly.

Right.

Well, it's also one of the things that potentially, you know, an LLM might help with suggesting things to fill in there. I think that'll be one of those things. Right. I mean, it. It works pretty well just in experimenting with that. And those still will obviously need a human to look over them. But just for example, naming a suggestion or two.

Yeah, right. Naming a sunset is something I really think an LLM should be able to do. Yeah. And to what extent do we have any hope of being able to get word frequencies for these different word meanings?

You mean for like at the sense level, not just at the part of speech level?

That is correct. Here's why I think that's not hopeless. Okay. I think an LLM might be able to do that because it can know what is its representation and embedding space of a green as a piece of land. Right. It has a representation as it, you know, that is an embedding. There's some vector that represents that meaning. And I don't think it's hopeless to be able to say in. If we just did the most basic search then and said, you know, how many things are there close to that place in embedding space in common crawl, let's say that would give us frequency. And the reason that's useful is clearly in the case of green, it would be very useful to know which of these are obscure. Like, I have no idea how ketamine relates to green. I don't know that I want to know. But, you know, that's not a meaning I'd ever heard before. Right. Whereas green as spectral color is certainly a very, you know, a very common meaning. And I bet we can deduce that.

Are you thinking. Are you thinking about this just at the level of. Among word senses, their relative frequency or trying, trying to do this, you know, this sense in the whole space of word senses?

I think. Well, I think relative would be. Would be necessary. I think in the space of all word senses would be nice. And I bet there are uses for that, okay, but I mean, you understand my idea that basically we find the embedding vector and each one of these things, potentially one of its properties might be an embedding vector. Okay, see what I'm saying? I mean, if we have a standard embedding now, maybe that isn't a good idea. Maybe we can't burn that in yet. But if we have a standard embedding, then we can say things like what is the distance and embedding space from this word to this, from this meaning to this other meaning? Which is very cool. If we could do that. I mean, that's what we have. You know, just to give a sense of how that works. Let me just remind people. Or does everybody understand how this all works? Everybody understand? Should I remind people?

It wouldn't, wouldn't hurt if you can do it. Quick demo, I think.

Let's, let's do something here. In the chat, GPT explainer, I had some stuff about this. How did I have this someplace? Here we go. Here we go. This is a good example. Okay, so this is the word crane. Okay. So this is projected embedding space. Crane as in the wild crane was born at this time. This is in the railway cranes. Okay. So I don't know quite how we represent this, but this is like the center of the wild cranes is up here, the center of the railway cranes is down here. Do you get what I'm saying?

Yeah, I see.

And, and in general, you know, we know in overall embedding space that's just a two dimensional projection of embedding space. I, I kind of think this is useful. I mean, I, I don't know who.

Would be the best person outside of this group to talk about trying to.

Determine and the machine learning people.

Great.

Okay. But I, I like this a lot.

Okay.

This is good.

Good. I mean properties similar. I mean this, this would be just updating things so that we're, we have something that's more typical of, of what we currently do. I mean, I think like other packlets that we've updated. If we're just talking about the word data function. Right. We can have a, a legacy method that lets people get the old stuff if they have code that's expecting these three.

I don't understand why in this case, by the time we're doing definitions, I don't know why we don't have a list of definitions here. I mean there's semicolon.

It's, it's also just kind of a naming difference. Right. Because the, the old word data is Just this is something we'll have to account for too. I'm faking this just because we have two different entity types that are feeding in here that have singular versus plural definition properties. For now, I think we would standardize that just to definitions.

Yeah, fine.

But I'm sorry, when you're talking about a specific sense, you're only going to have one definition.

I understand that. I'm just saying that on the right hand side here, I'm saying this should be a list instead of having a semicolon there break the definitions.

Yeah, but I mean, those aren't. Those aren't multiple definitions. Like, if that was in a dictionary, that would be. That would be the definition.

I know, but, but I mean, you.

Want to, you want to split things up like that.

I see. You're saying this is a single thought. Yeah.

Yes.

Okay, you're. You're right. In this case. Okay, you're right, you're right. So your basic claim is that anytime it's a single word sense, there's a single thought for the definition.

Yes.

Okay. I bet that's right. Okay. Sandra is asking on our live stream, would, would data, dictionary lookup and word list will be the same. I think we're about to get to that. We're not there yet. That is where we're going. But we're not there yet.

Let's.

We'll talk about that in a moment.

And just to compare, this is sort of the chaos that you get with trying to get all the properties for a word and word data.

American spelling better be not missing, not available. It better just be green.

Yeah, it can be. And a lot of the. Another question that I had here, because we've, you know, the old design in word data was listing all of the senses and then, you know, empty list. If there was no value for that sense for all of these things. Right now this is just kind of getting flattened out where if there's. There's not a specific value for some of these other properties, do we want it to collapse down to just, you know, it's missing for everything, or do we want to always return an association of.

Okay, so the issue is some of.

These things and Sorry, for spelling. I agree. Like that. That's at the, at the head word level. There's no reason that that shouldn't be populated with.

Right. An anagram, you know, just pure. Whatever it is. Yeah. So I think there are two categories here. There are things that are fundamentally properties of senses, and they're things that are fundamentally properties of the essentially Morphology of the word. Right, right. So, I mean, this word. Let's talk about, I mean, in terms of the standards of linguistic classification of, you know, we've got the morphological stuff, we've got the, we've got the syntactic stuff for words like what part of, well, part of speech we can't yet do until we have word senses.

Right.

Then, then we get to, you know. So, I mean, I, I think we. Yeah, I think, I think we should just decide for every property, is it a word sense property or is it a word morphology property, so to speak. Right. If it's a word sense property, I basically think it has to be broken into all its senses.

So we, even if, even if they are all, we don't have a value for that property for that particular word for any of the senses, still return an association with a bunch of missings.

I think so. Well, it wouldn't be necessarily missing. I mean, like inflected forms. Isn't it just a null list when it doesn't have one? See what I'm saying?

Yeah.

In the case of inflected forms, it's got to be an inflection, you know, that's got to be an association of what the inflection is, so to speak. Which, by the way, the LLMs do brilliantly.

We're, we'll get to that further below too.

Okay. Yeah. It's gonna. Tell me they're not as brilliant as I think they are.

No, no, no, no, that, no, actually, I, I think, I think generally you're right. I mean, but that's a, that's a, a level of detail that we haven't gotten from other structured sources to date. And it is something that I think they do well, and it's, it's another tool to flesh that out. And so the question is, how do we want to break those out? Like, we want to standardize and have associations of specific named inflections and their values for each thing. Right.

So what I think here is that the generation of a complete dump of everything can be quite large and that we should really aim for as much rectangularity in what we're generating as possible.

Okay.

Even if it's big and even if some things are missing, I just think there should be an expectation that you can go in, you get the word sense you get. Now, there may be. And we'll come to this, There may be. I don't think it makes a lot of sense to ask for the inflections of a word like green and without talking about its part of Speech or its sense or whatever else. I don't. Okay, here's a question. And this is maybe the syntax layer. Given that you know the part of speech, do you know the inflection or do you need to know the word sense to know the inflection? Let's determine that. I don't know. Yeah, My guess is you do not need the word sense in that case. Anybody else want to opine on that? Linguistics oriented folk any up names here?

Probably not.

Yeah.

So the inflections will be common to all senses that share the same part of speech of a given word.

That's what I'm saying.

Is your claim.

That's what I'm claiming. Which means that we might want to have a. You know, that means we might need an entity that is green as a verb.

Yeah, that's the question. That question had come up too. Right. I think in between. In between somebody else.

We have people here who know about linguistics more than I do.

So if we are going to extend this to other languages, maybe we will need that difference.

Which difference of inflections being distinct based on.

Yes.

Okay.

Interesting. Yeah.

I was going to say I don't see any immediate problem with it, but I haven't given it a lot of thought.

Okay, well, let's find out if there's an example of languages where the inflection depends on the sense. Okay. But. But that, that still makes us want to have a. An entity class that is green as an adjective, for example.

Right.

Which I think makes sense.

Well, we can. I mean, we can represent that. I mean that it's the kind of thing that. Because we've factored this out this way you can have an implicit entity class or implicit entity that is green and, you know, part of speech. I guess it's just a question of whether we want a unnamed stored.

My guess.

In the database.

Yeah, my guess is we will because I think particularly when we, you know, for example, we can. Absolutely. So here's an example of something we.

Sorry, I mean that, that actually that would be the thing that, that would let us align directly to things like the, you know, the current N grams data because that's the level of frequency that we get. Right.

So here's a very cool function that we should have which is an LLM based function that basically takes a sentence and takes every word and replaces it by its word sense entity. Interesting. Which it should be able to do. And moreover, we should be able to replace every word eventually by its fully inflected. You know, by its inflection Description and so on as well. And again, the LLM should be able to do that.

So then actually that. So there's a. There's a use case that maybe answers a question that I had further down, which is right now and sort of historically, with word data, we haven't necessarily had. I mean, they weren't entities, but let's call them entities. All the, the entries in word data, we didn't have one that represented every plural form and inflected form of every word. And we would. If you queried for. And I think I have an example further down below. Something. It's a plural of something or some inflected form of something. We would just canonicalize it to the base form and we'd return data for the base form.

Yeah, we want.

We want to have a decorator, an entity. Well, a decorator. Or do we want to have an entity for every. I think we want a decorative form of a thing.

No, we. We do not want entities for every weird inflected form. And by the time we're doing it in Ithql, every word is going to get multiplied by 60 or something for the number of, you know, tensors or whatever. I don't think we want.

Because for the most part, the day the data associated with the things would be, you know, a definition would just be. It's the plural form of this. It's the. Well, the pluralism, person, whatever we'll put.

Right. But. But the way, I mean, it's like geovariant. Something like geovariant. It's a wrapper that essentially commutes with a definition. That is, if we say, if it's a word, if it's the word green as a verb. Okay. You know, we do not want a separate entity for Will. He will have been greened. Okay. Which probably has occurred like three times in the whole web or something. You know what I'm saying?

Right.

So. But the whole point is he will have been greened. If we say what is the. You know, we want a. A definition or something like that of that. What we really want to do is commute the definition inside the will have been, you know, whatever that is, future perfect or whatever it is. Right. Designator. Right. So if we have a wrapper that means future perfect means, you know, I don't know what it is. It means third person singular, you know, masculine, whatever. Future perfect thing of this word, sense of, then I think we're in good shape. And then. Because then that can be factored out. And I believe that that's kind of one of the meta messages of linguistics is that such factoring makes sense. I mean, it could be the case that it's actually an interesting sort of theoretical point about linguistics. It's sort of implicit, I suppose, that you can factor tenses, you know, you know, number, you know, plurals, versus whatever, you know, cases in, in the, in languages that have that and so on, that you can factor those, which is not obvious, but we might as well use that fact that it is a, it is an assumption of linguistics that you can factor those things. I claim, I don't know whether anybody else wants to agree or disagree with that statement. If you couldn't factor those things, go ahead.

Sorry. One or two things this brings to mind for me. So first of all, because some of these terms are different parts of speech, they will have different forms. So for example, some of these will be able to be modified to greener, but not all of them. So if we, you know, type in greener and look and try and get all these properties, there will only be a subset of green. It will not be all, all, all of these words. So I, I think that's definitely something to keep in mind.

Take that, take that apart for a second. Okay, so you say if what comes in is essentially a, I don't know what the general case of the word inflected is, but a, a modified form of the green base word, then one very interesting question is which senses? What could this be a modification of? Because clearly greener cannot be a modification of the verb form of green, but it could be a modification of the adjective form of green. Yes, and so, okay, so that's a good function that we haven't really necessarily thought of before, which is given an explicit, you know, given a surface level word like greener, what can our symbolic representation, what symbolic representation is that consistent with? So essentially, another way to put this is the following. Imagine that you put in a word like greener. Okay? Imagine you put in green. If you put in green, essentially the semantic parser will say there are all these greens that it is. In fact, we've got a list pretty much here of those are. The semantic parser will say your green could be any of those greens. Am I making sense?

Yes.

But if we type in greener, the semantic parser is going to say there are now three possibilities for that and they all relate to, I don't know, like the sick green, maybe the unripe green, those kinds of things, the gullible green, etc. Right. So in other words, the Semantic parser is going to return something that is different depending on what the sort of surface word is. Does that make sense to people? And that seems like a reasonable way to do it as well. So. Okay, good point. Thank you for that point.

And this sort of ties into one other thing. I don't know if we wanted to get into the sort of the World Atlas of Language data stuff here, if that was something we were going to get into, like the. Okay, so the only comment I had to make about that is that that could provide us with a helpful framework to determine which specific rappers we are working with and how they function.

Well, wait a minute. Rappers as in, as in like plural.

Or the way we want.

Right, right. Now that is a. That's a. I mean, inventorying what can happen in all the languages of the world is worthwhile. I mean, you know, I think. Yes. I mean, I don't know how many I know. Like Ithquil, for example, has tried to inventory many of the possible tenses. And I'm sure the World Atlas thing has. Has another inventory of possible tenses. But I mean, I know that, that, you know, the, you know, hortative, perfect, whatever, whatever it is which may occur in no human language is a, you know, you ought to have had, you ought to have had or something like. I. I'm not sure I can do it. What, what that would be. But. Well, and the other point is, in English, we break all that apart and it's lots of different words in other languages. We might, you know, have all those things stuffed together, so to speak.

Yeah, yeah. I mean, I just think it could be a good starting point. I don't know.

It is. It's important. It's important. We have to inventory as much as possible. I mean, you know, the only thing is we have to make some base assumptions about how language works, such as the fact that we can, you know, I mean, it is a fact, I believe, somewhat surprising fact that the parts of speech do actually work the same in all languages. Basically, that there aren't, you know, fundamentally different parts of speech now that majors be. Because, you know, everything. We've just got a hammer and so all we see is nails, so to speak, you know, in the whistle languages of the Canary Islands or something. Maybe there's a particular whistle that. It's really hard to say that's an adjective, for example. But you nevertheless do that just because we have adjectives to talk about. Look, for example, in. In Toki Pona, for example, this minimal language, I believe there's no Distinction between nouns, adjectives, verbs, which is similar to, you know, I, I think, I mean, that language, it's. It's, you know, it's like baby talk type thing. You know, I think, I think young kids don't distinguish in many cases between kind of those different forms. But anyway, I. This is. Yeah. So, yes, I agree that that's a really worthwhile thing. And I think it would be not a crazy idea to start off with some LLM discussion of this because I think the LLM can probably give us, you know, I bet if we ask it, you know, if we say. Give a list of verb tenses of. Give list of all verb tenses in English. Okay, what happens if I say the same thing? Hey, what is command? L not work for chat. Let's say in. Anybody got an obscure language that anybody. That anybody here actually knows. I'm gonna, I'm gonna just type in a language. Wow.

You try Japanese as well. Just curious if it has variations for transitive and intransitive also.

But those aren't those moods and things or what are the. Oh, these do have moods here. This one does have moods. Okay.

Yeah. So it's not partitioning based on the form of verb, but.

Okay, well, look, it's. It's clear we can get a huge amount of useful data here and we should just go and do these and try and try. And I think it's also nice that it can. I mean, clearly it can take a word and, you know, a base word like walk, and it can probably.

We can, we can spit it. We could spit out a nice structured list of tense or form to variation.

Right. And by the way, I suspect this is doable by a fairly low level LLM. I don't think you need the fanciest to achieve this, But. Okay, all right, fine. All right, onward we go. So let's take a little look at a few of these other properties for logical derivatives. I have no idea what some of these mean, and I think we need to relook at them because I think some.

Yeah, I think. I think so too. I mean, we've just pulled things from the old version and so there's stuff to do here.

Phrases.

Phrases is a question. So we, for the time being, Leroy had split out anything that's a. That's a multi word phrase, you know, a multi string thing from word data. And those are living as phrases and phrase sense. These, I think, all could be aligned to phrase entities. Right. Now, I've never been crazy about the phrase type name, but haven't Come up with a better.

Okay, so look, I've got some comments about this. Okay, Bowling green. I'm buying it. Okay, Green frog. I'm not buying it. Okay, Green dinosaur. I'm buying it even less. In other words, there are things, things.

Things where they're clearly just somebody slapped an adjective on a. Yeah, another thing.

Yeah, yeah, where, where there is a factoring of meaning. In other words, you know, isn't it the case that one of the features of language, I mean this kind of compositionality idea for language, which is, you know, you can put things together and together they mean something, which is the together meaning, so to speak. And then there are things which don't work that way. Like a bowling green. I don't think. Well, maybe there's a notion of a green, but I don't, I don't even know what a bowling green is. So maybe I shouldn't. Clearly a green beret, which is somewhat a proper noun, has, Is not the same thing as a beret that happens to be green. You see what I'm saying?

Yeah, of course.

But a smooth green snake, I strongly suspect is a snake that happens to be smooth and green. And a green eyed monster is surely. Or green eyed is somebody who has eyes that happen to be green. I bet. Yeah.

I mean, green, I think it's a green eyed has. Yeah, green eyed has a, has a connotation of envy. And I'm looking at green dinosaur. And apparently green dinosaur is actually a, a term like referring to a living fossil or so called green dinosaur.

Okay, okay, okay.

Some of these things that seem a little nonsensical, but it's worth reviewing some of those.

Okay, but if that should be phrase entities.

Okay.

If they are non. Okay. The definition is okay. You know, and I bet we can ask an LLM this, right? Given this thing, is this compositional or is it not?

Yeah.

Okay, so, so but if green dinosaur is a phrase, I mean, I'm a little worried that, you know, we've got 40,000, 50,000 words in English, for example, by the time we have these phrases, how many more are there?

I don't know.

Yeah, well, so I mean, we have to worry about that. And okay, but it's, and I think it is of, of decreasing importance now. The only thing that worries me, let's take, you know, French, for example, I don't know where, you know, silver play is quotes the word for please. Right. But it's multiple words.

Right.

How are we dealing with that? Oh, there's got to be English. Give me an English example of that. Gosh, there are surely English examples. But.

But for. For English, you have all these verbs that are component verbs, like pick up, catch up.

I see. And then. Yeah, okay, so. But then you have weird things like web log or. Actually, I had one. You know, the ones that are hyphenated and might not be like, I had one yesterday. Co found. Oh, yeah. Which looks really weird. Smooshed.

But nevertheless to me, like somebody misspelled confound.

Yeah, yeah, yeah. Right. So, you know, but you know my principle that most words smoosh. So we should, you know, smoosh is the future, so to speak. But I think. And. And then the question is, what hyphenated words versus, you know, like purplish green here, you know, is that. Or red green? It's complicated. You know, I know my friend Chris Cole has thought about this endlessly. We might. We might try asking him some. Something about this. I mean, he's. He's tried to figure out. His favorite example is air vent. Is air vent. You know, it's not in dictionaries apparently, yet. It's clearly a word in some sense. But in what sense is it a word? We've got corners of this whole business, but we've clearly got a lot of stuff that isn't corners, so to speak. Yeah, okay. I don't understand how superset terms can be missing, not available. I mean, it's a color, for goodness sake.

Just because it's another one of these things that we can also fill in. It's stuff where there. Yeah, there's stuff like that.

I mean, that's crazy too. Written frequency is exactly this. So how could we have a written rank and not a written frequency?

The old frequency things. I don't know what the. Where that stuff came from. The. The word frequency per year is the stuff that we got from our own NGRAMS database that the written frequency, written rank things are.

Okay, I don't even know. All right, let's keep going. What's all this?

Just at the word sense level. Same. Mostly the same set of. None of the. None of the things that are only word level, but things that are specific to the sense. I guess there's a question here about whether we want to push things down to the sense level and duplicate them there. Like. Like American spelling was a good case. You want that to live only at the word level, or do you want to also duplicate those down to the sense? Right.

Don't know.

You know, I kind of like keeping them at the word level to more.

Clearly but is it correct? Or other senses in which it is. You know what? I'll give you an example between British. I'm pretty sure ensure insure versus ensur. If you say. In American. Right. You would still. I think you. You still have the en form in America, don't you? Or not?

Yes, yes, there.

Okay. I'm pretty sure there are some examples where the American form is degenerate, so to speak, where it's distinguished in the British form and not in the American form. So what for us, what does that mean? That means. I see. That means for a certain word sense in American, since we're basing our words on the American spellings. Well, again, somebody's got to answer that. I don't know the answer.

Yeah, yeah, no, it's good. We can. We can see if we can find some examples of that.

But I mean, we. We have to have certain assumptions about language, and it could be that it's good enough. Like this compositionality assumptions are, you know, or everything is. Everything has some part of speech. For example, there aren't words. Again, I don't know what happens in languages with whistles and things, whether there can be things that are. I believe everything is. Is bucket into parts of speech. Is that. Is that a true statement from linguistically oriented folk here? In other words, if you take tonal languages, for example, those are like diacritical marks. Is that true? I mean, they're treated that way. They're not treated as a. A tone on its own means something. Is that true?

Yes, I believe that's true, generally.

Yeah.

Although here's a weird case. Okay. A question mark on its own punctuation. What is that? Because that's a disembodied. That's a thing for which we have no representation in our, you know, in word data. Question mark doesn't mean anything. That isn't a thing in word data. Yeah, but I have to say I'm. I'm questioning whether there might be languages in which diacritical marks meets punctuation.

Yeah, I mean, you know, Mandarin Chinese is a classic example where you have basically a suffix on a sentence that turns it into a question.

Like we do in English.

Yeah, but it's not punctuation. It's. It's not actually a question mark. It is a linguistic. You know, you would add basically a single syllable word at the end that makes it a question. Or the same. The same is done for past tense. You know, Mandarin Chinese is not a heavily inflected language. It tends to use things that bridge between Diacritics and standalone words to communicate tense or to communicate conjugation.

Okay, so what is that? What is the part of speech of the word that is the thing that questionizes the sentence.

So look, I don't know off the top of my head.

And you're going to have to add questionize to the spelling dictionary now.

So.

By the way, we include, for example, the question mark, as in world frequency data. So that includes characters, too.

Really?

Right.

And they are classified as part of a speech punctuation, by the way.

So that's.

It's an interrogative word, a word that indicates a question.

Yeah, I guess it's a particle, but.

Yeah, you can't do that. You're not allowed to do that. You know, the set of parts of speech is a fixed set of parts of speech. An interrogative word, interrogative isn't one of them in the ordinary.

It's because words like who, what, where, when, why, and how indicate a question, but they also have a part of speech of their own. Right. A lot of them are like, who is pronoun, I believe. And, you know, so wait a second, wait a second.

You're saying that if I ask for the word data for who, you're saying, what parts of speech? Maybe somebody can do that and I cannot.

So what I'm saying is that these words have sort of two different. They are both. They're part of speech, but they also indicate a question. And that question indicator is separate from the part of speech it is. A question mark only has a question indicator. It doesn't have that other thing. And I can't remember off the top of my head, like, what, I don't know what. What all the technical terms are.

According to the LLM interrogative pronoun is, is the term. So it's a. Still a pronoun because it is, in fact a pronoun.

In English, though.

In English, right. In English, it is a pronoun because it is a. It is a placeholder for a noun.

Yeah, well, that's kind of what I'm saying is that it's almost like it is a placeholder for a noun, and additionally it indicates a question, but it's part of speech. It's a pronoun. Yeah, a question mark indicates a question, but it doesn't have that additional thing that words like who have that. It doesn't have the same, like, grammatical function in a sentence. It just indicates a question. It isn't additionally a pronoun or a noun or a verb or Something I forget the technical term for what that makes it.

Let's see. Dimitri on our live stream suggests draft Dra. Ught. And draft. Draft as in beer versus horses. Oh, yeah, you're right. To draft as in a draft in American is just D R A F T. Right. A draft. How do you spell a draft? Beer in American. D R A F, T. Okay, then we've got an example here. The word sense breaks differently into British and American. In British, you know, the spellings work differently. The. So there's an example. Good example. Thank you. Okay. Sander is commenting on here. Is. Is it the word order that makes it a question also? I don't know. In some languages, I suspect. Yeah, yeah. Word order is. Is something. Yeah. I mean, we're not. We're not. We're not accounting for that. Again, again, we are deeply using, you know, we have to have an assumption of compositionality of some kind. Otherwise we can't. Otherwise there's no meaning, you know, to discussing language at the level of words, so to speak. So, I mean. Okay, all right, let's keep going here.

What's this, this part kind of. We had some of these third argument things in word data historically that would return like the. The de duplicated, flattened list of values for all of the things.

I don't think those were a big success.

Okay. I mean, I'm just. I'm just wondering, like, is there. Is there utility in having some property or option of something somewhere? Because when you're working with word data, I could imagine there being cases where you. I mean, you can just always throw common name at things and get the words out. I just don't know if there's sort of value in working with word stuff where sometimes you're really not going to want to have all the yellow boxes. You're just going to want strings. You don't need the additional.

I get it. Yeah. I mean, I was just doing some stuff with bio sequences yesterday and it was a little bit frustrating having to say sequence string all the time. Right. And in fact, that's the thing we should pin. If somebody can take that to the next bio sequence discussion. I want to bring that up. Maybe a common name thing. I mean, maybe common name just does that for all of these things. Just it's.

I mean, that will work on words too, if you. If you want to get to the words. It's easy enough to just map that over everything.

Well, so long as, you know, we have this kind of big, big thing, somewhat like normal, that is just that, that Seems like a potentially reasonable way to do it.

Right. And this is just. This is just coming back around to the same discussion we were having before. Sort of what. What do we want to do about representing every different form of everything? And I think you're saying we want to go in the direction of having.

Symbolic wrapper. Yeah, symbolic wrapper that has. For every word, just like we've done in text structure. We have this kind of thing that. That indicates the part of speech and so on. So I think we want to generalize that and just have some, you know, text element like thing that is a word that is a carrier of lots of additional annotation data. Okay. And then just load it up on, you know, because we could load it up and it could say things like, this is in this language. This is with this locale. This is this, this, whatever. It can have all kinds of information. Because, I mean, for example, I don't know too many examples of this, but there are. There are. Here, I'm gonna ask it my friendly LLM. Give me a word that appears in English, French, German and Swedish with the same spelling. That is not very.

Okay, well, I think one example of a warning in every language is taxi.

Right.

Or T is a common example. There's only two different words for T. They just get transmuted into.

And you mean tea.

Tea or chai are the two root words for tea in all languages.

Okay.

But they might have a different spelling, right?

Yeah, and they might have different transliteration, too.

I'm just going to make it more difficult for the LLM. That's kind of a lame answer, but amusing. Definitely amusing. Okay, but anyway, so my point is, each one of these things, presumably our entities are English word. Know. Fish, as in search is the English word that presumably has to be recorded in the entity. Right? Yes, makes sense.

Think so.

But it shouldn't appear presumably. Well, you know, somewhat obnoxiously, perhaps. If it's not English, it should appear because our whole system is based on English. Like, if this is the word. No, you silly. LLM. You completely didn't understand what I talked about. No. Okay. LLM. Not a lot of points there. Okay. We're gonna have to tag it somehow, right? Yeah, I don't know how that's going to work in terms of the. I mean, look, the fact is in. In the case of a place, we'll say Springfield, Massachusetts, United States. And we. In this case, we are. We could allied. We could say it is fish. You know, word. Whatever it is. Fish. Well, adjective or something. You Know English and we allied off and that that's its canonical form is that entity with those list of things in it. And we allied out the English part when we are treating it, you know, when we're yellow boxing it and so on. Does that make any sense?

Not sure. Just trying to think about best way to handle it.

Okay, all right. Okay, let's have a quick look on for some of these other things.

Sure. We can also.

This is very again and yeah, we will need to continue this but, but I mean the fact is that I think it's really, you know, we are doing this. You know, the fact that we delayed this for a decade is frustrating, but it has the silver lining that we get to use LLMs now, which I really think is quite transformative in terms of making a lot of things that were, you know, very painful curation tasks now become rather straightforward.

I think we still have Christian, right. I think this is, this is his domain here. If you want to talk about what the changes are.

Oh yeah, sure. So basically the difference with this, with the previous classification part, the one that's now in production, it's. I mean you could say that the classification tree, it's not only looks more nice because we have a go for this time, but now you can see the classification tree for a single entity. For example, if you just ask in this case for French, previously you would only see a classification tree. If you ask for more than two entities, you would see the relationship between them, which. It's still something.

Okay, I get it, I get it. It's nice. By the way, just a small point. I mean in terms of visual appearance, this thing should be smaller. If there are so few entities, so few things in it. You don't need this vast space for that tentacle there. Small point. Do you understand what I'm saying? I mean, I think there's some auto sizing for graphs and it's probably being ignored for purposes of this rendering. I also think that it's confusing to have. I mean I know that this is pink tier for some reason I don't understand but. But I don't know, maybe this is not a realistic color coding, but I'm just. I think that using the yellow, which is the traditional yellow box thing, the, for these language groups might not be the right thing.

Would you want to reverse it or have a different scheme?

I don't know. I mean, just have a scheme. I mean, I just. It's unclear to me. Let me just take a look at what happens here.

Yeah, there is like for Example, the case that. Because it does. The case in which two languages have not the same root family.

Well, then you've got to say that. You got to say that explicitly. You can't just show this, that, that. Look, it's not clear that we didn't just put up two results here. I mean, in, in. In the case of. Can we see one where they are related?

Sure.

They. Yeah. Okay. And why is this going down the page and the other ones are going across the page?

It's mostly because this. The other ones, the. You could say the base trees, you just have like the first two top levels, right? Like the parent and you could say the grandparent. In this case, yeah, you could. You go up to root family, which.

I understand, but you can't have it be inconsistent. I mean, you could. You could draw this tree and you can render it nicely so it's a little tiny tree, but it just goes, you know, down two levels. You shouldn't have.

Also, the thing is that with these trees, the basic trees, we also show the siblings, like when. In these cases they have like one ceiling.

Right.

But there are some cases in which you have a lot of siblings. And when you put it vertically, it looks crowded. And you can.

How many. How worthwhile is it to show 50 siblings? Or should we just have like we do in the tree rendering system now, in the modern tree rendering system, we have a cutoff in the number of siblings and we just have a dot, dot, dot, and we say how many are missing?

I would say we have been trying originally because the original graph was vertically. I would say we could handle like three or four siblings.

Fine. But then why don't we do three or four siblings with dot, dot, dot? I mean, tree now has a specific way of doing this, which is pretty nice of eliding siblings when there are too many. Why don't we use the same thing? If possible, just use tree. I don't know whether we're in a position to do that yet, but use something functionally equivalent to tree. So I'm saying I just think it's going to be consistent.

So you would rather have all them vertical, Right?

That's what I'm saying. I think it's got to be consistent. It's super confusing if some of them are horizontal and some are vertical. And if we have to lose, you know, in these cases, we're not explicitly showing the siblings here, which I think is fine. Am I making sense? Right. You understand what I'm saying? Do you understand or.

Yes.

No.

Yes.

Yes.

So.

So the Idea is that we will have something like this also for the individual cases.

Right, that's what I'm saying. And if you want to display the siblings explicitly, do it up to some number of siblings and then just do these stubs. And if you get too many stubs, then do a dot, dot, dots with a number there. Make sense? Okay, can we get a yes, a no? You don't agree? You do agree. Please. So we can move on.

Makes sense.

Okay, fine. There's nice data here. Okay, do we have this data yet? Like order and things?

Yeah, so we have some data already. I think they're the other. So we have these structures and we're also looking on maybe extracting some more from walls.

Okay, but now wait a second. This here, This seems crazy. What. What's being actually said here in detail because you're repeating things. You know, if you say, you know, you just say. I don't understand why you need a left hand side here. The left hand side doesn't seem useful. So I'm saying because you're always going to show a subject of urban object, a subject to negation of urban object. Am I making sense? So you don't really need the left hand side which is just going to repeat the right hand side. Am I making sense? Do you understand what I'm saying?

Yeah, yeah, sure. So you would say like only present the values.

The values in this case I would say. And I don't think plus is the right thing because plus is a commutative operator and you're specifically trying to show ordering here. Okay, so I don't know exactly what the right thing is, but it isn't plus. This needs more review that there's. I mean there's a lot of great stuff here, but I think it needs more of you. What is an inventory size? Given that it's. Is this. What does that mean? What's that number for English? What does it mean? Inventory size mean you have a certain number of consonants. Are you saying that there are six consonants which exist but are rare or what?

Christian, do we have the list of definitions actually functioning on current. Do we have a definition for all of the terms from walls?

I am not sure actually about it. We could try it.

We have the definition. Can you click the link? Stephen? I mean this will work on the. On the. Not. Not there. The link to the site.

I know I did do it for English, which I actually know. Is anything happening or is it dead? Not encouraging so far, but I mean in this case, for example, vowels, you know, A, E, I, O, U in English. But why sometimes is that what this means? Hey, guys, come on, you got to say something. I can't deal with a meeting where you don't say anything. Okay, if you're not going to say anything, we got to move on.

Usually, are we. Sorry. Usually things like that are either for very rare consonants because it. It's less referring to the letters and more the actual sounds. So either they're sounds that only exist in some dialects.

Okay.

Or there are sounds that are perhaps slowly disappearing from the language be due to accent change and things like that. There can be a handful of reasons, but it's generally that there is a consonant that appears in this language, but not in every version of this language due to pronunciation reasons.

Fair enough. But then we need to clarify that, because I'm guessing here, but for example.

In Spanish, you used to have like.

A, B, C. And then there's ch.

Which is a che in. In Spanish.

And then the.

For example, the double L used to spend llama. It's only. It used to be only one consonant. So I'm guessing that's probably the difference.

I see. Well, but that needs to be clarified. And then, then there should be a list of the actual things. Right.

Right.

That makes sense.

Well, there's a lot of interesting stuff here. Okay, can I just couple to take for one second. Let's look at two. Two topics, and then we got a wrap here. One topic is the. There's the harmonization of wordless, dictionary lookup, etc. Do we ultimately think so? Some of those have inflections, some do not. Those kinds of things. What is our plan with respect to that? Do we expect to harmonize those things, which will be nice, or do we want to give a definition? I mean, like, word list should be capable of giving whatever dictionary lookup can use. Am I making sense?

Yeah. I think the case for dictionary lookup and word list should basically have complete overlap with each other. The differences that exist are basically bugs or semantic problems. I think the bigger question is how or whether we want to unify with the spelling dictionaries.

Hold on, hold on. Wordlist does not have inflected forms by default.

It does not include them by default. You can include them with an option.

Yeah, okay, fine, fine. And then with an option, it will be equivalent to dictionary lookup.

Yes. I mean, dictionary lookup basically just has a static list of words that are exploded with inflections.

No, I understand that. I understand completely, but I'm saying word list with inflected forms arrow true. Would give a list equivalent to the list which is being used by dictionary lookup. Correct?

Yes, yes. When we unify them, that's what.

Okay, fine, I understand. So that's the goal there. Okay. Now, the other case you were talking about was the spelling dictionary, right.

We have a dictionary word queue, which unfortunately has a very similar name to dictionary lookup, which is looking at the spelling dictionaries, the same ones.

Oh, I had no idea those were using different dictionaries. Yeah.

So spelling correction list and dictionary word queue both pull from the spelling dictionaries and not from the same resources as word list or dictionary lookup.

Okay.

And part of that is motivated by the fact that the spelling dictionaries need a different structure of dictionary. There is an ongoing question as to whether we can use the spelling dictionaries as a baseline and derive and fill in everything that's needed for word list and dictionary lookup from them.

Because let me. Let me comment that the world has changed here, okay? In the past, if we needed to give a definition for a new word, it was a big pain in the neck. In modern times, it's rather easy, right? Because we just use an LLM. So the fact that we may have some additional words that are like modern words that are in the. That we've added to the spelling dictionary that we did not add to the word list seems like, to me, like we should try to. We should look at what they are first of all, and then we should try to add them to the word list, to word list and to all the word properties, knowing that we will have to fill in definitions, inflections, whatever, using LLMs. Am I making sense?

Yeah.

Yep.

But we can take a look at the spelling dictionaries and get a sense of the.

No, I mean, you know, we may have added our trade names and things into the spelling dictionary, and we may want to exclude some words from the full treatment. I mean, there may be proper names in the spelling dictionary that we don't want to include those kinds of things.

Right.

Okay, let's see. The other thing I just want to cover for one second here is the phonetic data stuff. Okay? Look, How many phonemes are there in the world? Like a hundred or something that are traditionally identified. We should absolutely have an entity, you know, a, an entity type for phonemes, which are, if I'm not mistaken, I mean, that, you know, like you say the voiced pharyngeal fricative and things. Well, that doesn't. That doesn't help us. But anyway, whatever. If we get rid of the parenthesis there we might see a fricative. What the heck is that? That's an IPA representation. Okay. So they're IPA numbers, which are. Presumably there's a list of IPA numbers. Right. And then the question of which languages do these. Right. So the first thing we can absolutely do is. Well, there are more than. There may not be more. This is more like American house numbers. But not everything's been allocated. So, you know, whatever it is, 100, whatever they are, you know, labiodental approximant or whatever, you know, there's. There's no reason not to have an entity type which is all of these ipa, whatever they're called. IPA phonemes, basically, right? Yes.

Probably aren't some of these. Aren't some of these like, via critics.

Right.

I think that was the first issue when we were trying to come up with a name for the entity type.

Well, these are IPA legacy coding system. I see. Well, all right. Three digit numerical code. Okay, well, fine. So then let's use it, You know, phonetic entity. We can call them, if nothing else. If we, if we can find nothing else to call them. All right.

And subtype things as whether they're a diacritic.

So diacritic or.

Yeah, yeah.

Okay. Leaky voice. That's weird. The heck is that? That's an official designation?

Yes, it is.

What is it like vocal fry? Is it that kind of a thing?

Or it's like, I'm very creaky. Is that what it is?

I think so. I think there are some languages that use that as part of some kind of distinction.

You know? Yeah. Okay. So, I mean, this seems like a no brainer to be able to put this in. I thought we'd done this like 15 years ago or something.

I mean, I think there's a version in dev mode. Right, Alan?

There's. Somebody had done a rough scrape in Warsa, but I think that the main question was we're running up against the. What do we actually want to call it? Like phonetic entity seems generic enough. If we're okay with just having flattened out the consonants. Vowels, diacritic, super segmentals. Each, Each individual thing is its own thing. And we.

And then an IPA can say something.

About each of them.

Right.

And.

And the. They have a Unicode representation of these IPA things, so if we see that, we can map it into the spinet. Ma live stream is commenting about historical origins of words to pull out. Not quite sure I understand this. I mean, we, we. We have attempted. And that's another thing we should be able to get from the LLMs with some degree of accuracy is things about etymology of words.

I. I might have trust issues on that front, but we can try.

It might make it up. Right. What is the origin of the word apical? Since I see it on in. What is the origins of. Well, it's just an LLM. I have no pride with these things. Okay. I'm gonna. I have to ask this. Because this is a word that we have appropriated for a new meaning. Very conveniently, it's not corrected by spelling D correctors. Okay. Who knows? Well, okay. I mean, at least. You know, one thing that we can at least do in a case like this, we can get it to give more structured output. And at the very least, the question is, is its ability to such that it will come up with a word. Yeah, it probably will in that language. I mean, honestly, it's going to do the same thing that etymologists are going to do. It's going to say, this word looks like this other word. They must be related. Anyway. Okay, well, we should wrap up here for now.

Yeah. We've got a few details on the word side and other places. We can do a few more experiments with fleshing out some of these things and talk about enumerating all the different inflection types and everything else. Come back and talk about building on that design.

Right. A lot of very nice stuff. This is cool. And this is the right time to be doing this.

Yeah.

Okay. Thanks a lot, guys. Thanks to folks on the live stream. See you all soon.

Thank you.