Live CEOing Ep 734: Language Design in the Wolfram Language [Strings and More]

Sam. All right. Hello everyone. Welcome to another Wolfram Language design review, part of our Incremental Language Development series. And we have a variety of topics today. First one, strings. Strings. Strings. Strings. Predicates.

Yeah, so we're gonna be talking about a handful of predicates here. So I just took the liberty of listing all of our existing string predicates just for reference under the first subitem or fairly standard complement string format. Queue is kind of an outlier there.

What is, is, Is it, does this string meet this format?

It is, does this string meet this import export format? Yeah, it's not, it's not a normal string pattern. It's specifically tied into the import export format.

Okay.

So string patternqueue is actually a function that has been implemented about six different times in different private contexts in the system. And it, it's primarily useful because there's a bunch of different things that can be string formats. We can't just test ahead and say, is it a string or is it a, a pattern? It might also be a regular expression or, or some combination thereof.

Yeah, but, but that, you can't always just put a regular expression in places where you put a string pattern. And like if I have a string pattern of the form, you know, X colon, digit character, right? Dot tilde, tilde, blah blah, blah, blah, blah. That's different from, you know, the regex form.

Right.

I mean, all, all of the string patterns ultimately do have a regular expression representation, but sure, they do as regular.

Expression of blah, blah, blah.

Right.

Well, I mean, if we're going to have this, it needs a second argument which determines what type of string pattern it's asking for. Am I making sense?

Yeah, I think that's a good qualification.

So whether it's, I mean, by default it might allow a star or it might allow, you know, because the analog of that would be, you know, quotes A. I don't know whether star is anything. Yeah, star is anything. So it's this. Right?

Yep.

Those are alternate representations of the same thing.

Right, but that doesn't work. There are functions for which that doesn't work. I mean, string replace, you know, doesn't. Won't take that. Right?

Yeah, because it's specifically looking for a pattern based version rather than wild card. So. Yeah, that's a good point. That there needs to be some qualifications for string patterns in some cases.

Do we have an option that means allow these kinds of wild cards? We do, don't we? For some functions. It's a very old option.

I believe you're right. I don't know the option name off the top of my head, though.

Well, let's find out. I mean, I think there's a.

What?

Anybody know what that is? It's in maybe information or something has that. I'm not sure. Can we look that up? Because we could have it be an option or we can have it be a second argument here. I think it needs something.

Yeah, I'll. I'll follow up on that. Specific implementation. So the next two are closely related. String disjoint queue and string intersecting Q.

I didn't know that that was character intersection. I mean, this seems.

Yeah, this is specifically characters. I think there. There's a little bit of a nuance difference between the. The character forms versus the string forms, I think, in general.

So I thought this meant do these strings overlap? You see what I'm saying?

Yeah. So, I mean, there's. Yeah, there's the version of do these contiguous elements overlap versus do the component characters in them overlap? You know, I think the. I think we actually have Arnoud here, but these were originally versions that Arnoud proposed, and they're trying to map to what intersecting Q and disjoint Q do, which is looking at individual elements, but.

I know, but I don't think this is. Go ahead.

Well, I was just gonna say, I mean, what is the use case for these? And. And why is it, you know, sort of so compelling that. That you can't just run, you know, intersecting queue of characters? Yeah.

I'm not buying these, but I do think something I was trying to do recently with sequence alignment is something where you have two strings and you're trying to figure out whether you can line them up, you know, at the beginning or the end. So I'm saying.

Yeah, whether the specific sequence of characters have an intersection.

Yeah, I mean, if I do this and then I have another string that's, you know, X C C C ba ba ba ba ba. I want to. I want to align those strings. That's something that I wanted to do a bunch of times. And I mean, it's not too hard to write the code, but there's a.

Function, the function repository with your name on it, related to that.

How funny.

Actually, maybe a more general version of this that, like, I've wanted for some LLM things is you give it a string and a pattern, and the question is, could this string be the prefix of a string that matches that pattern? So, for example, you might have a pattern like, oh, I want any number of. I just want a bunch of letters followed By a dot. Right. So then if I have a bunch of letters that doesn't match that string pattern, but it could be the beginning of a string that matches that pattern.

Is there not a wrapper for the, for the pattern, for the string pattern that can ask that?

Not that I know of. I don't think so, but I could be wrong.

Okay, but let's see, what is this doing? This is asking, okay, this is asking for cases where the strings can overlap. This would be a more low level string overlaps queue. Right. I mean this is in any order, but this is just asking, can this string, you know, what is the unified version of the string with the overlap implemented? And you know how much overlap is there? Right. This is, this is useful. I mean it's useful for bioinformatics stuff. It's useful for these kinds of things that, you know, for strongly writing stuff. Did I use this for. Let me understand what Christopher is asking.

Right? I think this came out early in the physics project.

Yeah, Makes sense. That was about that time. Okay, so what was Christopher asking for in more detail? You want to know whether a particular pattern, you know, this like string prefix Q or something. We want possibility.

Right? Is this, well, is this string a prefix of this pattern? Kind of.

Ish.

The use case to be clear that I ran into was you're generating tokens from an LLM and you want to know do the tokens so far, could these match some pattern? So for example, if I only want to generate text that matches a certain string pattern, there's no point in continuing if already it's generated a token that couldn't possibly match the pattern.

Yes.

But, but maybe the right way to do it. I mean, one way to do it would be to have a different string match queue, string prefix queue function. Another way would be if we had some pattern construct for it. I mean, this is maybe much more complicated to do. But you have a wrapper for the string pattern that says match any string that's a prefix of this pattern.

Okay, but we already have things like start of string, which is a pattern thing. Match any string which is a prefix of this pattern. Okay, so wait a minute. Match a string that is a prefix of, blah, blah, blah. So what would that be for a list? What is the pattern that is a prefix of. So that means. That's a prefix of. I'm confused by that.

Another way you could think about it is there is a possible completion to this string that makes it match the pattern.

It's a potential match for the pattern, but not A complete match.

Right, Right. But there, There exists another string that you can append to the end of this one that makes it match the pattern.

Right, but this is. Okay, so go ahead.

I was gonna say, if. So, example, you're looking for XYZ in a pattern and you have X, Y. It is potentially a match, but it's unfinished. Right. Is that the idea with string?

Right, but if you had xx, then it couldn't possibly match.

Right? Yeah, so, yeah, rather than finding.

So let's understand. Match a string that is a prefix of. Is a prefix of. So you're asking for something which is the. The. Let's see, like a string prefix pattern or something that's not really right. In other words, that matches any string. Wait a minute, that is a prefix of means what? Means that the string that you are matching with this. I'm very careful.

No, I'm not sure the word prefix is quite right.

Yeah, it's like an incomplete pattern or potentially incomplete. The.

The pattern.

Yeah. If that's the pattern you're looking for.

As a prefix or what's that?

It's a string that has this as a prefix. Is that. Is that it?

I think the idea is this is the pattern I'm looking for. X, Y, Z. I want an input that is X Y to return true, but an input that is XX to return false. It is a partial pattern match or a potential. An incomplete pattern match.

Maybe the string XX to do that.

Is this not trivial to do with string starts Q and like a pattern test?

No, because you're asking. The thing only matches the beginning. So in other words, as far as it goes, it matches this.

Right.

String starts Q is kind of the inverse thing. It's saying that. What is it? It's basically saying that this pattern matches the beginning of the string as opposed to the string matches the beginning of the pattern.

Ian is commenting here that this will be the case that string starts Q. String starts Q. I'm very confused here.

Well, I mean, string match Q of string, comma, string prefix, pattern of string would also return true.

Yes, that's right. Not the starts part.

Right, I see. I mean, the start would work as well, but.

Yeah, yes, but let me just understand what this is. So this means. Why am I being so confused here? That this is. This means maybe try.

Try an example that's more patterny. So if you write something like, you know, letter, character, dot, dot, squiggle, you.

Know.

Quotes, period or something. Right. So it's basically any series of letters followed by a period or semicolon so if you wrote string prefix pattern of that, then this pattern would match a ab, abq, ABQ semicolon, but it would not match a one or something like that. So it would match anything that you could complete to form a string that matches the pattern.

So. But this is the wrong name for that construct.

So does this.

Yeah.

Does this take only string literals or both string literals and string patterns?

It takes a string pattern here. This represents.

I was thinking it was a pattern that matches the prefixes of a specific string.

No, it's this, for example, matches anything. This is the thing which. What is the right name for this? This is something where you want it to match this. My God, what should this be?

I mean, it's maybe a little bit like. I mean it's a little bit like up to. I mean, maybe it could be something like, well, prefix of pattern or something. It's a little obscure. But the point being that it matches anything that's a prefix of this.

Right.

I guess I keep circling back to the idea that it's a potentially incomplete pattern. You know, like I say, another example of an application would be I have some definite genetic sequence that I'm looking for, but I have a whole bunch of data that has incomplete genetic sequences. If they're all starting points, I can filter out and say, here's a 30 chromosome pair or 30 chromosome sequence that I'm looking for. I want to discard everything that just doesn't even begin the right way.

Yes, but the only thing is about the word complete is I think complete kind of, I don't know, like it sort of implies that, like it should be complete or something that like this is a. It's sort of like some weird value judgment on the string or something. I don't know.

Look, I want to come back to the case that I found important, which is this question of whether two strings can, you know, this is what you need to reassemble a shotgun sequencing thing in bioinformatics, for example. Right. Do these things. Can you puzzle piece match these things together? See what I'm saying?

That's a related one. Maybe you could implement that. I'm thinking of whether this would actually quite give you that. You might need to do some. Either have a string ends queue or maybe you can just emulate that with string reverse or something.

Well, I think the useful thing would be given these sequence of strings.

Isn'T there already a function that does that longest common sequence? Or are you talking about something else?

The, the strings are potentially disjoint.

Yeah, but the longest common sequence doesn't do that because it's not an endtoend thing. Right.

There's also longest common subsequence.

I know, but that's not now a long.

And I mean, if you just want to do this joining, right, you have the explicit strings, then I think you can probably use longest common sequence.

But does longest common sequence only work at the two ends or doesn't it work?

It works anywhere.

Yeah, you see, that's not right. We need a thing that works only at the ends.

Right. But it's not too hard to. Then you can call longest common sequence and then check whether that sequence is the other way.

That's incredible waste.

Right, but, but in any case, I, I, I think that the, I mean, the reason why I think the pattern thing is more interesting is just that it, it's, this is something that's a pure, like you don't need string patterns to do this shotgun sequencing problem, right? Like these, it doesn't matter if they're lists or, or strings or whatever.

I know your algorithmic problem. Right. But I mean, this, we need something that does this case for strings or lists. So that's one function we're going to need.

But, so yeah, go ahead, finish your thought and I'll come back to it.

Okay, but then, then separately, this thing about. I still haven't wrapped my brain around this particular thing. Go ahead. What were you going to say?

Yeah, so I was going to say there's potentially two cases we're looking for with the longest overlaps. You know, are we looking for concatenate these strings together with overlaps if they're there? Or are we looking for, you know, any cases where these potentially overlap. Is, is this sort of a cases instance or are we looking specifically for joined strings to come out of it in successful cases?

I think we want both. I think we want, look, the cases, the thing we want is, you know, join the strings by their overlaps. You know, we want, probably just give me the longest overlap and I can do string length if I want to know how long it is.

Mm.

Or length.

Okay.

And I would argue that what we want here is the longest overlap that goes, you know, string one, string two. It's very clear what that means. If we had string one, string, choose string three. You know, there's clearly a longest overlap between string one and string two, string two and string three. How do we return that?

So I guess the question is what you do when they're entirely disjoint. Is the longest Overlap in that case is just null.

It's just null, null, string. But I mean, it's unfortunate.

Right? You want a string in the first case, and in the other case you basically need to have a list.

Which is unfortunate.

But why don't you want to return a single string again, joining a string three to the joining of one and string two?

I think in this case we're looking specifically for what the overlap sequence is rather than the joined string.

Oh, I see, I see.

Okay.

Yeah. I think in the, in the vararg's case of the join these strings together, you would just have the strings get joined together with whatever overlaps laying on top of each other. And if they're disjoint, I guess they'd just be joined end to end.

I mean, string join overlapped or something. But that isn't the right name. And also, these could be both lists and strings.

Right.

Although I have to say I think the string case is more useful for the overlapping thing.

Maybe there's some use case with byte sequences on files or something where you have fragments.

That's true too. I mean, this, this business about reassembling things that have been fragmented is definitely a thing that comes up. And I can even imagine that you might want in some cases to do it, even when there is some, you know, edit distance that you're not getting. Right. I really think this is a good thing.

Okay.

Yeah, I really think this is a good thing. Now I still haven't understood this thing. Should we, should we try and explore this thing further or is it. Is the. How worthwhile is this thing?

I think maybe I'll, I'll talk to Christopher outside of this a little bit and we'll try and get the, the case down a little bit.

But this really does seem useful.

Okay.

Again, I've, I've run into it multiple times. Must have a name in bioinformatics because it, you know, it is the thing you need to reassemble a shotgun sequenced, you know, DNA sequence or something. Makes sense.

Yep. Yeah, I'll, I'll check in with a couple of people and we'll circle back.

Okay.

So this last one isn't.

It's similar to the. Sorry, it's similar to the was image assemble that does this. If that helps with naming. Yes, that's kind of an analog.

And.

In fact, assembly is exactly what it's called in bioinformatics. That's not a bad name. String assemble. I like that name.

It's also. Well, I mean, there's also the analogous Most similar function I can think of is image align, but it might be confusing with sequence align, which does something kind of different.

Right, but what is image Image assemble?

No. Image align.

No, I know, but images symbol.

Just image assembles like string join.

Now I think, I think you're talking.

No, it doesn't do advanced usage of.

Image assemble that, does it? That you can specify.

Not intelligently, I don't think.

Oh, is it not intelligent?

Okay.

I think it just. Yeah, it's like string join.

Okay.

So that's an argument against this. Okay. I mean, it would be lovely to have a name that would allow this to work for strings on lists.

We'll let that one simmer a little bit and we'll circle back to it next time with a more concrete proposal.

Okay? Okay. All right.

String within Q. Yeah, so the name here isn't great. The actual function that's being proposed is looking for sub expressions that match a string or a string pattern.

Why isn't that a general pattern construct? I don't understand here.

Yeah, so, yeah, that's specifically. So the second argument is a string pattern and it's looking at all sub expressions within the first argument to see if anything contains.

I mean, that's amusing, but it's, it's a crazy name, right?

Yeah, I think, you know, contain string queue is closer, but it's not quite.

How far down does it.

Why, why does this. I mean, I, I just don't understand what the. I mean, is this not just like member queue with level stack infinity?

No, because. Not member queue, because it's. Look, basically what this is doing is it's doing a search, a string search.

Right.

If this had been written out as input form, this would be like a string search.

Right.

I don't even know what the heck it's supposed to return that. It's there. That's not all that useful.

Yeah, I mean, the, the cases I run into this are. You're doing something like you're scraping through XML or you're scraping through sort of web dump, and I've got a whole bunch of data and I need to pick out which, you know, which of these elements actually has the sequence I'm looking for. I've got a bunch of auto data that's dumped out and I'm trying to find the VIN numbers. I know what the VIN number patterns look like. They're a sequence of 13.

So you can use string cases.

Yeah, yeah, you can use string cases when it's a single large string. But if I've got a bunch of XML data that's being stored as XML elements. It's trickier. Right.

So the problem with that is that string cases doesn't work on a thing that isn't. I mean, I don't know whether if I say string cases of this, you know, a list of strings here and I say, you know, something like a dot, dot. Right. What's wrong with that? I see, I see. It's giving the same structure for a list. It's giving the same structure. The issue is that string cases doesn't have a form that rodentifies, to use our old terminology, like a burrowing rodent.

This is like adding a level spec to a string contains Q.

Yes, Right.

But it's also the same as just member queue of the expression comma string containsqueue of the pattern comma infinity. Well, look like you can just use memberqueue to make any function effectively identify free Q.

How do you do that in this case? Okay, so how does it work in this case?

Right. Can you write member queue or maybe not free queue, but. But let's try member queue of the expression.

This thing.

Yeah.

And then comma, underscore, question mark, parentheses, string contains Q of your pattern. And then comma infinity outside the string contains Q, like out in the member queue. You might have to do zero to infinity, but yeah.

Okay, great. How do I actually find the strings.

Position instead of member queue? And you know, we have a whole bunch of. I mean, if you want to know how many strings there are, you could use count. Right. Like I had sort of. We already have a bunch of functions for doing this kind of drilling down. If the only thing this adds is that it drills down and it also drills down into strings, it's not hard to put those together already.

Why doesn't that work?

Select doesn't have a level spec.

Right.

Yeah. The throwing their select is which is how many results to return. Yeah. There's no level spec. It's not clear what that would be, I think.

Yes, right. Because it's not clear that it returns cases. Yeah, but then I've got to have this construct here. Yeah.

Which is what you have in all the other, you know, in count as well and member queue as well position. So it's, it's very.

I wonder whether we should have a, A, a piece of sort of a name for this thing. Blank question mark, blah.

Pattern test is the name. Right?

Yeah, yeah.

So if I just say here, is there one argument?

Is there one argument pattern test. I don't actually know.

Oh, I don't know.

No, but there should be. There should be.

I mean, the real thing is just.

Yeah, the parentheses on the underscore question mark thing are always weird too. Right. So, yeah, this would be nice because it would avoid that binding issue.

We should introduce that one argument for pattern test.

And it would just be a.

Okay, blank question mark, blah, blah. Right. Just of test.

Yeah, that'll be real nice. I'll use that.

Okay.

Yeah, I'll talk with you.

Yes. My gosh.

The one thing that's.

After all these years, the one thing.

The string within Q, if it was a position thing, could do is it could tell you when it finds a string where in the string is the pattern. Right. Like string position does. That's like the last element.

Sorry, what? Sorry, what were you saying?

One, the one thing that we could add with a dedicated function in this conversation is you could say where within the string the pattern is like string position does.

And how would you return that as.

The last element in the position in the result of each position spec.

Honestly, I would argue that instead of returning as the last element, you would have to return it as a pair. Like here's the position and then here's the string position.

Look, the mixed expression position and string position we've never done and it seems fairly horrible.

It does, it does. Right, the format.

Yeah.

We could come up with a better formatting of the output.

I don't know.

I'm just trying to come up with a reason to have this.

Yeah, I mean, I had. I had not necessarily thought of the form that Christopher mentioned. And I think like we said, maybe touching up pattern test makes using patterns like that a lot easier.

We should have thought of that sometime the last 35 years. One argument for the pattern test. It's a good idea.

I think we discussed it in May of this year.

Really?

The mail archive link chat.

We should do it.

And also an optional third argument for level spec for select that I'd have.

To review if we. Okay, I think that one seems much more questionable. All right, let's continue here.

Since we're talking about string contains. Is it proposal in the WFR to have a string contains all?

No.

No.

Okay.

Why do we want this? I mean, it just seems like a very sort of puzzle solving, obscure thing. For lists contains all. I don't even know whether that's, you know, sorts are equivalent or unions are equivalent or what is this? Sort is equivalent or sorts are equal or unions are equal.

Believe its unions are equal.

Yeah, like contains all and all the containments. I think they're all just, Just synonyms for set operations like subset Q and so on.

Yes, that's correct.

I don't get this. I don't get the point of.

Yeah, I mean, I guess again, this, you know, this is a fairly typical use case.

String contains all. I don't understand what is the operator form of it. Yeah, I realize that.

I mean, so I think it's testing that the string contains both naturalist and inhabitants.

Correct.

I think that's what's happening.

Yes, it is providing a list of substrings that need to be inside the. The string you're looking at.

Why isn't string contains doing this.

Is.

Because string contains. You give it a single pattern.

Yes, it's looking for a single pattern. String contains all would potentially take multiple patterns.

No, this is similar to all true.

Look, look, look, look. It's this case.

Any of the patterns. Right?

That's any. But not all.

Right?

Yeah, but you could do this with just. I mean, if you run string contains Q n times, maybe that's inefficient. I mean, I'm looking. This is from Jason Biggs. So I'm guessing this is a. There's some. If you have a, you know, a genome sequence or something and you want to know does this.

That Jason does genome sequences. I think that one.

Well, but, okay, but the thing you could imagine, I think you can imagine is I've got this genetic sequence and I've got these, these three sequences that correspond to some genes or something like that. And I want to know, do all three of these genes appear? But it seems like I should just run string contains Q three times.

Right. Well, you can also run, you know, it would be all true. Look, if this was going to exist, it would be string contains all Q. Right. That will be the correct. Would it not be that? Because it's the. It's the analog of string contains Q, but it's where we're forcing all of these. The alternative will be a wrapper around this, this thing here, right?

Like and.

Or all true. What's the analog for patterns of all true?

We have a contains all.

Right.

Yeah. This is specifically an analog of the contains all function.

We've got string contains Q, but yet we have contains all as a non Q function. Is that correct?

Right. The contains family of functions are not Q functions because they have operator forms, I believe.

But so does string contains. Doesn't has an operator form as well.

Okay. I. I don't remember the genesis of the contains family well enough to say exactly why they're not Q predicates.

Well, I mean, look, string contains all. Q is okay, except that it's annoying that it doesn't merge correctly with contains all. See what I'm saying?

Right. I mean, we don't have headdresses anywhere else that I'm aware of where we have a implicit or for. You know, we were talking about having a decorator for the second argument of string contains Q that would say. Yeah, this is an and. And I can't think of any other instance where we have an implicit or and some sort of headdress that changes it to an and. Right.

It's the opposite in text search.

Right.

What is the address in text search?

Well, list means and. And alternatives means or.

Which is what we should have done with string contains Q.

They would.

We don't have. What's that?

They. They would at least be aligned in that case.

Yeah.

Although I do actually think the or case is more common here. Right.

Generally when you're giving a list of patterns, they're probably alternatives, though you can give a single pattern with alternatives would be the counterpoint.

Isn't there a general pattern construct that means all of these must be in here? Like, for example, we don't have a pattern combination that this thing matches both, you know, because we could have something that says it matches both, blah, blah. Or let's say. Let me. Let me do a more extreme case. Right. We don't have that construct. Right.

So alternatives is just the or. But we don't have the and case. You can't.

We don't have the and.

We can't say and of two patterns.

I think that's come up. I think Nick had a case of. Of where he brought that up. You know, that seems like a more interesting thing. Yeah.

Can I ask.

This is the end of two patterns saying that the expression matches both of those patterns exactly. So that seems different from the contains case where you're giving two patterns and you want two different expressions to match, you know, each.

Yeah. Yes. Sandra on our live stream gives a piece of code here for the case that we're talking about. Right? That's correct.

What's your work?

Just fine.

But yeah, could be more efficient with a dedicated function.

Let's explore this question about a wrapper that means. Oh, I don't know. Seems crazy. I'm not sure. I do think this is worth looking at. Match this pattern and this pattern. But.

It should be called alternatives with two Ls.

I'm weakly positive about string contains all Q.

Okay, well, we'll we'll put that as a subject to revisit. When talking about a inclusive or an all inclusive pattern.

This looks bonkers.

Is it just switch, but instead of giving a pattern in the first argument.

You give a string pattern expression or first argument.

Yeah. So you're.

Or.

Right. I mean, it's just right. In every. Every odd numbered argument, you have a string pattern instead of an expression pattern. Correct.

Yeah, I think. I mean, I think the point is that switches are kind of a skew function anyway. I mean, a better form of this, which I just don't understand why this isn't. You know, this should be string replace of this thing here, comma, list, you.

Know, those as rules.

This R1, which looks much nicer to me. Is this not equivalent. What's that?

The. To the rule and the pattern. Yeah, yeah. You need parentheses.

Yeah.

Wait, you shouldn't. You can actually check that. No, no, the precedence is right.

No, you're right.

But that is a strange error. So string. String expression of two is just. Do you do squiggle squiggle. Like squiggle swiggle is the index restring expression.

Right?

Yeah, but this. That's really weird. Oh, oh, oh.

Well, I'm not sure where the string expression. Maybe it's because it's. It's replacing.

Yeah.

No, no, no, no, no. Because string.

The right hand side needs to be. Yeah. Right hand side needs to be a string and string replace list.

Yeah.

Right. So this is the problem. The problem is.

And it's not replacing the whole string. This is like replace. More like replace all kind of than it's like replace.

Yeah. So how do I do. How do I do a whole string replacement here? That's what. That's asking.

Maybe you just use replace. And then for the pattern you use, you know, underscore question mark, String match queue. Or in the future, pattern test of string match queue.

That's pretty bluffy.

Bring switch seems a lot nicer than that approach.

I don't think so. I think switch is a horrible, horrible mess with, you know, and even numbered things. I mean, something where you say.

It'S.

A very procedural construct, right?

Yeah, I just don't think it's a very good construct. I think this is a much better construct. But the question is for replace. I mean, if this was. If this was replaced and this first argument was, you know, characters of this. Right. And this. Is.

This.

First.

It's string split would do it instead of string replace.

What happens here? This is. Yeah, yeah, that works fine.

Mm.

You could also change string Replace to string split and put a first.

In.

Front and it should work.

Why does that work?

String split with rules as a second argument replaces Interesting. The left hand side with the right hand side. And it can be any expression because it doesn't try to join them again.

Yeah, I didn't realize that it didn't need to be a string.

Well, I think this functionality that it's a string pattern going to an arbitrary thing for the whole string is useful. The fact that it has to be back to a string is really kind of yucky.

I think we figured out why somebody decided to make a resource function.

I still honestly think though, this is just a very obscure thing to want to do. Yeah, I mean, you have to write a little bit more. More characters. You have to write a pattern test, but it's not that bad.

Well, the only thing I can imagine, I'm. I, I can see this showing up in a whole bunch of scraping like code.

Yeah. I was thinking there's scraping like code or some sort of prompt handling code. You know, if I get some input that has a question mark, I call a different function or a very.

What are we gonna do with this? I, I don't think this is, I mean, If we can think of a really great name for this thing that represents this kind of whole string replacement, perhaps there's a future for this.

Okay, well, also the use cases we just talked about, you want string contains instead of string match. You get what I mean?

Yeah.

So I.

Right.

I don't know. I, it's. It just seems like it's a really obscure.

All right. Thing.

I'm just not buying any of these strings set functions. Just not buying these. I just don't understand why you can't just say characters.

Yeah, I mean, in this, you know, the, these were a handful of functions we pulled. I think these were all our new functions. But originally, you know, the idea is basically you're pulling out, just as you see there, characters and then doing set operations. And maybe that's not really that interesting.

I don't see it. I mean, why can't you do it with complement, in this case? Complement of characters. I mean, is this wrong? If I, if I say complement of characters of this. Yeah, that's in a different order. What does it mean that it's in different order?

It's sorting them. Complements. Sorting them.

I know that, but I mean, string complement is not. For some reason. What's the argument for string complement not to sort them?

I probably should sort them, I guess. String union and intersection Resorting them.

So well then this then it's just a string join of this. Mm. I'm not buying it.

Perhaps there is a possibility that the contains family could do this itself. It currently doesn't do anything on two strings.

What do you mean?

So the contains family currently works on two lists or two associations. Lists and associations. I think we have some functions that work both on lists and on strings.

Oh absolutely, yeah.

Like.

Like those longest, longest subsequents.

All the dissimilar dissimilarity measures work on lists and strings.

So I'm just saying that the contains family could decide to apply characters to strings.

Okay, but what, what is the contains? What would the contains family do? So you're saying complements could do that, for example?

No, the container itself could. Yes, but I think complement feels more set related. Contains. My.

Like what contains function?

The 5 contains x contains all. Contains none contains. Exactly.

Well, I agree. I don't see any downside to doing that. Seems very sensible to me.

I put in the chat function Rick made in the function repository called string function that basically. Yeah, applies any set based function to characters. Right. It just does characters and at the end of the string join.

Right.

It's just a wrapper which you know, it is nice because we had a whole bunch of these come to function repository and this sort of stem the tide of having a dedicated function for each one.

Maybe this is the right answer. This is nice.

Yeah, like permutations is nice there. Other than having string permutations.

This is nice. I would use this a bunch of places. That's better because then then we could just say. Presumably we can say string function of contains all and then the arguments blah blah, blah, blah blah.

Yeah, or just a thought, what if you had an option for contains all or for these functions that was like make it for strings or something?

No, no.

Yeah. I don't think we want an option that changes what heads it operates on.

No, absolutely not.

Should the name be something more like characters applied?

Yes, it should be. Or string applied.

Right. And in the case where there's more than one argument, you know, each. Each function will have to have a little bit different logic potentially as opposed to what's pulled apart and what's not.

Right.

Okay, yeah.

I'll. I'll tear into what the thorny cases on that are.

String applied is interesting. It's really quite interesting. That's really quite nice. Okay, so that would in fact give us all of these string sort of. What is string prepared?

Basically just a special case of string join. Same with String append.

Why do we care? I understand. What's the difference between this and string join? Except it has an operator form.

Primarily there's an operator form. I mean the. I guess the same notion of why we have prepend and append and join.

But no, append and append take different list structures. Right. Because list a single element, they take single elements rather than lists.

Right.

I much prefer string applied. I think that's a much better thing.

So this is circling back to something we've talked about one or two times in the past, which is the idea of splitting a list before, after or at a specific case. The last time we went around it, we liked the idea of a split cases as a generalized ver version of it.

What is split cases?

The idea basically being that we give a pattern where we split it and we can pull out elements before, after or at the specific splitter rather than having a dedicated before, after and at.

Okay, so what do you mean, what was. How does it that work?

I mean, basically we're looking. Yeah. You know, with a split before, we basically are looking at a specific pattern and then we are segmenting the list or in this case.

So in other words, what you're saying is that if split, for example, I mean this ordinary string split. I don't know why I always type splot instead of split. I do it every damn time. I have not why. Fun word. Okay, so the issue is what happens to the separator in here? Okay, so why isn't split? I see split has a test rather than split at, which will be another function, but could have a split at. Right. All right, so you're saying that string split could have an assembly argument.

Yeah, Basically what do we do with the. The separator or the.

Look, you see, we got all these things. Right?

Because rule is a replacement and you.

I couldn't this just use the options that string template has. Insertion function and combination function.

I suppose.

Not sure if that accomplishes what we're. The main objective here.

What would be the main. What would be the point of insertion function?

I mean, yeah, it's. It's a little different than is in string template, right?

Yeah, it is.

I mean really what this is is a post splitting function. What you've got is. What do you want to do? You've got two elements, you've got the separator. Right. So you've got the separator. I suppose you've got the before string and you've got the after.

Right.

And for every split you've got those Three things which might. Which obviously.

Well, the separator is a pattern and not explicit. You also have the things that matched.

Well, I'm saying the separator is the raw separator, not. Not the input. It's the. It's what actually matched the separator.

So it's a list in general.

Why is the list. It's the separator. The separator each. For each element here, like this might be BBB dash dash, dash dash. That's the separator. This is a separator specific list. There's be a list.

Oh, I see. Okay. I see.

Okay.

So you have a list of these associations.

Yeah. For each splitting that happened before and after might also.

I was thinking it was the transpose of what you have here.

What were you saying about before and after?

I was saying before and after could be start of string or end of string, potentially for the first and last elements.

They could be. Yes. And by the way, I have no idea if you know, something like string join. I bet this doesn't work. Yeah, doesn't work. I mean, that could work.

I think it generally doesn't work on patterns. Right?

No, I agree, but. Yeah, but it could work for this case. All right, so in the split case, the question is, I mean, or is it the case that all that really happens is. What's the residue that gets left after splitting? How much of the separator is left after the splitting? Right.

You know, in. In data scraping, your delimiters are sometimes meaningless markup. They might be colons, sometimes they're meaningful. You might have records that are separated by dates. And so you might have a date pattern and say, I want to stick that with the element before it rather than just throwing it in the bin.

Yeah. Okay, so let's think about this for a second. So what other functions do we have? And I agree templates are an example. Maybe not much of an example. No, they're not really much of an example. What other functions do we have which pick up a thing that is essentially metadata or something like metadata. Yeah. So let me think about this for a second.

Form layout function, which I've still never figured out how to use.

Which one?

Form layout function for form object.

It'S simplest function.

It is not. I've never figured it out, but it takes some structured metadata and then gives a form object or a grid or something.

Yeah, I mean, look, date string has this kind of stuff as well. It has a way of specifying sort of a template for what gets put together.

Yeah.

Okay, hold on here, the main issue is this becomes more poignant when the separator is a pattern. When the specified separator is a pattern.

Right.

That's. That's the case that's particularly interesting. If it's just a repeated character, then you can basically just slap it on. But if it's a significant sequence or a meaningful pattern, then yeah.

Okay, hold on. I mean, the obvious function would be one where every other element is the separator. See what I'm saying?

The separators are basically riffled into the list.

Yep. Kind of like a version of string split.

That keeps the delimiters as extra elements.

Yeah.

String cleave.

Well. Or an argument structure for string split that makes it do that.

So it's a rule form that replaces it puts the given value at the delimiter.

Inserts VAL at the position of each delimiter. Well, it's essentially what we want. We just want. Pattern goes to itself.

Yeah, I've done that before.

X colon, blah, arrow X.

All right, but that doesn't get the before or after case where you then want to partition and string join the sub lists.

Oh, we can deal with that. I mean, that's a. There are a million different things you might want to do there. But if you want it, you know, the afters, you just do partition comma two. Otherwise you do partition two and then off with an offset of one. Right. That doesn't seem hard. Yeah. So really all this is. Is a string split. I mean, if I didn't think it was a. Yeah, really wants to wrap that rule. Because we don't want to say string split, string. Pattern, arrow, automatic or something. Or inherited. That seems too obscure. Yeah. Plus you might actually want to use those functions.

Yeah, you might want those literals in there.

Right. So we got to wrap the rule or we got to have it be something other than a rule.

And rule delayed works as well. So you can do.

Oh, yeah, there's no problem in doing this. Okay. X colon, pattern, arrow X. That will just work. Right. It's just rather bizarre. Now, another question is whether the default case of string split with one argument is a case where you might want to include the. Where you might want to include what was taken out? I think not. We might want to include the pieces of white space, so to speak.

Yeah, I feel like that's probably uncommon enough of a desire that having to use the two argument form with a white space character goes to. Yeah, whatever is probably.

Okay. So the question is, what could you wrap or put in that second argument that is visibly not. Because remember, split has a test there. So this is really a string split story.

We had like a keep head or something to wrap the pattern.

Yeah, but remember the. The second argument is usually a string pattern. There's no good. Well, all right, that's. The exercise is can we get.

Some meaningful syntax or some. Yeah, yeah.

I mean, it could be done with a triple rule, but I hate that. All right. Okay. Anything else here?

No, I think that's it for today. We've got a number of things to circle back to it another time.

Great. Okay. Just don't forget all the random things that are being sent in because there's some important once as I recall.

Yes, we have a running list of everything.

All right. Okay. See you guys later then.

Thanks all.

Bye.