Regular Expressions (Regex) with @nocodebackend

So one of the questions I often get, like in forums, also in StateChange, is about regular expressions, which is based on a very old language that is, that is, you know, still used today, often has some optimized parsing for performance purposes, but mostly is just super terse and flexible. And that language is called regular expressions. It's POSIX regular expressions. You can use them with grep, you can use them in all sorts of circumstances. They're very commonly used in both Python and PHP and, uh, and JavaScript and what have you. But, you know, you— and I'm seeing them more, I think, uh, because increasingly if people are doing sort of text manipulation problems and they go over to say ChatGPT, I would ask it to find all the instance of name that follows word hello, right? It will create for me a little bit of code that actually uses a regular expression. And this is a module called re from Python, and it's actually building it with a regular expression. So the fact that it provides this regular expression doesn't mean we have to go insert Python or go outside of Xano or whatever to go find ourselves Python solution. We can just do it. We can use regular expressions directly inside Xano, which can be more effective. Now let's talk about how a regular expression works for a second. You're gonna find the most regular expressions are bounded by these slashes, right? This is open and a close of a regular expression. And by the way, this site regexr.com is one of a few sites you can just Google up, learn regular expressions. That are all pretty fantastic for learning how to work with this technology. And the cool thing is you can pass in a little regular expression up here and it will show you how well it works, right? So in this case, it's showing us everything. And the meaning of this is it will look for everything that starts with a capital letter, right? And then all of the word characters, you know, alphanumeric, what have you, that follow it. And that way it can create, it basically is identifying what all of my capitalized words were. Now if I go back over to my Python over here, I can grab, you know, this little bit of business and go over to here and say I just want to paste in that expression. And then I'll take the text that they gave us, right? And I can put this into here. You can see how it's not actually about Python. You know, it will say it will find hello, uh, John and also find hello, uh, Sarah, which is, uh, which is super helpful, right? So the next thing we want to do, and you know, there are more complicated things you can do with regular expressions. I would actually recommend that like you use a combination of ChatGPT and like a playground like this to kind of dial in to make sure that whatever you're putting in and a little bit of sample text that it produces the results that you're really looking for here. But let's leave the question of how to dial in a particular regular expression problem to another time. That's a very deep subject. Let's talk about how we can bring it over to Xano, because Xano actually has a pretty good portfolio of what they call filters that use regular expressions. Now, to do that, we open up the, you know, go to the left-hand side here, go to Library, we click Functions, and we click Add Function. And here we're going to say regex test. RegEx is just a common abbreviation of the term regular expression, and why it's called regular is actually a little bit beyond me. I don't know. So let's go to the function stack, and we're just going to create a function just to— we'll do it all inline because that'll make it easier to demonstrate how this stuff works. And we're going to create a variable. We'll leave it as var1, don't care. And the important thing to understand about how regular expressions work We'll just add a filter here just to show. And if I just type REG, you find the regular expression ones. And they are, you know, allow you to get, you know, everything that matches, right? Like here, you can see there are multiple matches for a given criteria that I'd set up. In fact, I was asking about, gave me all the names, they got me all the names. The, oh, it can only just get the first match. So basically the same thing, but just limited to one. A Boolean that says either this matches your criteria or it does not, which can actually be super helpful if all you're looking for is whether it exists or not. This is kind of like, you know, queryAll for a string versus getRecord or singleRecord for a string versus hasRecord, you know, the Boolean of whether or not this is there or not. This is a utility function, it'll probably not come up as often. And then regexReplace actually gets used quite a bit to use regular expressions to find things and then replace them. It's a more advanced version of the replace filter. I will say about all of this that if you can relatively intuitively figure out how to do whatever your manipulation need is without using regular expressions, I recommend it because regular expressions are really hard to read and harder to understand. You're gonna find yourself kind of dialing 'em in, hacking 'em together using some ChatGPT, some regex, maybe somebody else is advising you on it, but that you're gonna look at them and they're gonna be just kind of Greek. So, you know, this is really for those advanced use cases that are harder to do just with ordinary, you know, text manipulation techniques that might be easier for you. So with all that said, let's talk about how to use regular expressions inside Xano. Now, the first thing you need to understand is that regular expressions use, you know, take the— we'll get back to the subject here in a second— take the regular expression itself as the value And then they have this string you're trying to search on is called the subject. So it's not going to be that you're passing it like, hey, here's a string I'm searching and then here's regex I want to apply to it. It's the other way around. Here's a regex I'm using and then the string that I want to apply to it. If you take a look over here, you'll see the expression is done first, followed by whatever the test is you're trying— whatever the string is you're trying to run on. Similarly in Xano, Here we would put in the regular expression, and you have to put in those slashes too. Those slashes are— XAML seems to be very sensitive to that. And then, oh wait, this is for the expression. So this actually would put that up here, which is the value. See, I get caught up in this too. Now you'll notice here that I end it with a slash, whereas here it is /g. g means that it is global for iterative searching for finding multiple multiple results. And here, if I were to take off global, you'll see it only finds the first one. So the global meaning is like, it's usually in regular expressions, if it's in find all and just find one, but that's not how XAML treats it. If I put a G in here, it won't work. I'll demonstrate that in just a second. And then what I want to search for is going to be whatever our string was that we had from ChatGPT, right? So I'm gonna grab that string, copy that, and then put that into here, right? So I have the same John and Sarah thing, and I say update. Okay, so now what I'm trying to do is apply this hello word thing to regex_get_all_matches. See how well it works. So this is the variable being set to that result. The response is gonna be the var1. So if I run and debug, this should give me the output of that. See, it found hello John, hello Sarah as one array, and it makes a second array of John, Sarah. So it has both the expansive version of it, the first one that includes a hello, and the version that does not. And in this case, you'll probably just want to use the second version of these two. Because when I did ChatGPT, you'll notice it gave me the result of John and Sarah, which is what we're going for. But in RegEx, you'll see it gave us the "Hello John," right? And the "Hello Sarah." This is what we call groups. And there are more things you can do to try and tweak the regular expression to make it not do that. But I'm going to suggest that you don't want to overcomplicate your regular expressions. It can be very tempting when you start learning this stuff to start getting more esoteric and dialing it in. Get to the point where it does just enough to be getting your text going, and then do the rest of your manipulation using simpler tools. You will thank me because you'll have a better time understanding the vast majority of what's going on here. And whatever's in the regular expression is gonna be hard for you to understand after, once you've walked away from regular expressions for more than say 15 minutes. So this is how we would use RegEx GetAllMatches to look for everything that's in here. And if all you wanted was like the second of these matches, right? You don't want the version slow, version says John, Sarah, you can just do an add filter and you can say last and just get the last entry in that array. Watch this. And then you just get— oh, because it gave me an array, right? Okay, so it— the, um, add filter, last, last update, save, rerun. Yeah, see, now it gives me just John, Sarah, which is really what I wanted out of all this, right? I just wanted to get the names out of that list based on whoever we'd said hello to. Pretty cool. What if I just want the first name that came out of this? Well, instead of using REGEX.GET_ALL_MATCHES, I could use, you know, REG and just get the first match. And I think, as you might, you can probably expect, I'm just gonna grab this text here, otherwise it was getting blown away. Okay. And if I just change this now to REG_FIRST_MATCH, and I put that thing in, And I say save. I can run debug. Ah, last, nope. There we go. Confirm. Save and run. See, now it says hello John and John, which means it was only getting, which means it's getting the pair, right? Based on whatever that first match was, right? The more expansive version of the group and then the less expansive. So once again, if I just wanted to get the John version of that, I would just throw in the last filter on that. Boom. Update, save, and rerun that. And now I just get John, which is exactly what I was looking for. So that would be the way I could use get_first. So you can sort of see how get_first and, you know, get, um, uh, and get_all. And then, you know, similarly, if I just do regex_matches, I do paste of the same string. In that case, I want to remove the last thing because this will return me a Boolean, true or false. Illustrate, see, true because it matches. If instead I made sure it just did not say hello by corrupting each of the words hello, right, by saying, you know, hexlo and hefello Sarah, right, now nobody's being said hello to. Save, rerun, and you see now it turns false. So that's a way I can use the matches to the equivalent of a has record. Fantastic. So if I look at some of these others, I can do quote and I can do replace. Leave quote for another time. Let's talk about replace because replacement's actually kind of a fun one. What if I wanted to replace each of those names? Like, what if I'm trying to anonymize the code, right? You know, hello generic person. So if I want to replace each of those with generic person Let's see how that works. Generic person, it was nice to meet you. Generic person, how are you? And in fact, probably what I would do is bring back the hello because you can see the way it replaced was using the expansive version. So we'll say update and save and rerun. Hello, generic person, it was nice to meet you. Hello, generic person, how are you? Right, so now we no longer have names, we're replacing them with the idea of generic person. This can be, again, super handy to very quickly anonymize data or whatever your other needs are. So when you go on to ChatGPT, when you're looking on Stack Overflow or whatever to solve your problem and you see one of these weird pattern matching things, um, and, and the way you'll recognize them is probably through their slashes, right? Because usually things are between slashes, uh, you know that you can bring them to bear in Xano. You just need to remember that first you don't bring the G, uh, you do bring, uh, the slashes. Um, you're going to pay it, you're going to run it once through one of these filters to see how the it gets delivered back. Because while the language of regular expressions is somewhat similar, the way they get implemented does not. They do not work the same in JavaScript and PHP and Python. And you, you should— and Xano is actually a variant on the way PHP does it, and you want to pay attention to how that, how it's actually going to look. So by running it once, seeing the results, and using tools like first and last, you can make sure that you're going to get the tools you're looking for. And then being able to do things like replace or what have you if you're looking to do more advanced manipulations. And if I were to say, you know, get all matches, like, what if I— what if all I want is like just the last name? What if all I want is just again the, the, the last of the names that showed up here? Then I can add a filter of last into it, right? And it gives me just the John and Sarah. And I will again want to do a lat— I can get just the Sarah by getting by Once again, re-filtering it on last by saying the last list, and within the last list, just, you know, give me the last person, right? And this is a way, all right, I could just find out what is the last person who got, who it said hello to. And so the way you figure out how to do this is through iteration. Set yourself up a nice simple function, test to make sure you like the way it works, because the fact that sort of backwards in this way is, you know, a little confusing. But once you have yourself set up, you're able to do your analysis, do your analysis for whatever the slicing is, and then serve that into the rest of your function stack to do all the cool manipulation you're looking to do. Look, regular expressions are hard. So if you've got questions about them, I'd love to hear more about them. Please put them in the comments. And if you're looking to be doing more complicated things around your data manipulation for, you know, your economically important project, you know, please check us out over at StateChange where we are working every day with office hours to solve the hardest 5% of problems. And I gotta tell you, regular expressions definitely one of them. I'll see you next time.