Reducing Complex Arrays with Xano

So data management is always a complicated topic, and there's a great example of that sitting over in the XAML Community right now. There's a question about how to create what they call a recursive loop using a foreach. Basically what they want to do is have this list of data and identify which of these are duplicates and just pull out those things that are unique and also find out how many times these things show up. So the way we want to approach this is to identify what constitutes a canonical version of this because these arrays are a little bit tricky to manage, right? You can say, well, ABC and CBA both contain the same thing, right? But what we need to do is identify how should they be sorted so that they look the same, and then how would we represent those as something we can compare? Because we can't really compare two objects. What we can do is compare two strings. So those are gonna become a couple of things we need to do as we sort through this to figure out how to get these kinds of answers. So to do this, we're gonna make a little function over here. And I just sort of start with the input. And I took the liberty of extending that input a bit to, and this will become clear if I just do this, to just wrap some quotation marks around these things. We're using the exact same examples came from the, the user who's having challenges, ABC, CBA. And you can see ABC, but because in JSON, whenever you're using actual letters, they need to be showing up up in quotation marks. Uh, your actual use case may involve numbers, may involve variables, whatever, but for the purpose of this, this will illustrate the point. Um, and then we're going to build out a function stack. Now what we— this kind of pattern we usually call a reducer pattern. We're taking an array of stuff and turning it into a different kind of shape that contains summary information that we're going to find useful. And in a reducer pattern, we basically, as implemented in no-code, uh, we usually have two things we need to do. The first is we need to create what we call the accumulator or output variable. That's going to be the thing that we're going to return at the end of all this, and then we're editing along the way as we go through this list. And then we're going to need a foreach loop to, well, iterate through the list. So we create this output variable, which is just going to be a nice little empty object. It doesn't have to be that big a deal. And then we're going to foreach over this input. And remember, input right now is an array. So each one has item. So the first item is going to be this, for example, this ABC. So we know the item is going to be an array. So consider that one of our situations is that we have ABC and CBA, and we want each of those two to be considered to be the same thing. To do that, we need to reduce them to what we call a canonical value, and that usually means applying a sort to it, maybe applying certain types of filters, etc. But right now it's probably just a sort, and the simplest way to sort is going to be alphabetical. So what we did, we took the item, which is the ABC, and we said, well, let's sort it using the sort filter in Xano sorting by its default, which is case insensitive text. It doesn't really matter exactly which one you use as long as you're consistent as we go through. And that will probably turn ABC into, well, ABC again, but it'll turn the second CBA into ABC. And you can sort of intuit then these are probably going to be the same thing. Now the trick then is saying I have a list of ABC and I have another list of ABC, you say, are they the same thing? And sort of, we could use like an array intersect or an array diff type approach to figure out what things are different between those two. That's kind of indirect. And what I would do is just return— is reduce these into a key, or sometimes called a hash value, that doesn't maintain the original structure, but just says, all right, what are these things? There are a couple different ways of doing that. One is we could take the, the original canonical value, which is to say this array of A, B, and C, and just join them together. Just say we're going to join them using a comma, which is default, by the way, and that will turn each of these into just a, b, c, which will then become the key. Because that's now a string, we can then compare a, b, c with another one that's also a, b, c to say this is what we want to make it, to understand these two things are the same. So by having done those two things, one producing canonical value, second producing a join, we reduce it to a string that becomes much easier to compare. Then comes the process of wanting to add it to this accumulator or output variable. And there's a reason why I created an object here instead of having an array. It's because we want to have one element per key or per canonical value, right? So at the end of this, we want to know that A, B, and C exist in the array. We also want to know that showed up twice. And for that, we're going to have two different pieces of information. We need to know what it was, and we need to know what the, what the count was. So how do we do that? We're going to ask whether the output contains a key, a key member that is equal to a, b, c, because if it does not, then we're going to need to initialize it with a count of 1 because the first time we ever saw this. And if it is there, then we need to get whatever the previous value was. So we're going to go get, find the key and we're going to find the count that goes along with that key, and we're going to add 1 to it. So previously the count was 1, next time the count will be 2, next time the count will be 3. And that becomes sort of the key conditional thing here. And then inside this loop again, we want to update the output variable by setting it to, at that particular key value, which is say a, b, c, we're going to want to set it to the count in question, which is, you know, the 1. It'll be initially, it'll be ABC with a count of 1 and a value of ABC, and then it'll be ABC with a count of 2 and ABC. Then it'll say AB count of 1, CD count of 1, right? And then as we go through sort of looking for other duplicates. So that means the 2 things we're going to have in this object are going to be the count, the number of times this particular pattern has shown up, and second, this particular pattern represented as the full-on array, because remember, we can't key an array with— we can't key an object with an array. We can only key it with a string. So we can use that key that we created before to set that to the, the pathname, and then have the rest of this come through. So let's see how this works. So what should happen here is I should get a, b, c as a key to an object. It should have a value of, I I think just 2. Yeah, so have a count of 2 and it should have a, um, you know, a list of and a value of A, B, C. Let's see what happens. That's kind of what I get. So I have ABC, count of 2, value AB, count of 1. You can see the comma wasn't there. I was wrong about that guess. It just sort of slammed them together. But if you want, we can make that happen because you can offer in the join that you want separated via comma. I could say save here and then rerun that. You see, now you can have the key be whatever you want it to be. We can have it be, you know, um, you know, underscores or dash dash, uh, whatever is going to float your particular boat. Okay, but the idea is to turn into a string. And then we have the counts, and then we have like the input data. Like, okay, fine. So what if we said that we wanted to have, um, an additional piece of input data here, like ABC CBA, and then like, it's not, they're not just together, right? We're going to add another one at the end, which is going to be BCA to see if we can get all tricky about it. And then let's try running that. Hey, look, it figured out there are in fact 3 of them and figured out all 3 of the versions of the ordering. Of course, there's more possibilities that can go into this, figured out what the counts were, and then you have everything that you need. So what we did is implement a reducer pattern where we have the original array, We turned it into an object, just different shape. We collected our counts as we went by using this if-then to find out either I need to find out what the previous count was and increment that by 1 by adding 1 at the end here, or we were just initializing at 1, and then we're updating output to contain this stuff to be storing our data as we go. And then once we've done all that, we have the data in hand that we can be using for whatever our next process is. Reducer patterns come up a lot when you're doing complex data manipulation, and they are not super simple to work with. But hopefully this gives you sort of a visual representation of how all this works and helps the person who originally asked the question over in XanoForum. And this is exactly the kind of thing that we do over at StateChange, where we work on the hardest 5% of problems that people have, including a lot with Xano, and just trying to figure out where are the sharp bits of the project so that we do not get stuck. Hopefully this unsticks you, and we'll see you next time.