Time Travel with Xano (@nocodebackend)

So here's a fun one. Let's talk about how we can create a scheduler with Xano. Basically, we want to combine some ideas so that we can say, I want a certain event to run in the future, and then have that event fire in Xano at the appropriate time. And to do that, we're going to compose the idea of having a table, we're going to compose a background task, and we're probably going to need to create some custom functions in order to make all of this, you know, work in the way that's going to be interesting to us. So let's, let's just sort of start walking through this here. The, the core idea here is that we want to have an event that we can fire sometime in the future, and that we want to probably use the background task functionality in Xano to schedule up to run the future event. Let's call this, you know, scheduled event runner. And, you know, really what we're going to want to do is have this fire off whatever the event is or whatever the action is we want to take at that future date. You know, when that, when the time comes around. Now, sometimes people look at this and say, well, I want something to fire at a specific time, but what we really want to do is have a runner that's going to be aware of whether that time is now. And so for that, we're going to need to have this thing run on a timing basis. That's probably as quickly as you can allow it to do.. And we're going to have that compare the information with a table that's going— that we'll create in just a second, that contains the list of events that are scheduled for the future to ask basically, is this supposed to run now? So the question is going to be, how frequently do you really want these things run? How close to the appointed time? And you're going to manage that using the question of what the schedule is like. Do you want this to run every few seconds? Want this run every minute? Do you want this run, you know, on just the daily? I'm going to presume for a second, we want it to run every minute. We need to keep in mind also that scheduled events in Xano run serially. So if it takes more than say 5 minutes for it to run the previous set of events, well then it's the next one's going to be 5 minutes late to the party. So it's going to be important that we ask, has the time passed? Not just is this time. We'll talk about that in just a second too. So we're going to have this thing that runs every minute. And probably what we're going to need to do is have it go get a bunch of data from a table. Well, you know, in order to even create the request for that, let's create the table. Table that will be able to store the requested elements. And we will go over to database. And we will say, I don't really have one here. So let's create one, we'll call it scheduled events. And no CRUD endpoints. This is entirely for internal use, please say add. Cool. Now, for this, we're gonna need to know a few things, we're going to need to know, well, of course, that the core things has when it was created, what its ID number is. But we want want to know when is it supposed to run, and that's probably a timestamp, you know, which is run_at. And then we're going to have, well, what's the, basically, what is it supposed to do at that particular point? We're going to need to keep track of its status. And I'm not a big fan of using enums as variables, I tend to use just strings. So I'm going to call it status.. And then we're also going to need to know, basically, the, probably the payload that we're going to send, whatever that function is. So the idea then is that we can have basically the scheduled event sometime in the future, is going to know that at this certain time, or as close to after that as is possible, we want to run this particular function. And we're going to, and we're going to run it passing an argument of this particular payload. And this is going to assume that whatever function we're going to run, it's probably just going to take one argument that argument's gonna be JSON type, which will be a payload. And it'll be up to that function to go figure out what to do with it. This keeps things nice and generic. Now, because we have a table here, and that's going to be used by the background task, let's, let's talk about how the background task can make use of that particular table. So we go back over here to tasks, and we go here to schedule event runner, right? Remember, it's designed to run every minute. And we're going to ask about everything in here, where the We're going to do a database request and we're going to do a query all records. We're going to do it against the scheduled events table. And we're going to ask, hey, Mr. Scheduled Events, give me everything where the run at is in the past. Right? So basically now, so that will give me everything that was scheduled that should have already run, but also We want to give me everything where the status of it, we're going to just say is scheduled. And we're going to make it a convention that whenever we add something to the table that's supposed to run, we're going to set it with a status of scheduled. And what will happen is we will change it from scheduled to running to completed over the course of actually running this particular function. So we'll say save here. And we'll say save here. And now we're pulling everything from scheduled events that is like supposed to run.. And that has the, the, the supposed to run as of now, and hasn't already run. So we're now going to loop over this, right, we're going to do, you know, we're going to do data manipulation, we're going to do loops. And we're just going to foreach over this guy. And we're gonna say foreach of these from scheduled events one, we're gonna call that an item, fine. And we're going to basically find out what its function is. So we're gonna do data manipulation, I'm gonna create a variable, I'm going to call it function. And that's going to be from scheduled events one dot function, because I know that actually came from that particular table. Save that. And I'm going to create another one, I'm also just going to pull out the, the payload from it. So I'm going to go data manipulation, I'm going to create variable, payload.. And I'm going to create a func. And I'm going to call this from item, ooh,.payload. Now actually, I just made a classic mistake when it comes to foreach loops. So let's all learn a lesson from this. The correct way to do this is using what's called the iterator value, which is what item is here. And you can see that I did that correctly down here. But in here, I did it wrong. I use scheduledEvents when I should have used item. If I do scheduledEvents, I'm gonna get a big array of functions. And I don't need that, I just need the function for this particular item, right? So then, you know, the— I probably need to update the event right now. So we're going to do, you know, database requests, I'm going to edit the record in scheduled events. And I'm going to get the item.id, right, the ID for this particular record. And I'm going to set its status to running. You know, it occurs to me as we're doing this, that it would be handy in the table for us to include— you can sort of tell I do these things on the fly— for us to have something that would be additional field called, you know, modified date. Or how about timestamp run started at, and then also just put in here another timestamp for completed at. Right, that way, you know, each of these two will have, you know, I'll tick these as we go along and actually run our particular scheduled event. So let's go back over to the task and go to the scheduled event runner. And I will say we're going to edit the record scheduled events, we're gonna set to running. And we're going to say started at now. Great. So the next thing we need to do is now we indicate that it's running, we need to run the function. Now this gets to the part where you can't just sort of, you know, do copy and pasting some code, because we need to figure out what function we're supposed to run. And the way I found sort of most straightforward for doing this is basically just do some sequential if-then. So we do data manipulation, we'll say conditional. And we're going to say if the function is equal to test_func, then we want to run test_func. In fact, let's, let's make test_func right now. So we'll do a, in the library here, we'll go to functions. And I have lots and lots and lots of functions. And we'll call this YT scheduler test func. And now when I create the function that's going to be handled from the scheduler, the way I like to do this is, and you're going to find that in Procode this happens a lot, just sort of having a standard input which can collect a number of different things. And then it's going to be up to the actual function to figure out what the meaning of it is.. And so I'm just going to create, you know, in here something called JSON and payload. And I'm going to, you know, have inside the function stack, it should do something. How about we, you know, what would be a good thing to have it do? Let's have it Oh yeah, we'll just have it fire off into the event log. Because I have another function that is my event log function. And that will, that'll work fine. Great. So we'll, we'll do this. And we'll say when this happens, we're going to run. Maybe I don't have an event log function. Log, log, log, log. Nope, I don't. Okay, so let's, let's create that function. So we're going to create a function, we're going to add a function, and I'm going to call it you know, log event. And it's going to involve doing an edit— not an edit record, go back— doing an add record, add record to the events table, which doesn't currently have much to it. So we're going to go back over to the, to the database. And we're just going to edit the events table to handle this situation. So I go over to events. And yeah, I don't really care about the user. Oh, I see. It's— I must have used this for something else. That happens. So what I'm going to do here is say this, I don't need this. And I'm going to say my usual way of handling events. In fact, you know what, we're just making a new table. Let's talk about how to make an event log because it's a useful thing to talk about. So we're going to add table, I'm going to call it event log. And the— and in here, we're just going to have source, the message, and the, I usually create JSON here for data. I'll make a separate video to talk about how to make an event log too, but you'll get it for free in here. So I'm gonna create an event log and the job then is we're gonna create a function, which is going to be add function. And I'm gonna call this, you know, YT log event and inside the function stack, Well, I'm just going to say that the input will be, you know, the source, the message, and an optional JSON of data. And its job is just going to be a one and done of a database request to edit— no, not edit. We're going to say add the record, add the record to say to events. No, not to events. To event log. So let's kill this off. Yes. And we're gonna say to database requests and to add record to event_log. And in event log, we're going to fill in all these nice 4 things, right? So that new element goes in all the time. Now, the reason why I log an event using a function as opposed to just doing it, you know, directly using an add record, which you might think, hey, it's a one-liner, why am I bothering, is so that later I have the option of being able to say instead of sending this to database, I can send it out to, you know, some external logging system, use NCscale for that, or use, you know, Sentry or Bugsnag or whatever you want to do for your external logging, which will have the advantage of being very, very fast as compared to this, which is, you know, can be as slow as talking to something on the database. But it's a perfectly good place to start. And it's a great way for us to start because we can use it to indicate that we have actually sent something to the you know, to the— we've done a thing, right? So if I were to quickly say, the source is YT, and the message is, I just made a log message, you know, I could run this. And it's going to be super, super, super— oh, here, we're just gonna say, don't worry about that, make it one and done. And then you'll find that in the database, you know, you've created something super handy and publish this, publish. And I go to the database. And I can go take a look at my event log. And you'll see I have these couple of records here that I just made the log message, right, because I just did that thing twice. So now I have an event log, which will allow me to demonstrate that the task actually worked. So that means I can have a function where I can go down to scroll down to do this test func for my YT scheduler, right, which can be fed an argument of payload. And we're going to say that we want to, we're going to just extract from it. You know, we're going to do create variable. And we're going to call it message. And we're going to get that from payload.message. Now, I'm assuming right now that payload is going to give me something that contains a variable called message. This is kind of important, because the message here is, this is all assuming the payload is going to come in a given shape. And if it doesn't, it will explode. So you want to make sure that we're telling, you know, the scheduler to please run test_func sometime in the future, that it will run it correctly, right? And that will be passed the good payload. And so now we can say after that, Well, let's do log. And we're going to say we're going to log an event with a source of test func, right, came from me, the message is gonna be the message. And we'll just say there's no data in there at all, right? We'll just say save. Let's try running this as a test, right? And the way we'll test this is by saying the payload needs to contain a message of Hello from StateChange YouTube. And so what should happen is running this function should cause Hello from StateChange YouTube to get fired, to get loaded into the message variable and get put into the log table. And then it griefed me because this thing down here doesn't like its answer. In general, for these kinds of functions are just utility functions, I blank out the return and say don't even worry about it. That way, you don't have to worry about making sure that you're responses in sync with your function stack, you just nuke it. Now, this should be fine. I'm going to quickly publish this on spec. And then let's go take a look at the database and take a look at our event log. Oh, look at that. From test mode, we have hello from state change. Excellent. So the next big thing we're going to do is we're going to tell we want our task to be able to handle this. So let's go over to task, we'll go to the event schedule test runner. And we'll say, well, in the event that the function that was in that table of scheduled events is called test_func, then we're going to run test_func. And we're going to pass the payload, which is going to be equal to to item.payload, actually, just, just probably just payload. Right. So now we're going to do is we're just going to pass through the payload that we put into the schedule table at the beginning, into test func, and then just sort of all goes right through and the amount that I need to worry about is really, really small. And you can rinse and repeat this too, right? You can say if I have another function here, conditional if-then statement, you know, you can just say if the function is equal to, you know, something else, right, then you want to go run, you know, this other function passing it the same argument. And we're going to, you know, from whatever your list of custom function is, it doesn't really you know, matter at that point, and just needs to be able to take the inputs that are going to work along, right? In fact, this one is kind of cool, because it would actually add the task back into the list. So we would choose, you know, let's, let's kill that off. Okay, but hopefully this illustrates the point, right? We can have multiple functions in here that would each do a different thing based on their, their work. Now, after this thing has completed, Then we can say, well, let's do another database request. And we're going to edit the record in event log. And we're going to say, where the ID is equal to item.id, right? And we're going to change this thing from a— oh, not event log. Shoot, that's the wrong table. Turn that off. Yeah. Okay. We're gonna say add database requests and we're gonna add a record to the scheduled events. And we're going to say, yep, I don't need to indicate when the run at is. I don't need— oh, they're not even add record. I just keep doing this wrong. We're going to add a step to database requests to edit record. We're going to go to schedule events. Uh, we're going to say the field, uh, based on the item.id, and then we're going to tell it to please set the status to completed. And the completed_at to now. Right, so what this should do then is it will only pull it when scheduled to run. It will, you know, pull up the function, the payload, pass those to whatever the function is. In this case, I'm just calling it test_func. And then if the function is something else, you do something else, do something else. You know, if you were, you know, if I were just to say, you know, you know, equals Bibble, right? And then we could say, please go run the Bibble function, right, or the test function or whatever. And by just by knowing that we're going to pass the payload every time, just like we do here. And I will just illustrate here. Right. So here, we will just say, by knowing I'm always just passing one argument, the argument's a payload, by making it a flexible thing like JSON, makes it super easy to work with. And then I would just take the same structure here, right? I can just basically call it as if it were the same thing. In fact, I'll just probably say it's just testFunc again. Right? And then, you know, pass the payload. And you're done, right? So I can just create other functions that will just look— that look the same shape as test_func in that they take the payload in the front, they don't care about the result. And then we just say we're going to add the record in scheduled events to indicate this particular thing was completed. Now you have basically the, you know, the full spectrum in here of how we actually run a given scheduled event. And I can just put this and tell it to go, I can publish it. And then I can enable it and publish the fact that it's enabled. I gotta tell you, it's not totally clear to me what the relationship is between publishing and enabling, but you know, that's the thing I'll sort out later. I just turn them both on. And then, and then it's supposed to run every minute. And every minute, it will check the scheduled events table to see if there's anything in there that's supposed to run. So if that's, if that's the catch, where's the pitch? This thing is checking scheduled events every minute to see if there's something it's supposed to do. And what we want it to do now is tell it to, you know, go schedule a message sent for me. So let's add a task. And we will go to functions. And we will say add that we're going to create a YT Uh, and the inputs are basically going to be a timestamp. So, you know, which we'll call, you know, run at. And we will say we are supposed to run the function. Passing it a payload. And where's my JSON down here? JSON. And in here, we will just say we're going to do a data manipulation. We're going to— oh, sorry, it's just going to be a database request. We're going to add a record, add a record to scheduled events. And we're going to say it's, you know, create at, we're going to tell it to please run at this time with that function, status is scheduled, right? This becomes where the schedule part becomes important, because I'm saying it's not done yet, I need you to please do it. And I don't really have opinions about start at or complete at, because of course, they're not supposed to go, right? So I'm just gonna say don't, don't make opinions about those. And It's good. And what it will return back is going to be the ID of the event that we would have scheduled here. The— okay, so this is also cool because, because it's going to return back the ID of the event that was scheduled. In fact, I'd say just return just.id. As, um, you know, as itself. By doing that, it makes it really easy for us to set up a canceller as well for a scheduled task, um, so that the, uh, so this will allow us to schedule for the future. If we just publish, publish, we can now say let's run it, uh, and I want to run at, well, some future time, um, Let's have it run at, I don't know, at book converter, right? Let's find out what the current time is. So this is the current timestamp in Unix. And I can turn that to milliseconds. And then these are the seconds. So I'm going to tell it to run in 100 seconds by just incrementing this up by 1. And we're going to say run test func. With a payload of message. This was a successfully scheduled test. And yeah, that looks valid to me. I'm going to say run that. Right. So now create something with an ID of 2. Let's pay attention to that. Let's go to our database again. And now we can have in our database, we have our scheduled events, let's go find those. Right, so the first guy was the one we created 20 minutes ago, sort of, you know, nobody's idea of a good time. This one is scheduled to run at 8:45. And currently, it's 8:44 on my computer. So we're just going to give it a little bit of weight. We're going to go to Tasks, we're going to go to the Scheduled Event Runner, and I'm going to tell it that if it wanted— well, let's just sort of see how it's doing. Let's look at the task history. Uh-huh, 8:44 was the last time it ran. See, it didn't have anything to do, right? Hits a foreach loop, then doesn't do anything inside that. Let's see how it does. It should pick it up, run it, and then running it, we'll be able to see the fact that it ran as a result of being inside the event log. So that's history 845. I don't know. Does it have anything in there? Oh yeah. Look, see, it has a little opener in here. That's because it did some stuff. All right, let's, let's see what did. Let's go over to the database. First, let's check out our scheduled events. You see it went from status of scheduled to a status of completed. That's a good sign. That's a good sign. So that meant that it caught our event and ran it. So previously, when it was not scheduled to run, it did not run it. Now it has run it. And we know that if it ran, it was supposed to put something into my event log. Right? So let's take a look at the event log table. And that's always exciting coming from Xano. Let's give that a refresh. Ah, 24-hour problem. One second. All right. There's our successfully scheduled test. It worked. Let's, we can. And now that we've created this function that we can use, we can put that, say, into an endpoint. So let's go, we're going to create an API group. And let's say, you know, we'll just use my FGF group, because why not. And we'll add an API endpoint, and just going to start from scratch, we're going to call it Procrastinate. And we'll make that— we'll make it a GET just so it's easy to work with. And we'll say save. And we will say, well, let's add an input of a message. Oh, there it is. Yeah. Okay. And I'm gonna say my function stack, I'm going to run my, my custom function, which is going to be log the event. Oh no, I don't need to log the event. What I need to do is instead say, and I'm gonna call it procrastinate. Saying, you know, I scheduled my event, right? That's gonna happen now. And then I'm gonna say save. And then I'm going to tell it to please you know, I'm going to say schedule task. And I'm going to say please run it at now. And this becomes like where you can do something like adding, you know, 24 hours to it, which is, you know, super, super handy. And I'm going to do add a filter and I'm going to do a transform of the timestamp to say please run this in plus 2 minutes. And by the way, this here is something you can also tokenize, like, because the format is just a string, I could replace the 2 with number of minutes with delay, right? And then, and then by doing that, I would just be able to add a certain number of minutes to it. So how about I just also say I'm going to, you know, add in here an integer of delay. Right. And now I have delay delay. And now that text of delay minutes, I can add a filter and say replace. And that I'm going to replace it with delay, we'll put the string I'm looking for and the input of delay, right, that way, it'll replace it with, you know, 2 minutes or whatever it is I need. And then the function I want to run is going to be test func, right, just like we talked about before. And with the payload, and this actually gets really, really important, the payloads, this now starts to look like, you know, XAML's form of an external API request, where you sort of build out your object right in the payload area. It's just basically the same thing here where I'm just trying to say, if I can get to focus for me, there we go. Add filter. I'm going to say set. Where's my set? Set. And I'm going to say set message to Actually, and actually, this is a key thing too. When you have something that actually happens to be the same as input, often wants to catch that and say it's going to be for the input. So for that, I need to go click on where it says constant text in order to make sure it turns into text here. Whereas down here, I do want it to be the message, right? So I want the key to be the literal text message and the value to be, you know, this bit over here. Cool. Great. We'll say update and save. Cool. So now what I have is an API endpoint that will send a message and put that into the event log at some point in the future. When, you know, after those certain number of minutes. So if I wanted to, I could do publish. And of course, I can also just do that with a run debug. And I can say I want the message saying this fired, you know, 2 minutes after the log, the other log. And if I just say delay is going to be 2, I run this. You know, it puts this into the queue. We can go take a look at that in our database. And we'll see that in the scheduled events table, we now have another thing that's scheduled, is scheduled to run a couple minutes from now. And really, we just need to wait for it to fire. And then it will add things back into the event log. I like using this combination of event log and tables and what have you, just to, you know, get a better sense of how the data is flowing. One of the great things about it is I sort of get my timestamping for free through the created_at table. Or create that field in here to know exactly what happened. For example, I know that at 8:50, I scheduled my event. Now, a really important idea here is I told it to run in 2 minutes. You might think it should run exactly 120 seconds later. It will not. It will run the next time the, the scheduler decides it's okay with running. And that's going to be subject to— usually runs on the whole minute, at least as far as its own internal clock is concerned. It's also only going to run if the previous run of the task is done, right? Because it runs one after another. And that's the reason why this thing, you know, particularly as you scale up, you're going to find that the timing of when events actually go is not going to be exactly right. It's going to be off by matters of usually minutes because of the delay associated with working with whatever else might have been in the queue and dealing with performance considerations for that as well. Let's, let's, let's, let's, well, actually, let's go back to the event log and see if it actually finished the job. All right, so it says it scheduled this event and it says scheduled event a couple of minutes ago based on my clock here. So let's, let's give it a little refresh. No, has not gone yet. Let's— there we go. Okay, hopefully I fast forward that a little bit in the video. You'll see this here is when I scheduled the event. And this is the one that fired 2 minutes later. You'll notice that shows 8:50 and 8:53. This is perfectly normal because the kind of rounding thing that we were talking about just a couple minutes ago. Hopefully that gives you the tools that you need for this. You need to create the table. The key table here is the one called scheduled events. This is the one that's going to keep track of all of your events. Of course, you can delete completed scheduled events if you want to scale it down, that's totally fine. This will eventually help with scaling concerns, but in the medium term, it really doesn't matter that much. You're looking for the time scheduled to run and start putting in this information. I think it's very funny that I misspelled completed when I was doing that. Um, you need to, so in addition to the table, you need to create the background task. The background task has these considerations. It pulls, pulls information from scheduled events that is currently scheduled and that should have run before the present time. When you put those two things together, those things are my to-do list. And then it just iterates the to-do list using a foreach loop, uh, creating variable, editing the record, and then, uh, figuring out which function is supposed to go fire. After firing those functions, it edits the record and marks it as completed. You know what? We're going to take the opportunity here just to fix that spelling error. That's the core of it in terms of how we catch the events and run them. The way that we pitch to trigger the event, to schedule the event to get fired in the future is we have an API. We go to— sorry, it's a function.. And it will be, you know, schedule_task. And that function is going to really just have the one job of adding something in the scheduled events to tell it to go run it at some point in the future. Also note that the key idea in here is the unified interface for a function that's gonna be run from the scheduler, which is why we just have them be simple things that just take one argument, which is the payload, and tell it to deal with that payload inside it. Basically the same way you would with, say, an API request or whatever that's sending over a GET or a POST, and your job is to parse it. Just our job is to do it entirely in the function stack here. Hopefully this gets you started with creating a scheduler in Xano to be able to time travel your intent to at some point in the definite future. If this is interesting to you or you got questions about it, please put comments into, you know, down below here on YouTube. And if you're finding particularly challenging work as you're, you know, working through your no-code, low-code projects, that's what we do all day long over at StateChange. See you next time.