Wolfram Notebooks as a Game Engine

Hi, everyone. Welcome and thank you for joining this live stream. My name is Kevin Daly. I'm a user interface developer at Wolfram Research. I've been with the front end team now for about six years. I code exclusively in the Wolfram language and I usually make interfaces like dialogues and toolbars. But today I'm going to demonstrate the versatility of notebooks by showing you how to make a video game. So I'm going to demonstrate that you can use the notebook as a game engine. And if you're not familiar with what a game engine is, you might have heard of, say, Unity or Unreal. Those are two examples. But a game engine is an integrated development environment that assists in building and playing games. So they'll have all the bells and whistles you need. They have some graphics, either 3D or 2D, which will have a scene that you can play the game in. They'll have physics, whether that's particle physics or gravity, things like that, and then user input, so mouse and keyboard or a gamepad. And then they'll also have things like collision detection, animation, sound artwork, etc. So what I want to demonstrate today is that we can build up the same set of tools within the Wolfram language and actually run all of this stuff within the notebook interface, which is what you see here in this presentation. Just for a little background as well, there were a couple of talks earlier this year that I want to mention. The first was by Alec Shettelbauer. He showed how you could talk to the Unity game engine from a notebook and communicate geometries and things back and forth. That was pretty neat. And then there's another Talk by Lou DeAndrea. He was demonstrating what dynamics are and going through the principles of dynamic interfaces. So I would have liked if people have seen that talk before this one. I consider this talk a bit more at the intermediate level, but I'm going to try and condense everything he said in 40 minutes down to a single slide. So that'll go great. So ultimately we're going to be making a two dimensional side scrolling platformer. That's the goal by the end of this talk. But we need to first build up some intuition about how we can do that within the Wolfram language. And the first thing we need to understand is this wonderful function called dynamic. So what dynamic does is basically it wants to display whatever's inside its brackets, inside this payload. It wants to display what that value is right now, as it is so unsurprising, if I were to set the symbol X to a Half and then wrap that symbol in dynamic, it displays that value as you see it. But the cool thing is, is when we start to put these dynamics other places on screen connected to other things like controls, like the slider here. So in this built in control here, if I have this slider thumb and I start moving it back and forth, the value of that position of the slider thumb is instantly communicated to another part on the screen. So this output 31 is effectively talking with output 32. And so with just that understanding of how dynamic works, we can already see our first game using a lot of the built in interface components within the Wolfram language. So what I have here, this is not an original idea, ProgressQuest exists elsewhere. But what I have here is a couple of interface components. I've got a couple of buttons and I've got this progress indicator here. And then if I start a new quest, the progress indicator goes based on the interaction with that button. And then you see certain interactive elements that are dynamically updating. For example, as the progress bar goes, I see my experience points are increasing. Every time I get to the end of the progress bar, my level goes up. And I also get text here that displays only for about the first half of the progress bar. And if I get tired of questing, I can just pause my quest and move on with my day. So what's happening? This is how I'm going to condense Lou's talk into a single slide. So what's happening with all these dynamic elements? The first thing is the principle we want to follow is that these interfaces are made programmatically. So from a shift enter evaluation. So what you see above was generated from this code block you see in front of you. And when I shift enter evaluate this, all of these dynamics. The kernel computes a dependency graph. It looks inside the arguments which are held, they're not evaluated initially, and then it figures out what needs to be tracked and at what rate, et cetera. So as you saw in the first row as the game started, it only displayed progress quest, but it was tracking this symbol level. And so if it was zero, it displayed the initial text. But it also tracks the progress indicator value which is determined by this symbol X. The other thing to note, so I got buttons and things that are just one off evaluations. Another thing to keep in mind is everything is scoped within a dynamic module. This is a very nice way to basically keep a part of your screen independent from another part. So what I mean by that is if I were to take this game up Here and copy and paste it. I can start a new quest, pause this quest completely independently of the output above. So truly an independent scoped interface. So the other neat thing to point out here is, is this dynamic. This is what would be called a self triggering dynamic because it's trying to update this value of X and by updating that value the value is changing. And so it's going to basically indicate to the front end that this needs to change. So the entire payload of this dynamic is going to get sent back to the kernel, that's going to get recomputed and then sent back to the front end for display and it's going to get a different value over and over again as fast as the CPU can run. And so this actually I'll tie into the first question I see in chat. Does this approach have any obvious advantages like increased performance at runtime, easier to program, etc. From my point of view, in terms of ease of programming, we're going to build things up step by step. So that's always a nice way to approach things. You can sort of tackle things in bite sized chunks. And then having this run as fast as your CPU can allow is sort of the way we have to do this. We could have done other clock like mechanisms, either using clock or animator, but to get a nice smooth transition of pieces on the screen, we want the front end to be running as fast as it can. And I'll explain that in the next slide. Well, next, next slide. The other thing we want to have is pretty things. We want to basically use graphics and not these built in buttons and things. So we do this within a graphics expression. What I have displayed here is a graphic. I've made the background light blue, just so it's a bit distinguished from the white background of the notebook. I've also turned on both the frame and the grid lines. And this is also going to demonstrate what's neat about graphics is it has this internal coordinate system for convention. I'm going to set the plot range which is basically showing. It's kind of like the camera of you know, what's being displayed on screen. I'm always going to take 00 to be the lower left corner and then however big my scene needs to be. The other thing to note is that there are graphic primitives. So what I have displayed here is a disk at 2, 4. I have a rectangle and then I have a line here. So this is how we're going to build our scene. We're going to use graphic primitives as our hitboxes. And so there's also additional things you kind of get for free from a graphic, which is some mouse event tracking. So if I click once, you see that I have an orange border around this graphic that indicates it is a graphic. But if I double click, it turns into this gray border. And now I can actually interact with each of the graphic primitives by clicking on them. So with just the mouse I can click and drag this disk to a new location. And if I were to peek behind the curtain, you would see that this disk position has actually updated the first argument of disk, which is its current location. Instead of being at 2, 4 as it was initially, I've now moved it to about 3, 4. So we're going to use this and basically do these updatings programmatically instead of with a mouse and keyboard. So then how do we do this smoothly? So what I have displayed here is two self triggering dynamics. I've got a purple disk and a red disk. I'm following one of the principles of dynamic interfaces in that I've taken this dynamic and put it as deep into the expression as possible. In this case, it's the first argument, the X coordinate, the horizontal position, however you want to think about it, of the disks and they're self triggering because as it runs off the screen that needs to recompute and put it back on the left. But it's also self triggering because it's constantly just updating its position. Things to note here, this, because these are self triggering, it's running as fast as your CPU can do. This. This is kind of reminiscent, this will date me, but when I was younger, the first computer I had was an IBM 386 and it literally had a turbo button on the outside of it. And so games were basically coded to the CPU clock speed. And you sometimes had to press the turbo button to make sure your games ran at the appropriate speed. It's a very similar idea here. The dynamic is running as fast as it can. And so we need to sort of externally control how fast the game updates. And so that's what this time factor is here. And you know, it's a fairly small number, but basically we're just chopping the. It's basically a frame rate. That's one way you can think about it. And so we get this smooth animation by basically just chopping the updates to very, very infinitesimal sizes. So the displacements are very, very small. And if I change this time factor, I can make things even smaller or if I go to a smaller number so the intervals are larger then at the apparent motion is faster. But note that the relative motion between the two disks is always the same. It just depends on the velocity that we have in front. So what should this value be for your computer? You can run this dynamic here to get an idea of that. This is using an animator, which is kind of like a clock for the front end. This is just running from zero to one, a single time when one second. And I'm counting how many times I can update a symbol in one second. And so you see, I get about what I saw in the previous example time step about 0.005. And that'll basically be a factor that you're going to see in the rest of the interfaces that we have going forward. So if you give me a sec, I am going to momentarily stop sharing my video. So I can do a transition here because I want to show things you can do with a gamepad. What you might not be familiar with is that for a very long time now the Wolfram language can recognize gamepads and you can do this using the controller information function. So I have other things connected, like my headset. But what we want to focus on is this Xbox controller here, which is Bluetooth connected. Didn't do anything fancy. I just Bluetooth connected the controller. And already as I move these thumbsticks, we can see that some of these variables on screen right now are changing, like the x and Y axes, the directional pad, which is called a hat switch here. The other thumbstick is a rotation and then true and false Boolean buttons, etc. So lots of fun controls that we have accessible right within the Wolfram language. Now those are the RAW controls. I could have also used these other symbols, but I'm just going to stick with the RAW controls for this talk. So that's how we can see what's connected and available to the Wolfram language. The to access those values, we're going to use the function controller state. And so you can put as the string here any of those strings that we saw inside controller information. So if I do the x and Y axes you see moving this thumbstick will update those values on screen. And just in case the batteries in this remote die, I'm going to wrap all of the future interfaces in this controllerready queue function so that if the controller state happens to fail at any time because the controller dies, we're just going to get this nice display instead of say like pink boxes everywhere. So now that we know that this controller can talk to The Wolfram language. And we also know how to move things on screen programmatically. How do we then do the movement with the controller? What we're going to do is everywhere we saw Velocity in the previous example, we're going to replace it with our controller state. Now, as I move the thumbstick to the right and to the left, I can move these red and purple disks to the right and to the left. There is an issue of controller drift, which is something you might see in the news every once in a while. I don't know if I can get it to do it. There we go. So I barely touched the joystick, which looks like it's still in the centered position, but there's a slight drift to the disks. That's because the thumbstick is a little worn out. And so to control for that, I actually just round the values coming back from this to the nearest decimal place. This is just something you can do this different ways too. I could have, you know, clipped this based on the absolute value. For the purposes of this talk, I think rounding is just fine. Yeah, that's all you need to do. Now. The joystick is basically the velocity component of updating the player position. So now that we know how to do this programmatically, we want to create our first function to sort of compartmentalize how everything is updating inside our game. So the first thing we're going to do is put this new knowledge into a function. When I move the joystick, it's going to have some restrictions eventually, like I'm going to run into a wall or the scene boundary, et cetera. And so the idea that I'm showing here is that all of the motion based on the controller input is going to be a suggested motion, which is what you see here. So basically, the player's position is going to have added to it something based on the controller input. And that's going to be the suggested new location for this player. And so I just call this in all these functions x0 right now, this has no restrictions to it. Basically any of these suggested updates are instantly going to be accepted as the new player position. But we're going to have additional logic as we go through this talk that's going to restrict player motion. Another thing to note about this function, this is a convention that I use. My first argument is always going to be held that way. The symbol here, the player's horizontal position can appear on the left hand side of this equation. And so anything that needs to be updated basically is on the left hand side of the equal sign needs to be held. One of the questions on chat, can you also incorporate a dead zone amount to ignore? Yeah, absolutely. So you would do something like the absolute value of your control, you know, if this is less than, say 0.15, then you would accept the value, otherwise you would ignore it. So you can create a dead zone quite as large as you want. In the vertical direction. This is identical right now to the horizontal direction. We just have free motion in both directions, coordinate directions. The only difference here is a minus sign. If this was a plus sign, then the vertical axis would be considered inverted. I'm not one of those players, so I like having that minus sign there. And so here's a new scene in which I have a player character is this purple square. I've taken those two functions and thrown them in here to move both the X and the Y components. And now I can move around in my scene with the joystick. And you might be like, hey, hold on a second, Kevin, what about that principle of minimal dynamics? I see I have a dynamic here that's not quite as deep inside the expression as it could be. The reason here is because the hitbox that I'm using is rectangle. And this primitive actually has two arguments. And so one is like the lower corner of the box, the other one is the upper right corner. And so I would have to have two dynamics here if I wanted to put it as deep as possible. The number of dynamics you have is the number of independent updates that the front end has to think about. And so this is basically an optimal position for this dynamic. So I'll actually show one other thing. You see that as I move this, I can actually leave the scene boundary. So the first thing we're going to do is sort of lock this motion down and we're going to just assume a rectangular world because that's sort of what we see. And this is our first instance of collision detection. So what I mean by this is we're going to detect whether or not we've collided with the edges of our scene. It's very simple to do because I've taken the convention that the scene has its origin in the lower left Corner, which is at 00. And so if the player's minimum hitbox side goes off the left side of the screen, we just replace the player's position to zero and we do that for all edges. I'm assuming a rectangular hitbox. And again, by convention, I'm taking the player's position to be the lower left corner of this Rectangular Hitbox. The scene has some width and height which we're going to allow to be adjusted. And so if we go off the right side of the screen or the top of the screen, you always have to take into account the width of your players. Hitbox. So that's our first boundary check, our first example of collision detection. So here's that exact same scene we had before. I've just added the boundary check. So now as I move the control around, I am trapped within the scene here. But we want more. We're eventually moving towards a game which is going to be a two dimensional platformer and so we need additional platforms, additional things to hit. So how do we detect those? Again, this game has a lot of conventions. I'm probably going to use that word a lot conventionally. I'm going to assume that these platforms are all also rectangles. One reason is that the detection of whether or not two rectangles are overlapping is extremely simple. Basically, if one rectangle is to the left, if it's above, if it's below or sorry, left, right, below or above another rectangle, it's obviously not hitting and so throwing this not boolean thing in front of it. Yeah, it's extremely simple hit detection. The other thing I like about putting this into a function which I've called impact queue. Q being a question of whether or not we've hit is that we can easily substitute or basically extend this definition to include things like disks or other primitives that we want to eventually include in our scene. Not for today we're just going to be doing rectangles. And so seeing this just briefly in action. If I had a set of rectangles already in my scene, I could then easily detect what those hits were by just using this impact function. So I can either select certain rectangles out of a list of existing rectangles or I can visualize those overlaps like in this basic graphic here. So I was just kind of showing, yeah, it works, it's nice and it's fast. So how do we do this if we're going to have multiple platforms on screen? If this was a very high end game, you might actually participate your screen to only look for objects that are near your player character. We're going to have a small game and so we're not going to do any of that fancy stuff. We're just going to basically iterate over a list of obstacles and check every single one for an impact at all times. And I just put this into an impact anyq function. So Pretty basic. But now we have to update our horizontal and vertical movement functions because now instead of having just free motion in the screen like we saw earlier, we only want to accept the suggested location if there were no impacts. So here we have the player character's position. If the suggested new hitbox of that player interacts with any of the obstacles, we're going to round down the player position to an integer value. Because yet another convention that I'm going to be using in this game is that the platforms only exist at integer values. It's a very nice way to do things. Other games do this quite a lot. You can think of it as like tiling your game world. It just makes game development pretty easy. Y component is exactly the same. And so seeing this in action, I have a scene here, I've made it a bit wider. Obviously, I've included a few default obstacles to get around and so I have free motion. So long as I'm in empty space, I can still hit the boundary. But now any of these rectangles, if I impact them, I don't move through them. They are solid to the player character. So that's really neat. But wouldn't it be great is if instead of sort of guessing where we're going to place these blocks in the world by turning on the frame, etc. Wouldn't it be cool as if we could place these dynamically? And so this leads us into our first tool that we're going to be building. This is going to be a object builder. So I'm taking the exact same scene from before with the same set of initial rectangles. I've just taken this function we haven't looked at yet, and wrapped it around my scene. And so now if I click and drag with the mouse, I can add blocks wherever I want them. And as something fun to do, if I'm drawing this rectangle, the suggested rectangle, and I hold the control key, I can actually delete those rectangles. So this is a step towards a nice integrated development environment where you have these additional tools to make level building easier. I don't want to. This is going to be a bit of code spew, so hold your breath. But basically, just to see what I'm doing, I'm using an event handler wrapped around the graphics. I'm adding this new temporary rectangle based on mouse events. So the mousedown event, mouse dragged and mouse up. I'm only accepting integer values for the mouse position just to follow my convention. And this will create a suggested new rectangle for the scene. And if it's not a zero size, then I will add that to my set of rectangles. Or if I'm holding the control key down, I'll use the impact function. Very useful, very convenient. To only select out those rectangles that don't impact. The suggested box, this dotted box that I had. So yeah, very nice, very convenient. So one of the questions is this is pretty cool. Thank you. I don't understand on what version of Wolfram Notebooks I can do all of this. You can do this as far back as as the controller. If you want to be using a gamepad, oftentimes you can still use a mouse and keyboard, but you can do this pretty far back. I won't say for certain how far back. I'm using our current Release, which is 13.2, but I was making games six years ago using very similar methods. That was easily version 12, if not older. OK, so again getting back to the side scrolling and platformer aspect of this game. We don't want free motion in the vertical direction anymore. Now we're going to actually move towards using a Boolean control like this, a button here and we're going to have that be a jump button. So we need to understand how gravity works. So what I have here is a tool that I built. So here's our second tool in which if I press the jump button, this character block here will go up to a certain height and come back down to the ground. I know that I've gotten gravity right because if I change this time factor, it jumps up to the same height. It just does it a bit slower depending on what that time step is. But what I can do with this tool is say increase gravity. So I don't jump up as high, but I can also increase the impulse that pressing this button gives to sort of go even higher. So this is a way to play around with gravity before you implement it in your game to get sort of a feel for what type of player movement you want. Another thing to notice here is that if I hold the A button down, you see that it only get does one jump. We're going to have to implement that when we go to change the vertical motion of the character. But that's a basic element to games is if you hold the jump button down, you don't want to be jumping continuously. So we'll see how to implement that. So how do we do gravity? So here's our same movement function instead of it being controlled by. So this is actually assuming just free fall. So gravity is a constant and it's a change to your vertical velocity. So if you haven't had a background in physics, don't worry. Just understand that this is how you do it. So we have a displacement in the velocity value, and then once that is a newly suggested thing, that becomes the update to our player position based on that velocity coefficient. And so this is what would look like if we were just going to be free falling through our scene without any things stopping us. And so just the bottom line here is acceleration is a change in velocity, so we got to do that change in velocity before we change our position. So then how do we do jumping? Well, this is the complicated bit that I was hinting at earlier. We have a boolean control now instead of a joystick. So this is going to read true or false as we press it. But if I were to keep the button held, it would always register as true. And we don't want to always have this added impulse happening continuously. So I'm going to throw in another secondary boolean here of whether or not it's allowed to press the button. So basically, if the button is pressed and it's allowed to be pressed and we're not in the air, then the character's vertical velocity will get an added impulse. And then as soon as that happens, we turn the button off effectively and we realize that we're now in the air. And as soon as you let go of this button, so boolean control is false, then we allow the button to be pressed a second time. So this is as basic as I can get for this boolean control. One of the sort of common things to have inside scrolling games is like a double jump. So this logic would get a lot more complicated. But it's totally possible for the sake of this presentation, we're just going to have that single jumping action. But that's not all. We need to update our boundary check and add that same logic to our Y movement so nothing too drastic happens to the boundary check. I'm basically having a hard boundary that if you hit the ceiling, your velocity goes to zero. If you hit, say, the floor, the velocity also goes to zero. And I just set this logic that you're no longer in the air. Pretty obvious. And then hopefully I didn't lose you with all this code. The same A button logic is the suggested location of the player that we saw previously. And now I'm reintroducing the player's hitbox. So now if that suggested location for the player hits anything, hits any of the obstacles, then the Velocity goes to and you hit it from above. Yeah, that's the key point here. If your velocity is negative, you must have come down from the above the object. So therefore you're no longer in the air because you're touching the top of the object. Your velocity goes to zero and there's that convention of rounding to the nearest integer. Otherwise if you haven't hit anything, you must be in the air. So just accept all suggested locations. Yeah. Now, thank you for getting through all that code. Spew. Here's seeing it in action. So I've changed my scene. I've made my player character two blocks high, but now I can jump through my scene by pressing the A button. So the character is always heading to the floor. And if you've noticed, I've already included my rectangle builder so I can make a series of platforms to jump on. Oh, I didn't make it. So maybe I need a secondary platform here. And you see, I can't get through the top of the screen or the right or left. So we have quite a bit already of what makes a side scrolling game. What are we missing? The side scrolling bit. So this is also very easy to add. One of the options inside the graphics expression is plotrange, which basically we've seen before because I set it to like 0 to 5 at one of the very first slides of this talk. But basically we can set both a horizontal and vertical sort of camera that views our world. And so what I'm going to do here is effectively just track the player position. And based on how wide our camera is, we're just going to go to 1/2 to either side of the player position if he's centered in the screen. Otherwise the camera view is going to be glued to the left if the player is to the left or it's going to be glued to the right. And this can be done in both the vertical and horizontal directions. I wrote this very generally and so seeing this in action, exact same scene as before. I've just added these two sort of scroll functions here. And now my graphics option has this new plot range. Oh, I actually forgot to mention previously. The other thing is I've switched to using the directional pad so it's the same X and Y movement. I've just changed the controller state to be the hatch switch, hat switch. So yeah, I can still run and jump. This is fine. But we don't actually see what's going on unless say I build a staircase to heaven here. So now as I jump from block to block, the whole Screen moves with me. So now I claim that we have all the elements of a two dimensional side scrolling platformer. Well, wouldn't it be nice if it looked prettier though? So I'm not an artist, but I do want to add artwork to my game. So just to show you how we can do that, we need to add these bitmaps because I'm going to sort of reminisce on, you know, 16 bit or those style of games. So let's say I have this random image that I like and I want to put it into my game and put this into the graphics scene. I can do this in two ways. I can either use inset, which takes four arguments here. You know, putting the thing in the first argument, the lower left corner is going to be at 4, comma, 2 in my game coordinates and it's about 10 pixels wide. Very easy to understand, but it's a bit too powerful for what we want to do today. Inset actually lets you put anything inside a graphic. It doesn't even have to be itself a graphic. So the way that we can do this a bit more efficiently is to use the raster primitive. Raster is just a two dimensional array of pixels and we can get the exact same effect much more efficiently using this primitive where we need to extract the image data of the object that we have. But we need to keep in mind that in these image coordinates, the Y axis is inverted compared to the graphics coordinates. So we can take care of this in either two ways. We can reverse the image data or my preference is just to reverse the max and the min of the vertical components. So if this was actually following the same sort of rectangle convention that we were using, the image would be upside down. So now that we know how to place new artwork into our scene, let's do that. I mentioned earlier that games like this were based on sort of a tile concept. And so you can actually find tile sets online. I use this open game art website and so I'm going to be using a set of tiles that interconnect very nicely together. And so I've labeled these, you know, single bottom, center, bottom, etc. But the patterns on all of these things nicely connect side to side in a uniform manner. And so what we're going to do is use these tile sets and add them to our game. I'm not going to show you this because it's a little bit of code spew, but basically I'm going to image assemble all of these individual elements in an appropriate way. With rounded corners, you know, et cetera, and just put them into a single raster. That's gonna make things more efficient than having a whole bunch of rasters, you know, for each individual tile. And what I'm going to do is update this rectangle builder tool to include all of this new artwork. The only change I've actually made to this code is that every time a rectangle is accepted, I'm now also appending to this list of artwork in my game as well as these hitboxes. And so seeing this in action again, this is the same scene I've just added this way to choose a tile set using a setter bar, another built in element for interfaces. And instead of being dynamic of platform, this is now dynamic of artwork. But otherwise everything is exactly the same. So you can pick your artwork like this stone here and you can start drawing and I can move my player character around in the scene and maybe, you know, make another staircase out of snow. So yeah, this is already a very simple way to add artwork. You can find it online and then import it. With a bit of understanding of how image assemble works, you can get basically point and click level builders. But that's not where we want to finish at, is it? What we really wanted to do is also add animation for our player character. We've added platforms that are rather static, but how do we do something with a bit of motion to it? We're going to be using things called sprite sheets. I'm getting this from the same free game art source as before. And this is like a little alien with a helmet. This is called a sprite map. It's basically a set of frames of an animation. And nice sprite maps also tell you how to extract the location. So this one does. It basically tells me like, here's a player walking. Frame number is a very nice way to extract these. And so what we can do, knowing how, where these are all located, we can extract all of the images into our animation cycle. So the cycle starts with the player standing, goes through the walking motion and ends at standing again. So that if this were to loop, we would see a nice continuous motion that looks like this player character is walking. So normally we would have a single animation cycle for every animation we want to do. I'm going to cheat just a little bit for today. I also want to see the player standing still facing the camera and he's also going to have a jump frame here. And my cheating aspect is I'm going to sort of jumble this animation cycle by appending these two frames to it. And you'll see the impact that it has in my future code. And so ideally I would just use the animator function we saw many slides ago to do the animation. But I'm basically going to do the same thing, but just sort of hand rolled. For this presentation I have this animate function which is just keeping track of some variable which is a clock. And the key point here is that this clock is updating slower than the CPU clock speed. And so there's additional time factor here which I call animation factor that just slows down how often those frames in the animation cycle update. And so I'm using the mod function with round here just to have integer values. And it's basically it would cycle over your animation cycle length over and over and over again. So that's if we had an ideal situation, but I cheated a bit. And so here's that same ideal animation, but if we're in the air or we're standing still, then we're no longer animating. And so in that sense I reset the clock to zero. And then if we're in the air, I pick out either the first or second end elements from this animation cycle. And so also the animation depends on which way the player is facing. So I still have the controller state here which is going to be tracking this directional pad. And so if I'm if the directional pad is to the right, then this will change the animation direction to the right and vice versa for the left. But if I'm standing still, nothing happens. And then lastly, the last code, SPU is going to be using raster instead of rectangle. So where we had before was basically showing the Hitbox itself. Now we're going to be showing this raster image. And so raster as we saw, has two arguments, both of which can be dynamic. And so I'm putting the dynamic as deep as I can. This dynamic is basically just watching for which animation direction it is and reversing the image data if it's seen to be moving to the left. It's also tracking the index of the animation cycle. The second argument of Raster is where we're putting basically locating this image. And so it's also where I'm going to be putting all of the updates for the player position. So the same motion and boundary check. Here's our new animate function. And then the last thing is what's actually displayed by the dynamic. And so in this case I'm taking the player position and just slightly shifting it such that the image is Sort of centered on the Hitbox. Because what we want to do is also make the player character slightly larger than the Hitbox. That makes a much fairer game if your Hitbox is smaller than what's displayed on screen. So now with all of that, there's our animating character. So if I am currently walking, so basically if the directional pad is pressed, then I have the walking animation. And if I'm in the air, I just have this single frame of the character jumping. And if nothing is happening on the controller, the character is facing forward. So I think that does it. So let's put it all together. So here's the exact same scene. I've included the platform builder in addition. So now I have a running and jumping character on some pretty blocks. And so this sort of reminiscent of, say, Mario Maker or some games like that, where I can build out my scene and play, test it in real time. And if something goes wrong, like, you know, I can't make that height, you can do things like, oh, I'll just add a sand block right here. So now I have something I can jump on. So this is about how far I wanted to get in demonstrating how you can use a Wolfram notebook as a game engine. We've built up a couple of tools along the way and making games a bit easier to develop. And I've shown you the basics of how to animate at artwork, etc. There is a little bit more I wouldn't mind showing. What I did is I made a game about six years ago which is basically taking the Castlevania game and only taking the first level. So this is from the Nintendo Entertainment System, a very popular game at the time. And what I'm going to, what I did. I'm not saying that this game got me my job in the front end team, but it didn't hurt to have this in my portfolio. This is taking a lot of the same elements, but from myself six years ago. I consider myself a better programmer now, but basically I have a start button. You can hear, there's already some music. And I have a player character that has way more animations than you saw in the previous example. And so I made a game that has a horde mode. So you see in the upper right corner, it's keeping track of how many enemies you kill. The enemies are randomly spawning off the screen, and if an enemy hits me, then game over. And so the point of that game would be to basically maximize your score. So that was something fun that I developed. If you want to add sound effects to your game, you can use the function emitsound. The ones that I got were a bit too loud. So you can use other options like sound volume to control that. Hopefully I didn't just blow your speakers with that example. But see. Oh, there was another question. I'm into machine learning and virtual reality. Is it possible for me to build the player to be a machine learning model? Can this be incorporated into virtual reality? I believe at least the second question. Can this be incorporated into virtual reality? I think the answer is true. I've seen that demonstrated before, But I think using the Unity game engine to walk around in. I don't think I'm qualified to answer your first question. The machine learning model, probably. But again, I'm not qualified to answer. I'm sorry. A couple of other bonus slides. We've been doing things that affect just the player velocity by adjusting the position. Joystick is acting as the basically the velocity of the player, but you can do things that affect the acceleration. So here's a quick game just showcasing a couple of bonus things. Very reminiscent of the classic Asteroids game where now the joystick is adjusting the velocity which as soon as I let go, the velocity is whatever you left it at. And then the right joystick will actually shoot bullets if you're facing a certain direction. So I didn't quite finish this game for the talk, but just an example of using the joysticks for different aspects. Another thing, I personally like indie games like Stardew Valley. So in this game has a fishing minigame where if you press the boolean control here like this, a button, you can actually move this rectangle here on screen. And so this is also an example of affecting the velocity by giving this rectangle an impulse. And if you hover over that fish, then this progress indicator goes up and this is how you would indicate catching the fish. So yeah, just another quick example of a game that affects velocity and just to not, you know, leave this out. Another game I made about six years ago was a port of Flappy Bird and again using ideas of gravity and using Boolean controls. So one thing you'll notice here is that I've overloaded my impact function to now be between a disk and a rectangle because Flappy is basically circular. So I made him a disk Hitbox. So now if I hit the start button, I can now jump Flappy through these pipes. So if I were to do this again, I would use the raster function. I think I am using inset here. So it's running a little slower than I'D like. But the idea would be similar. If you take the concepts from this talk, you could easily make a game like this. Oh, and one last bonus. Since I've got some time, I created a while ago a painter function or a painter interface. So this is like an additional tool for making games, especially pixel art games. So let's say I've attempted to say, remove the background of this fish. Now this is actually a real life example. I tried to do this in making the Stardew Valley example that we saw previously and I needed to sort of chop out because I basically copied and pasted this from the Internet and I needed to remove the background and that function just didn't quite cut it. And so what you can do instead is use this tool here to basically paint and update your pixel art. So on the left here is a way to select which region you want to work with. And on the right is a zoomed in representation of the individual pixels. And let's say I take the eyedrop tool to take the color here, which I know is transparent, and then I can switch to the bucket tool and I can start basically cleaning up this image to make it what I need. So yeah, just a quick example. And then I of course can save this to some external symbol and then when I look at that symbol, you see I've actually saved my results to this external variable. So it's kind of like Microsoft Paint but doing everything within a Wolfram notebook. That does it for me. I'll take any questions, anything you want me to go back over in detail, but I'll just leave it here if there's any other questions. Oh, and I will go back to actually looking at me. How's that? Oh, I'm looking to see if there's any questions in chat. Just a few minutes. Well, well, I'm waiting. I'll just play around. Can't do that. This is me twiddling my thumbs. Almost made. Well, it looks like we don't have any other questions today, so again, thank you for joining me in this live stream. It's been a lot of fun presenting the versatility of Wolfram notebooks and what you can do, like making a game inside effectively a game engine like environment. And yeah, until next time, have a nice day.