Neural Networks in System Models

Hello. So today we just wanted to talk a little bit about the progress that we have been making in how to integrate data into system modeling. And especially now that we have provided support for the introduction of trained neural networks and system modeling. So we want to explore some examples and I'm going to show this to you in case, well, maybe people are already familiar with System Modeler, but in case people are not super familiar with system modeler. System modeler is a very nice tool that uses physical modeling. So it's based in the use of components and connections. And it allows you to get a very nice picture of a digital twin of an actual real system. And you can go from very simple systems like this to full robotics modeling, which is a lot thanks to all the domains that you can cover in system model. You can do electrical modeling, mechanical modeling, thermal modeling, and all those components can interact with each other. So behind all of this structure, there are a lot of differential algebraic equations that are doing all the, all the work for you. But the idea is that thanks to this language of connections and components, you get a much clearer picture of what you are modeling. So now that we have this idea of what, what, what we are doing in system modeling, the natural question is, what are we going to do with data in system modeling? And there are some very obvious ways in which we have already, we have already provided the support for the introduction of data in system modeling. So the most directly, most natural thing to think about with data is to just pick some data from the real world and create an interpolation from that that you can just use in Synstet modeler. So the idea in that case is just very natural. You just pass this data to create a component and then you can use that component in system modeler as an input, for instance, for any other components that you have in your modeling libraries. In this case, for instance, very simple example, a very straightforward example. So we know that Wolfram has access to all kinds of real world data. And we can, in this case, we can for instance, pick data from of air temperature in my current location. And after cleaning the data a little bit, we can pass it directly to a function called Create Data System model. And that's just going to create this component which is directly just an interpolation of the data that we have in passed. So it's a very natural interpretation of it. And then afterwards I can just directly connect this component to whatever things I have in my libraries. So in this particular case, I'm creating this kind of model where I'm just Passing my outside temperature data, I'm just doing a little bit of unit conversion and then I'm passing that to a model of room heating where I'm doing some temperature control. And so I can use directly room real world data and test that against my controller in the room. And then I can just plot the results and see how my controller behaves and how the temperature is behaving outside versus inside. So that's a support that we have provided for a very long time. And that's kind of the first very natural thing to do with data. But the second thing, the second natural thing to do with data is to just directly to do some kind of fitting. And you can think of two kinds of fitting. In this case you can do the kind of natural like fitting into a traditional statistical model. So for instance you can do of course you can just extract some real world data. So in this case, for instance, vary on topic. We can extract the value of the, of the British pound against the dollar throughout the year and then you can see the data in a plot. And of course you can create a linear model fit of that data with like the standard warframe language tools that we have. So you can do a linear model, you can do a nonlinear model fit, a generalized linear model. The output of those functions is a fitted model which in the system modeling functionality you can just pass to create system model directly. And that's going to create a component that extracts the best fit model. And then that thing, you cannot just plug it into your components directly, just like previously mentioned, and you can just observe how things behave. So for instance, in this case I'm plotting the data that we originally had and then we just pass. I'm not using my component in system modeler that has my linear model bit. I plot the behavior of that component. When I pass just a linear input, then I see exactly what I would expect from a linear model bit. So that's one natural way in which I can work with data and I can just extract the statistical information and pass it to system modeler. Another way in which we can interact with data is a little more connected to system modeling, which is parameter calibration. So for parameter calibration, the idea. So the context for this is let's say that I have a specific model in mind. In this case this is just a simple electric motor and then I want to or clearly my motors, it has a lot of components, resistance, inductance, etc. And I want to adjust specific parameter values in my model so that they match some real World data. So basically what I'm doing is I'm using my system model, I'm extracting some real world data from somewhere. And then this is just, this is something that we're looking forward to streamline in the future. So obviously I can just use function in Mathematicas. So anything in the side of optimization that we just take it will just consequentially simulate my model with guesses for my parameter values. And it's just going to reduce the distance between what the simulation is produced and what the real world data. And the real world data. And I just minimize the distributed those two quantities. By doing that, I find a couple of parameter values that adjust my system model in such a way that it matches the real world behavior. So now I can just simulate and I can plot the results. I can see how the behavior of my model matches the data that I use for the minimization. So this is kind of a workflow that we are very much interested in improve for the user. So this is something that we're looking forward to investigate. But this is another natural way in which you could use data in system modeling. And now for the new thing. So neural networks. Right. So of course if you, if you, if you're interested in training more traditional statistical models, you are also interested in training the newer kind of things. So in, in, in the most basic case that you can consider for like a use case in neural networks in system modeling is well the standard way of thinking of machine learning algorithms. So you just train a neural net with some, with some data and then use the trained neural network to create a component from it, just like how we did with the linear fit in the previous example. And obviously now once it lives in the system other world then you can connect it to how, whatever you want to connect it. So for instance, let's go through that example here we are actually using some, again some data that is connected to the energy performance of residential buildings. So in this case again we are talking about heating. And the standard workflow for machine learning algorithm training is to well, extract the data, to clean a little bit, to do some splitting into training and validation. So this is all in the standard language of machine learning, machine learning. Then you can pick a specific neural network structure, you can initialize it, and then you can just use the data, the validation data and the training data to train your neural network. This is all well and good, but now what I would like to do is to take my training, my train neural network and then I'm just going to Create a component from that neural network. So again I just pass it to Create System Model. And then one option that you can set in Create System Model is generated as a location. So since you are working in the system modeling world, you're probably interested in storing your neural network in a specific file location. So what is happening in the background is that we are taking the trainer network, we're exporting it as an ONNX file and then in System Model we are using, we already have the functionality, so we're using ONNX runtime to run the train neural network and just consider it as another component in our libraries. So now that we have this neural network here, so I can just go to it in, in model center. So this is how the icon looks internally. It's just referring the ONNX allocation of my ONNX file and specifying the number of inputs and outputs. But that's just enough to be able to pick it up and then use it wherever I want. So for instance, just like Jan showed in his presentation, I can just use it in a completely in a much bigger model. So I just take my component, I place it here, and then it can interact with all the kinds of components that we're mentioning before. So we have some thermal modeling components, some thermal inertia, some temperature sensors, some radiator component. And then I'm just passing the, using the information that I have already stored in my neural network to simulate and to be able to use well, the real world data and the trained model that I had in my actual model modeling environment. Yeah, and of course, and I kind of, I can plot now the behavioral Y of my variables and see how they behave. So this is kind of like the standard, the very, very standard use case of machine learning. Well, in any environment really. But the one actually we got to talk today about a nice industrial use case that we just finished working on. So we worked here, well, here in WorldFront Nat Core, we worked together with the Linkoping University and other companies that are associated with the Lillynchopen University in a project about the use of machine learning algorithms in the prediction of substances, in the prediction of how much substance is produced in some bioengineering processes. So the idea is that there are a lot of biological processes that are still not easy to model deterministically in bioengineering. And this is a world in which like a kind of black box modeling, like the kind of that you could find in a neural network could actually play an important role for prediction. So the idea is that they developed well, they worked on new measurement devices that were producing different kinds of signals according to the concentration of substances that they wanted for the production of drugs and medicaments. And they wanted some way of predicting the amount of, of useful substance that they could produce given the measurements that they were making. And the idea is that this particular plot, what I'm just showing is one of the specific lab equipments that they had. They were producing one of these curves that you see here. And the idea that each of these curves corresponds to a different kind of concentration of the systems that they are interested in. So the idea is that we should. Well, we are. And that's what we showed in the work. You are able to use the data stored in these curves to predict the amount of concentration of substances that you can use later. And for that we just. A neural network. So we worked completely in the. So we worked both in Mathematica and in system model. So we did all the training on neural networks in Mathematica. So we did some dimensional reduction of this data, we cleaned it, we trained a neural network. And then the advantage of system modeler is that once the component corresponding to the neural network was in the world of system model, like this one. The thing is that system model is extremely well equipped to connect with all kinds of lab equipment, thanks to some libraries that have the kind of connecting protocols that you need with all kinds of equipment like OPC or Arduino. So now that so often what you have in one of those laboratories is that you need a way of centralizing all the information and all the data that you're gathering, sometimes in real time, into a single machine. And system modeler is the perfect application for that because you can easily connect all those things to system modeler. You can put there your trainer network, make your own models of your systems and then keep running that on real time and keep getting instantaneous information of how well the model is behaving and how much, how well the production of your medicament is going. So this was kind of an interesting industrial application of this work. And now the good thing is that yesterday they told us that this paper was approved in the Journal of Analytical Chemistry. So that's, that's good news for us. Good. But now let's go into the future. So the things that we are looking into working for aspects like in open neural networks in system modeling are generally. Well, okay, so the kind of things that you can think of for like interesting applications of neural networks in system modeling, two things always come up on top, which are. One is the kind of fuzzy controllers based on machine learning algorithms. That's an aspect in which we want to look into. But the other aspect in which we want to look into is reinforcement learning, where the idea is again. Well, similarly, the idea is to. I think I have a better example here. Yes. So the idea is that just as the closed systems that Sua showed you, the idea is that the controller in your system could just be a machine learning algorithm. And in the case of reinforcement learning, well, you might have already seen some of these videos of like artificial intelligence playing video games. And the idea is very much that, in the sense that you want your controller to observe the current state of your system and then be able to predict what is the best future move to make in your model. So we are still exploring the avenues that are available here in reinforcement learning. There are already a few good references to check. So these are already in the notebook if you want to look at them. But the example that I wanted to show you today is, is very much connected to system modeling. So the idea is that. Well, one of the standard examples, and that was interesting to find out, One of the first examples that people try to work on in reinforcement learning is the model of the inverted pendulum. So this happens to be also very standard example in. In system modeling. So let me just. Right, so the idea in an inverted pendulum is to. You have some kind of situation like this. You have some kind of mobile object, some cart, and then some pendulum at the top, that is that you can push the cart with specific forces. And then the challenge is to keep the pendulum at the top upright for as long as you can. And the idea in the. Often the challenge is to. Is that you can only affect this, push this car with a specific discrete types of forces. So it's not like you can act with any kind of soft, continuous force. You can use like specific pumps to the cart, and after specific sampling periods, you hope to maintain the pendulum upright for as long as you can. And another. So, but we're going to go beyond that. So another standard example that people use in system modeling, and the one that I want to use here, hopefully if I have enough time, is the model of the ball and beam. So you have. You have just a ball standing on top of a beam. And the idea is that in control systems, the idea is that generally you want to be able to generate a controller such that even if you push the. Even if you put, for instance, the ball off center, you want the controller to act in such a way that it goes back and pushes the ball back to the center so that it stabilizes again. So that's kind of an extended example and that's the kind of controller that you can generate with all the functionality that we already had. But now we want to do a version of this with reinforcement learning. So very quickly, because I'm kind of running out of time. But the idea is that we are going to allow ourselves to bump the beam with discrete bumps to the beam to see how long we can keep them, the ball on top, and in order to create a controller that actually keeps the ball on top. Whereas we're going to train a neural network. The standard procedure to do that is to use actually two neural networks. So one is going to be the actual controller and one is going to measure the loss for the loss function in them in the neural network training. So in this case, we're actually not going to use data that has already been produced. The data is going to come directly from the previous trainings of the neural network that we're training. So it's kind of a recursive process. And the nice thing about this approach is that it can actually be done in the Wolfram language functionality. So I'm going to skip a little bit of the details because I'm running out of time. But the idea is that in the training, In the training, you pass a function that does the sequential training for sequences, steps, and keeps improving the model in such a way that ideally it minimizes the loss and also increases the reward. So the reward in this case is just the reward that we're giving the controller for doing the right thing, for keeping the ball on the top. So I have the neural network also in the files, that's for the presentation. But the idea is that we can do the same thing. After we have already trained the model, we can pass it to create system model and generate a component with that. And that's the neural network that I'm using here in this closed loop system here and now this is what the system is doing. So I'm extracting information about the angle and the position of the ball, taking some derivatives because I also care about the velocities, then holding the information for a sample period, then passing that information to the controller so that it decides which kind of bump to use on the beam, and then to keep doing that repeatedly. This specific neural network was successful in a couple places and not so successful in other places. So there's still work to be done. But at least for instance, here we can see an example in which I place the ball off the center. And then the neural network tried to actually make some bumps such that the ball stayed a little bit longer on the beam than it was expected. Then in other cases, it didn't do as well. But the avenue is completely open, and this is something that we're definitely going to want to look into, and so thank you.