Live CEOing Ep 701: Language Design Review of System Modeling & Control

Hi everyone. Welcome to another Wolfram language design review. This is going to be a pretty quick one, I hope. I think, I think so too. Several live streams today and this is the first of them. All right. System modeling.

Yeah. So there's only one new function and this is called System Model Calibrate. And that basically calibrates parameters in a model with data that enables you to build what's called a digital twin.

Yeah, yeah, yeah. Let me ask a general question. I mean the, the business about things like making entities for System Modeler, that's, that's not part of this release, is that correct? No. Is that moving along or is that stalled?

I think it's stalled. We need to reinitiate that.

We really need that. And in the world of LLMs and so on and that whole flow, I mean, I assume that that.

So the Wolfram Alpha people need it is what you mean, right?

Not really, that's. No, the point is we can use it, the flow that we have. We are increasingly teaching LLMs to deal with entities in Wolfram language.

Yeah, but this is not a repository of ready made models. This is a repository of building blocks for which you, from which you can build.

I understand that, but I mean, to give you an example, you know, we had an example from Nick the yesterday, something about the quantum framework. Okay. Where using a. So Christopher built this thing that is a way of making custom endpoints for LLMs. Okay. So this was, I think done using an endpoint specifically that knows about our quantum framework. Okay. And you could describe a quantum circuit in natural language and it would make you a quantum circuit and then be able to compute from it. And I have to believe we can do the same thing for system modeling.

Yeah, right now you probably could. I mean, because if you can build a programming language to does it, you can certainly describe it in text and there is a context.

Right. You can describe it roughly and it will give you starting points, which is, I think, what, what is really useful and then you tweak it up using, you know, potentially using System Modeler or using code. Does that make sense? Yeah, I think it's really an important thing. And I think that the, the enabling glue, if we had entities, we could just do it immediately because we already.

Think so, but we'll take another crack at it.

I mean, this is what we had hoped to do right. When we started working with System Modeler, was to get a flow from Wolfram Alpha through System Modeler. This is now a much richer way of handling natural language descriptions. Anyway, let's Come to what we have to do specifically now. Boy, this is complicated. Wow. We don't usually do that of having lowercase first letters. And names like this. No. Usually tend to uppercase them.

Yeah.

Okay, we can change it.

Yep.

All right, so let's see the example. Wow. Why is this. Okay, so here we've got a model and here we've got. Oh, I wish you wouldn't do this. So putting these multiple things in one cell. Please don't do that.

All right, I fixed that.

Yeah, I think. I mean, unless it breaks how our example thing works.

No, it doesn't. You're usually the one that wants it in one cell. Just because copy to click is easier, but.

Well, okay, fair enough. But maybe it should be in one cell.

I'm not very keen on it either. I don't particularly like multiple things in a cell.

Okay. All right, so what are the two numbers here for inertia? What does inertia two mean?

It's a. The modeler of this example used that to call one of the parameters in the model.

Yeah, but that parameter is a two vector or something.

In this case, that's the variable that we are trying to fit.

And the parameters. I'm guessing. I don't know the details of this model, but I'm guessing there is an inertia one and an inertia two.

Okay, but then the first element looking at the big time, is that correct?

Yes.

Okay, is that obvious from this? Is time always a parameter in these things or could it be something different? I mean, could it be.

No. So the system models always simulate things over time. The parameters are parameters, so they don't vary with time. In which we don't call them parameters when they vary with time.

Okay, but Inertia 2 is a non variable parameter. Is that true?

Right, That's a variable. That's a variable that changes in time.

So you're answering different things there. So this is a variable, not a parameter.

Yes. And the last ones, the other ones are the parameters.

Oh, yes.

Sorry. Okay, So this is a variable and it appears so. What are you trying to compute here? What is the difference between this model and this model?

The output model has calibrated parameters. They are consistent with the data. So when you build models, if you need to use a resistor, there is a parameter called resistance, but it has a default value. All the components in system model has a default value. And I don't.

How would I look at this thing? How would I look at the parameter values? Should I say parameters or something?

Yes, you can ask for parameter values, but first you need to load What? Yes, you can ask for parameter values, but first you need to load the system modeling functionality.

Why? What do you mean I have to load it?

Yeah. So the symbol.

What do you mean I have to load it?

The symbol on its own doesn't have to do this.

Something like this.

Yes. Yeah. Why wouldn't that do it on its own?

We haven't fixed that. Yeah, it's.

We'll fix it.

Yeah.

Okay. The other thing that you haven't fixed is this needs to have an autocomplete. If I give that picture. Right. See that? And I start typing here, that should give an autocomplete menu.

Right.

I looked into that.

Yeah. Why is this taking forever? Okay. All right. So this has inertia dot. I don't see an Inertia 2.

Because these are the parameters.

I misspoke, Stephen. It's a variable sort of things over time, parameters and.

Do I say variables?

System variables.

Okay. All right. So now in what sense is this. So. So then you're calibrating resistor 1R and damper spring. Damper one, whatever. I mean, it would be nice, wouldn't it, to have some indication in the icon like. Like for a neural net. If I just say, you know, if I say something like, you know, net model of Lynette.

Well, you can open it up and system modeler has all that full drill down.

Okay, let's look at that. Okay, so, I mean, because what this shows is that's an uninitialized neural net, which is essentially.

But it's still initialized. It's not uninitialized. It is initialized. This is just wrong.

Values modeler. Yeah.

You know, so people build what we call do physical model building, but they may not know how those parameters that they use. They might even use symbolic parameters relate to the real world parameters.

Okay, but this, again, doesn't really have that information, right?

No, I thought it would show, you know, the components, you know, the diagram as opposed to the icon. But their icon diagram was the same thing. So there was nothing more.

Right. I guess. I guess you get the tooltips over the components, but that's a little bit of information.

Yeah, but they don't give the values. I mean, whatever it was that you were setting.

Well, system modeler is the tool for that that we sell.

Yeah. Okay. That for setting. For doing what? For determining all those parameter values.

I mean, if you want a. A GUI where you can drill down and know the exact structure where you can build it with a gui that is the tool for that.

Right.

It's not mathematically.

I'm just trying to understand in the obscurity of the fact that I have an input here, I have this thing here, and I appear to get the same thing back.

No, I. Well, that's the default thing. You know, it's kind of like many things where we have many of the things that we can, you know, return. So you can. The, the spec up there, you can tell you can get other things like quality measures. You know, you'll see that often. You want, for instance, confidence bands. How is this fitted model reproducing the data?

Right, But so here the default is model to model.

Yeah.

So the default is that we create a.

Excuse me, why is this blank?

What was blank? I mean, there was a progress mark.

It did nothing. Was doing nothing.

Okay, can you try again? I think that's.

I can't interrupt. Okay. Should I just try it again?

Yep.

Why was it pink at the beginning?

Yeah, there was something.

Yeah, I can look into that as well. So the default is that we make a copy of the model with the new parameters.

So.

It'S like getting a formula out.

Yeah. Why, why is it conniptioning as it. Why is it kind of. You can, you can make it, you know, you see what it's doing? It's got the little pulsification there, which happens when the foot. When the number first comes over. You need plot range padding to prevent it from. From having that little conniption when it, when the number first comes into range. Did you see what I'm talking about?

Okay. Like an adaptive range.

No, the frame should not move. Right. Did you see what I'm saying? The frame should not move.

All right.

As it was. It was wiggling around. The frame was wiggling around. If you ask the machine learning people, they have a solution to that.

Okay.

Okay. And this picture should look really like the machine learning picture. Am I making sense?

All right.

Yeah. I think these fonts are a bit big. I think, you know, if, if you're setting more stuff. Okay, did you see what I'm saying? That it wiggles every time. Did you see that? Do you see. Do you see what it's doing?

All right, okay.

Do you, do you see what it's doing?

Yes.

Yeah. Okay, so it shouldn't do that. And it's doing that because it's leaving more space when a number first comes in and sticks out of the end. I see what I'm saying. That can be fixed with plot range Padding and that steps had some kind of backtick after it, I believe. Just details. But we go. Right.

Okay.

So if you scroll down, Stephen, to the next example there. See that? Here's a different output there. You can see that you get a confidence bands.

Yeah. As a function of time.

Yeah. I think that's the money shot.

That's.

I think that's the most important thing people want. But there's. As you could see up there, there's a ton of other things that you.

Can get so calibrated system model. So let me do that for this case. So that is saying instead of these. Right.

Not at the end. So because that's like a property, I.

Can'T get it from the model itself, I can only get it from the calibration process. Is that correct?

Yes. Right. No, this gets you an object where you can get everything.

Why is this autocompleting wrong?

Because it usually needs the documentation build before it can do derive the auto completions.

Okay, So this is. Why does it have to appeal? Let's take a look. Why is this. I mean, the loss. What. What is it now doing? It's not showing an increasing range there. Why is it. I don't understand the steps. Why is it keep. Why is it still running? Do you understand what I'm asking?

Yes. I think it's just.

So.

I don't think it's.

Okay.

It crashed. Brilliant. It crashed. Crashed the kernel.

That's what it was busy doing.

It's not great. Okay. All right, let's take a look some more. Your thing model predictive controller.

This is. There's no design change. It's just faster and better algorithms.

Okay. Obviously, we need to know why it crashed here. Obviously. Okay. And then, I mean, this thing, gosh, has a million options, which I'm never going to understand.

No, those are properties. There's a million properties. There's very few options. And we probably will. But that's part of another discussion. We should have, you know, kind of introduce subsections in notes so you can have options and properties.

I think we should do that. I think it's time.

Isn't that nice?

Yeah, it is nice, but let me understand what it is. So, as a function of time, that's showing the data and then it's showing. Okay, so those are values of inertia 2 as a function of time. And then this is showing what the model is doing. But I don't understand what the bands are doing.

Those are uncertainty for a function of time. It's like if you had an around function.

But the parameters. I understand in the end this thing fitted certain parameters. Yes or no.

Yeah. And so, you know, so there's some uncertainty in those parameters. Now, what happened to the result when you solve those differential equations, when you have uncertainty in some of the parameters that goes in. That's going to lead to a whole slew of different solution trajectories coming out.

Right, I see, I see. So this is the envelope of all those solution trajectories.

Exactly. Yes.

But is it all 95. Confidence level.

Confidence level. I mean, it's interesting to do all too. That's a. That's a different kind of problem, which is.

Yeah, I get it. It's nice. It's very nice. I still want to use this stuff for something. My. How similar is this in the end to the machine learning framework, you know, with validation sets and so on?

I mean that, you know, the machine learning framework is similar to the regression framework. Right. Because it sort of is. Right. And this is similar to both of those because it sort of is. But what's different here, which we haven't tackled before, is, is that you're minimizing an integral where you have differential equation constraints.

Yes.

You know, in the other cases, we'd minimize the sum with algebraic constraints, typically.

Right.

If, if, you know, machine learning usually doesn't even have constraints. Right. They just. I think, I think machine learning doesn't typically have constraints.

It has, it has some. I mean, it tries to use things like normalization layers to not have things blow up. It's a different approach.

Right, right. And you know, so, so this is, this is an example of dynamic optimization or sort of in, you know, another level of optimization, if you like to do this.

Right, but so, so for example, the use of the term loss here, has that always been used in control theory or is that new to post machine learning?

No, no, no, it's been used.

So that was. So machine learning got that from control systems.

Well, yeah, this is, this is part of it.

It's not what you say in statistics.

Yeah. But they're kind of all the same thing. So this part of, if you like control is, you know, does have a name. It's called system identification. And it's a big kind of field in its own. Right. But that field itself basically picked, you know, almost everything up from statistics.

Indeed. But like the term loss. Yeah, I had not really heard that term except in machine learning.

Oh, yeah, no. So I was in a group where we were sort of world experts on system identification and we used loss Since.

Okay, but that's not the term in statistics. And statistics. It will be residual or something like that, wouldn't it?

No, I mean, you use loss function there too, because you vary the loss function.

Okay.

You know, and it's the same guy, Stephen, like Trevor Hastie was a statistics guy that became, you know, the machine learning guy.

Yeah. I mean, I understand it's one of these. It's all the same thing in the end.

Right, right.

So can we in fact use this. Here's a question. How does this relate to, like our reinforcement learning efforts and things?

That's, that's a little, you know, it's a little bit a different angle, which we want to tackle as well. I think that there are two kinds of things that we can do with re. You know, with respect to reinforcement learning. So for reinforcement learning, we need an environment where you can learn from.

Sure. But you have one. And system model, because you have the actual.

Exactly. So I think that that is a use case. Okay. So. So we need to package it up so that one can take those things and kind of learn from.

Right. Remember, what you've got here is your model here is differential equations.

Yeah.

In typical machine learning nowadays, the model is a bunch of neural nets or a neural net.

Yeah.

And so in principle, you could have a version of this, you know, calibrated, whatever it is, where.

We could do also in a way, I mean, interestingly, we could. For this very function here, in principle, I could do, reinforce, you know, sort of, I could fit differential equations to differential equations. Okay. So if I have, instead of finite measurement data, maybe I have a really high fidelity physical model. And now I want to calibrate a much simpler physical model, but I want to get the best one, the one that approximates it the best in this sense that I defined the loss.

Yeah, right. Well, okay. I guess what I'm asking is, is there a way with our frameworks to say instead of there being a differential equation inside of the system model, there's actually a neural net inside of the system model. And system model calibration calibrate would basically train the neural net instead of the ode?

No, but formally you can include a neural net in system models. We added ONX support. So if you can represent your neural net as an ONIX neural net, then it can be part of a system model.

Which we can supposedly export to, but exactly a bit of a floppy format, so.

Right.

Okay. All right. Well, looks good. I mean, I. I think it will be really worthwhile to understand this plumbing, you know, given that we have a, you know, well developed neural net. Machine learning.

No, no, I mean we. We are aware and we are talking to each other repeatedly. And so, you know. But this is. This is an important use case that this addresses, as you can imagine.

Yep.

Here's another cool thing which I'm sure that you thought of, but you're going to call it something else and I'd be keen to hear what you call it.

Yeah.

Okay. So you. Let's say that I'm observing a system. I'm taking measurements from some of its variables. Okay. And I run the system calibrate. That will estimate and correctly calibrate some of the parameters. Maybe what I really wanted to find out was the parameter, because that is what. It's something internal to the model, like viscosity inside of the system that I couldn't measure.

Sure.

Okay. So this is a way to build a sensor. In fact, many sensors work this way that you compute your way to.

Yeah, I mean, whatever they're called. Virtual. What are they called? Even virtual sensors. What are they called? There's a name for that. Yeah.

Yes.

Synthetic sensors or something.

Synthetic sensors, I think they're called. Yeah. So that's.

That.

That might be itself. Something interesting. Right?

That is interesting. That's quite interesting. And it's. Yeah, that's quite interesting. All right, well, cool stuff. Okay. I don't think there's any more. We need to go over with this right now.

That was it.

Yep. Okay. I'll Roger. I'll see you in another half an hour or something at another design review.

Okay.

Okay. Thanks. System model control systems folk.

Thank you.

Good work. Okay, see you. Bye.