Updates for the Wolfram Compiler, System Modeler and External Language Integration

Hi everyone. Thanks for joining us today. I'm Jamie Peterson, Director of Wolfram U. It's my pleasure to welcome you to the third session in our new in 14.1 webinar series. Let me open up that series page here. Over the past couple weeks we've had a comprehensive overview of the new release and presentations on the latest mathematical computation functionality. You can access recordings from previous sessions right here on on the webinar landing page. Today we're pleased to welcome developers and engineers responsible for the Wolfram compiler, system modeler and external language integration. Tom Wickham Jones is the Director of Kernel Technology at Wolfram Research and he will lead things off for us today, talking about the compiler and related new features in version 14.1, bringing dramatic speed enhancements, opportunities for accessing new technology, and other exciting developments. Next, Ankit Naik of our Wolfram Math Corps division will share key features of System Modeler 14.1, sharing an interesting look at how this graphical tool for modeling and simulation of dynamic systems can be applied to a wide variety of engineering and scientific fields, as well as social systems. Our third speaker is Ricardo Divergilio, a senior software engineer who will update us on the new external object and external operation functions. As always, we have a full schedule ahead of us here, and I'll just say that version 14.1 brings an impressive roster of long requested features and additions to the language, and I encourage you to read more in Stephen Wolfram's release announcement to try the examples from today's webinar. You'll want to access version 14.1 and we're going to share a free trial offer so you can get started right away. I'm also going to ask a poll question. We asked this at each of our new and 14.1 webinars and it's very interesting and informative to us to see how many have already upgraded to the new release. We have a YouTube livestream audience as well, and for those of you on YouTube, please feel free to type your responses, comments and questions in the chat. We have Wolfram webinar and technical staff joining behind the scenes today to help us coordinate Q and A between the webinar and the live stream. So I encourage you to type your questions as we go along. Our staff will answer what we can in the type response to you, and we'll save some questions for the team to address at the end of each presentation. So thanks for answering that poll. Let's take a look at those responses and it looks like, yeah, 79% have already already upgraded. That's. That's great to see. And again, that free trial gives you immediate access if that's needed. There is a download link to today's handout I did. If you have any trouble with that link, let us know by typing in the Q and A. But we've checked that out now. It looks to be working just fine. That download link contains presentation notebooks from all the webinars in this series, and we'll continue to add notebooks each week as we continue on. So I hope you'll plan to join us at upcoming Wednesday webinars. So thank you for your attention. Without further delay, I'll hand things over to Tom Wickham Jones to get started.

Right, good morning everybody. Or afternoon or evening, depending where you are. So my name's Tom Wickham Jones. I'm the main architect, the architect of the Wolfram compiler and I'm going to talk about what's new in 14.1. So I'll just set a little bit of background on what the compiler is. So it's a long term project that integrates and converts Wolfram language into native machine instructions, so it can provide dramatic speed enhancements for Wolfram language programs. In itself, it's a high performance language with unique and innovative features and it gives new opportunities for linking to external libraries and accessing new technology. It's used to build an increasing amount of new Wolfram language features, one of which is the the Wolfram data structure library, of which it's how the data structures are built. So in this presentation I'll give a very brief introduction to the compiler and I'll describe some of the new features in 14.1. So here we go. So the simplest way to interact with the compiler is to use is to write a regular Wolfram function, but you wrap type annotations around the body and then you pass it into function compile, which returns this compiled code function. So when you execute the compiled code function, you give it arguments and it returns the expected result. When I do this, I'm also using the uncompiled function and so it strips out the type annotation and executes. And that's kind of a useful way of verifying and understanding your results. Now, even though it gives the same results, if I give it a large amount of data, then you start to see I'm testing the performance of the function and so the compiled one is getting a very large just do a quick comparison. So the compiled version is getting a very significant speed acceleration, so that's quite useful so in the compiler there's a static type system, and the type system is quite a sophisticated modern system. So one of the ways is it uses these type variables where you can say, in the type annotation, you don't have to say exactly what type something is, you can give it a variable. So here I'm making a declaration, a function declaration. That's the name, this is the type here, and this is the body. And the nice thing about this is the same declaration can be used both for packed arrays of integers and for packed arrays of reals. And so that's, you know, I don't have to write two different versions of the function, I just write one and then it sort of selects which, you know, whether it's reals or integers and figures things out. And in this way you, you get enormous amounts of code reuse. So a little bit of code can go a long way. That's pretty nice. One of the things that we've been working on with the compiler is to build features for the Wolfram language. One thing that it's very well suited for is building data structures. We have this library of data structures that we've created over the years, and we add to and extend and enhance these in every version. And just here's a little demo. So the idea of these is they are, some of them are mutable, this is an immutable extensible array. And if this is what you want to use to build up results. So here I'm appending to this data structure, I'm appending 10,000 elements and just contrasting this with the speed of doing appending to a regular list structure, the packed array, the data structure version is very much faster. So if you want to do this type of building up of results, it can be very advantageous to use a appropriate data structure. So now I'm going to talk a little bit about some of the new features in 14.1. So one of the areas that we wanted to work on is to address the fact that the compiler is powerful, very powerful, but it can be hard to use, and it's harder to use than regular Wolfram language, but it can have useful results. But in order to reduce the pain of the compiler, we've been trying to do some work to improve this in 14.1. And we've done this in a number of ways, with reporting of errors, giving progress as compilations are running, and providing introspection functions to see what definitions and things are available. So let's have a look at some of these. So in the Error reporting. We're talking here about compilation failing. So in compilation can fail, if the types aren't lined up or something, it's going to stop. And in the old days you'd get some sort of error message, but now you get more information available to show you where your error is. So in this function it's got an argument that's a boolean. So we apply the not function. So that also replies, turns a boolean, but then we call a sine function on it and it says can't find a definition for sine that takes a boolean. But now we also have this sort of error markup that shows where the location of your error. So that's pretty handy and it's a nice integration of the compiler with our nice notebook technology. And this goes down a little bit deeper. So even if the the program's a little bit more tangled, it's a bit harder now. So here I compiled something. Can't find a definition for a function, but here is the function there, but it can be a bit harder to find it. And using this source location is kind of useful. Another way that it can be useful is if you have a chain of function declarations. So again, here I'm calling a function, it's going down through some chain and right at the bottom of the chain you see here's the call stack and that's the function where your error is. But this was the thing that you actually entered. And so in the feedback it's showing you this chain, the names of the functions and the their types. So the idea of this is that it can help to track down your errors and see you know where they are and then you can fix them. So another sort of issue with the compiler is so so good thing it's written in the Wolfram language. So we want to make the Wolfram language compilable and build huge things fast correct applications with it. But at present in the interpret, a lot of it runs in the interpreted mode, it's quite slow. Now we're addressing this by using the compiler to compile itself. So that's all nice, but while we're working on that, we've added a progress monitor that comes up by default. So, so now I'm compiling a bin counts function and we get this progress indicator that tells us how it's going long and that's quite useful because sometimes when you run things and then it sort of goes away and doesn't give you any feedback, it can be a little bit annoying. Just to show you this New. This bin counts implementation is new in 14.1. And so I've generated some data and I'm doing a bin counts computation and it's giving kind of a nice. Yeah, there's kind of a nice speed up here. So this is quite good because this bin counts here is already implemented in the sort of internal code of the Wolfram language. It's just that the compiler is a more modern, easier to use and easier to access features like sort of multicore, and that means it can be faster. So another new feature of the compiler is this thing we call compiler information. So the compiler comes with a great big library of function and type declarations, but now you can introspect on these to get information about types and functions. So if you call compiler information on like a type integer64, you get this little box comes up. It has. Since this has a documentation page, you can click on this and it jumps off to the, you know, documentation page. It also has some other information that comes up that we see here. So the integer64 type implements various abstract types, and we can see them here. And we can even see the abstract types have a sort of hierarchy. And so we can use the nice Wolfram Research graph plotting features to sort of plot things out. And you can click on something like the Integers Abstract type. And you can see here all of the types that are using this Integers abstract type. So now another thing is you might want to look at what support is there for a Wolfram language function like Map. And so here you can see various versions of Map. Some of them are for data structures and some of them are for. So here we've got various sort of packed array definitions, function declarations for Map. And again, you can sort of navigate going on through these. So that's all pretty useful. One of the important things about this compiler function is it works with your own declarations. So here I've got the function declaration that I made earlier and I'm adding it to the default compiler environment. So I can do that now, once I've done that, then I can use compiler information to see, you know, to learn about, you know, what's been added and what's available. And this is showing a key point about the philosophy that we have. The compiler is it's kind of an open technology. It's not like some hidden secret thing that only people at Wolfram Research can use. It's a powerful open system that can be used to build innovative solutions, all of which are going to be very tightly integrated into the Wolfram ecosystem. But it's not something that's sort of locked out from everybody else. So that's kind of useful. So you can make your own work and use tools like compiler information to explore them. So another feature that we work on steadily as we're in between releases building up to new release is in the compiler world is to support more built in Wolfram language functions. So super important to, to support functions. And this is a list of the things that we've added support for in 14.1. And some of these actually turn out to be quite, they can use some of the interesting innovative features of the compiler. And so for example, I'm showing you how to use the cross product function. So I'm giving it two real vectors and we can test this out. And this is the compiled version and it comes out running like this. And this is the old legacy uncompiled version. And so we're getting some, you know, pretty, pretty nice speed up here. Let's just have a quick look. And so this is the uncompiled and this is the compiled and this is nice speed enhancements. You might say, well, this is pretty quick anyway, but if you're doing these things in a loop or something, having this sort of speed acceleration is pretty important. And it's able to do this because it's optimizing and using some nicer functionality to get, to optimize the case where there's only two, you know, two vectors. So this, this cross product is like, you know, very general, but it's, but, but the compiled one is optimized for, you know, smaller numbers. You know, this, this particular case, which is probably kind of, kind of an important, important case. One of the things we're looking at now is being, having a smooth workflow to being able to re, implement, you know, deviate the built in function to use a compiled version when, when that's when that's available. Right now another area that we did some work on is trying to make it more flexible, the handling of runtime errors. So these are different from compile time errors. Generally it's much better if there's a compile time error because you see that the author of the code sees that and they can go and correct it. But sometimes runtime errors happen and the default normal behavior of a runtime error. So here I'm taking a 64 bit integer and I'm doubling it. So if I double the integer, the 64 bit integer 10, that's representable as a 64 bit signed integer. But this number here is at the boundary of the maximum value, and when I double it, there's an overflow internally in the compiler, there's some exceptional termination of the program and it exits. The default behavior is it reruns this in the evaluator and you get the result that you see there. Now, sometimes you might not want that to happen. There's a number of reasons why, not least of which is this software arithmetic is going to be slower than the hardware computation that gets done. And so we can set the compiler up to do this by setting the runtime error action option. So we set that, and now when we run this calculation, we get the same termination of the program, but in this case we just get a failure object being returned. Right. Another new feature or a feature that's enhanced in 14.1 is the component system. So the component system for the compiler is a packaging system that's set up to make it easy to deliver solutions to group declarations and things into a, into a component with a name. And what we've added in 14.1 is a way of declaring. So these components, you can go and build a library. One of the, you can build a library from them. And we've set up a way of creating raw library function, RAW exports from a component library. So in this library we're declaring a function. So this is the name, the function declaration that's accessible if you're accessing this component. And here we've set up a raw library function that's going to export, it's going to build a library and it'll contain this function. So now I'm going to build the component and I export it. So I actually built a library and it says that, you know, there's a RAW library function with this type. So one of the ways that this is useful is this library can now be used as a foreign function interface. So one way to explore that is to use the foreign function interface functions of the Wolfram language. And so I'm going to load demo library finding this function, I get a foreign function interface and I can call it. Now, there's other ways number of other ways, including directly through compiled code, to call this library. But the foreign function interface is a very easy way to access the code. It's also interesting because quite a lot of applications provide an FFI interface. For example, SQLite, you can extend SQLite with FFI functions. And so this is nice because we can build an FFI interface from the compiler using this feature. So that's kind of nice. So Another enhancement in 14.1 are some new ways to declare functions that are connected to a specific type. So here I've got a type declaration. So it's a product, like a structure. This is the name, these are the data elements it contains. But it also supports some operations like here's an increment operation. So here I've given an implementation. So this is like simple value and the increment operation, and here's the code that implements the operation. And then in a function somewhere I can access some data of that type and I can call the operation like this. And so this is like adding a method or trait in other languages and the Wolfram data structures are built. We developed this because we found it a useful form of abstraction. You don't need to give all these functions names. It's a sort of neater, more elegant type of way of working with this. And it's quite nice. You can even add these to the abstract types. So here I'm declaring an abstract type and I'm saying it's got an increment operation. And there's. There's the implementation, which I've. I've written there, and I've written this implementation abstractly. So I've just said it's in terms of, you know, it's just something that's an incrementer. So it's got to have a value thing. And here is, I've declared a product type and all I've said is it's got the incrementer abstract type, so it's inheriting the implementation that we saw there. So we can look at this with compiler information. So I'm doing compiler environment append to, and now I can see, for example, the abstract types we have incrementer that we see here, and we can see the operations that are available through this mechanism. So that's something that we found quite useful. Right. Coming to the end. So we've also. I described the data structure library and we have some enhancements in 14.1 in the data structure library. So one of these is, we have some of the data structures, we have a thing called a sorted key store, and we have a sorted multiset. So these are stores and sets that maintain an order of keys, maybe integers or something, but there's an order to them. And we've updated this by using a red black tree data structure that we added in 14.0. And this is showing a very common thing, is that building up this library of data structures is very often one data structure is built out of other things. So that was a useful thing and another thing that we've done is enhance another data structure, the least recently used cache. So here I'm making a recently used cache of size 2 and I'm inserting a couple of things into it so I can do this. I can do lookup based upon a key. So I'm looking up the F1 element and it's returning the value. But now I'm going to insert something else into the cache and what happens now is it's ejected the least recently used thing, which was this one here. So this was put in first, but I looked at it and so it's ejected the F2 version. What we've added that's been. People were requesting this in 14.1 is a way to call a function. So we call this function. So we've added two elements to it, we're going to insert a new one. This is going to cause an ejection. And so when there's an ejection, we invoke this ejection function. And this might be useful. You might be storing database connections in your least recently used cache. When they're rejected, you want to know that they're rejected so that you can call some function. So that's kind of useful. Right now I'm coming to the end and I'm going to have a few minutes to answer questions. And if we run out of time answering questions, there's a mechanism that they'll be stored up and I can answer them by email. So just in summary, I gave a very brief introduction to the compiler and I've looked at some of the new features that we've been working on in 14.1 and trying to explain some of the philosophies and directions, some of our reasoning for all of the functions like function, compile, type, they all have extensive documentation pages. In 14.1 there's a new tutorial on the foreign function interface that we added. So people were saying it useful to have some tutorial on this. And another source of information is there's various videos from Wolfram conferences and this one here is one from the last Wolfram Technology conference. I'll just point out there'll be another Wolfram Technology conference in this autumn. And so if you're interested, come along and join in and you can learn about the compiler and all the other interesting new features that we're building up to in the Wolfram language. Anyway, so that's the end of my presentation. So I'm going to come and see if. Let's come and have a look I'll try to. Right. So I'll run through these questions, starting at the top and going down. So when should I use function compile instead of compile? So I would say always, unless you have some particular legacy type of thing. So compile is an older legacy compiler, it's limited, does some nice things, and it produces not, you know, sort of okay code. But function compile is the modern thing that we are working on. So can I use a compiled function inside a code of a compiled function? So the answer to that is yes. I think there's an example of that in the documentation. You can pass a compiled code function in as an argument and that, that, that. That works. Yes. So that's the answer there. Can I use a package where all the functions are compiled and not need to recompile each time the package is loaded? So the answer to that is. So that's a good question. And you could. One way of doing that is to compile. So let's just make a. I'm going to make a function. You could have a build step for your package. So I'm making a very simple compile code function. But you can take this compile code function here and it can be. You could load it from a file or I could do, you know, print it into a string. Let's do tostring func, input form. So that's the, you know, the input form of the compiled code function. And then if I call to expression on the string, it sort of rehydrates the compiled code function. And this is done. This is possible because in the compiled code function, I'll just. This is a slightly internal way of looking at things. But we have this. We have a bytecode of some intermediate code. It's actually the LLVM intermediate code. And when you run the compiled code function, you can actually compile it, do cross compilation and build it for many functions. So you could store code functions for your package. I think that would be a good thing for us to document. Try to document. Right. Let's have a look at some more questions. Where have the questions gone? There were more questions.

You might need to refresh that Q and A. You want to look under the new area.

I want to look at the ones that. There were more than these at the beginning.

Yes.

We moved the ones you've answered to the answered column. Right.

Tommy's answering, but I didn't see this in the answer.

Oh, sorry.

Right, sorry.

Right.

So I answered that one on Silicon Apple, Silicon Macs. Does the compiled code make Use of GPU acceleration. So the answer to that is not at the moment it uses LLVM and it's accessing the, the cpu. Now if there was a library or something that used the GPU then that would be sort of relevant Right. Now let's look at some more questions. So you're saying I should look at the new ones now and does the compiler generate, multiply and add machine instructions if needed? Absolutely yes. It's generating native machine code. So I could do something. Yes, so I could do function, compile, export. Let's see if I can do this string and give it a function. Let's do typed arg64. So I'm going to do arg1. So this is the LLVM instructions. So this is calling. What have we got here? This is calling an intrinsic. This is how it does correctness. So this is quite a fast thing I think I can do. I'm trying to do this quickly. There we go. And this is the assembler code. So there's quite a. There's a little bit of code here. I think somewhere here there's, I think there's a. Oh yes, here we go. Here's the add instruction and it's checking an overflow flag. So, so it is using sort of the, you know, native instructions. We have some ways that we can turn off this checking and we're looking at making this documented. These can have some speed checked so we can do native unchecked block. And now we've, let's take this out. And so now this is calling a simple add instruction. And if we look at the assignment assembler. There we go. Then this is just calling a simple add instruction. So now another question. True Q. This code could be.

This one is a little more an involved example. It looks like Arbin is answering that already. So there we go. Arvin's provided answer. Maybe not a compiler specific question.

This is not a compiler, but Arbit is answering it anyway, I'm going to be neutral on this because questions like that, sometimes they have. They can have complicated issues and things. But basically actually. Right, so this is. Here's the issue though. This isn't particularly right. So it's unable to simplify this. And the trueq function returns true if its argument is literally so that's going to return true and. So anything else will return false. So that's why it's returning false. I could probably do. Let's do. I do simplify. So right. I mean, so this is, this is what you would have to do. So that's not a compiler question, but I'm answering it anyway.

Thank you Ta.

Right, any more questions from anyone or should we keep them coming? Or there's various, you know, if you send questions to support or something. I'm always interested in trying to see what people are doing with the compiler and helping out.

Right.

And Wolfram U is a fine place to to communicate. We can always share that feedback with Tom. I know Tom has to get going here, but if you have con if you have further questions, feel free to put them in the Q and A. We'll send them off to Tom and get some answers for you after the event. But I guess we're probably ready to move on to Ankit as a presenter. Thank you Tom for your time.

Thank you everybody and yeah, thanks for your attention. Okay.

Right. Okay, bye bye Tom so Ankit Naik is up next of our Wolfram Math Core team and he'll be sharing key features of System Modeler 14.1 ankit. We'll go to your presentation next.

Hi everyone, welcome to this presentation on what's new in System Modeler version 14.1. I'm Ankit Naik, Applications Engineer at Wilfram Math Corps. I have been modeling with System Modeler for the past six years. Before we dive in, let me provide a brief overview of System Modeler. System Modeler is a graphical tool for modeling and simulation of dynamic systems. This dynamic systems can be from different fields. For instance, you can model a large scale solar power plant and carry out techno economic feasibility studies. Or you can model a drone and check how it follows the design trajectory. Not only engineering systems, but you can also model social systems like the spread of an infectious disease and analyze impact of policies like social distancing. In this talk you will encounter various other models that will give you a better understanding of how System Modeler can be applied in different fields. I will be presenting the key features of System Modeler 14.1 that will accelerate your modeling workflows. Specifically, I will focus on five major enhancements. Improved GUI support for managing nested models, Ability to publish models and simulation data to the cloud for easy sharing and analysis, Concise and clear error messages, Ability to configure how errors are handled, and finally, the ability to save animation properties for reuse. System Modeler now provides new interfaces to quickly switch and parameterize nested models. When working with libraries, you might have components that allow you to switch between various lumped models based on your design requirements. For example, in Fluid Networks, you may want to test different flow configurations such as laminar, turbulent or transitional flows. These configurations rely on different models to calculate flow characteristics like pressure drop, heat transfer rates, and flow velocities. With System Model 14.1, there is now enhanced GUI support that allows you to directly access and edit the parameters of these lumped models. Let's open System Modeler and check an example. In this model, if you click on the pipe component, you can see that you can set the flow model parameter. Whether you are testing strictly laminar, strictly turbulent, or automatically detecting flow types based on Reynolds number, you can easily switch among different flow models. Select a flow type and click the Edit button to open the new interface. System Modeler automatically detects all dependencies and provides an interface with default values that you can modify as needed. This makes it convenient to work with library components that allows you to switch among different nested models. With system model of 14.1, we have improved the error messages and provide you the ability to configure the severity of error messages. Let's explore this using an example of a drone model. When we validate the model, system modeler 14.1 highlights keywords and formats the code to focus on key aspects of of the messages. In this case, we can see a warning about incompatible units. Height has wrongly been multiplied by velocity to obtain time. By clicking the link, you can directly go to the programmatic part of the model. After fixing the issue, you can validate the model again. The model validates successfully. In addition to keyword highlighting, there are other enhancements like better styling, more negative space, and improved color coding, making messages clearer and easier to read. Moreover, System Modeler version 14.1 gives you the flexibility to configure how modeling errors are handled. You can set a message to be silent, a warning, or an error based on your modeling needs. For instance, if you are in the initial exploration stage and focused on getting your model to simulate, you might opt for a more relaxed error configuration. In this DC Motor example, unit related errors are treated as warnings, allowing the simulation to proceed. However, when you are ready to validate your model, you can switch to a Stricter setting. Click on the Strictness Settings link. This opens up a window where you can configure your settings. You can set the strictness for things like Modelica compliance, unit checking, and symbolic processing. Let's change the unit violations to error and click ok. Validate the model. The unit violations are not treated as errors and you cannot simulate the model until you fix them. Let's fix the error and validate again. Our model validates successfully. This Easy to read error messages, and the ability to configure errors really comes in handy when you're trying to troubleshoot errors. So far we have covered improvements which focused on the modeling interface. Let's look at improvements that make it convenient when visualizing results as animation system modeler 14.1 allows you to set and save your animation properties such as timescales, default views, visualizer scaling, and more. Let's explore this with an example. Visualizers for velocity, force, and other vectors are essential for understanding your model results. However, they sometimes need adjustments. For instance, in this animation, the force and velocity visualizers are small and hence end up inside the object. You might want to scale them for a clearer view. With System Modeler 14.1, you can easily do that. Just right click Go to Properties, adjust the scales and click ok. The visualizers are now properly scaled for your analysis. In addition to scaling, you can also set other properties such as default camera angles and time scales, and save the settings. This makes it convenient for later reuse or when you share your model results with others. System Modeler is tightly integrated with Wolfram language. With version 14.1, you can now automatically upload models and simulation results to the Wolfram Cloud for analysis or to share results with others. Let's look at an example. Let's simulate the DC motor example. Here you can see how the rotation speed of the shaft and the current in the circuit changes with time. In Simulation center, you can create an interactive slider to check the effect of input voltage on the plotted variables. Let's go to the Explore tab and add a slider. You can see as you increase the DC voltage, the rotational speed increases. Once you have performed your analysis, you can easily share the results with others by publishing the results to the cloud. To do this, go to File, then select Publish to Cloud. You have two options Publish either the Explore panel or the Experiment. Exporting as an Explore publishes the model to the cloud and creates an interactive interface, whereas Publishing as an Experiment uploads all the model results for further analysis. Let's publish the Explore. You can customize the generated URL and set access permissions. Once you are done, click the Publish button. After publishing, you can open it in the browser, copy the URL for sharing, or even use it as a Cloud object in other Wolfram applications. Opening it in the browser creates a cloud notebook with the model and its dependencies automatically set up for you. Let's create a copy and evaluate the functions. This creates the interactive dashboard that you created in System Modeler. This notebook can be shared, modified and used for post processing based on your needs. Your model users do not need to have System Modeler to use it in the wilframe cloud. In this presentation you can you saw the new quiz support for managing nested models and how to publish your results directly to the cloud, thus making your workflow more efficient and your results more accessible. The enhancements in error messaging and the ability to configure severity levels provide greater flexibility and clarity, allowing you to focus on what matters most in your modeling process. Plus, the ability to save and reuse animation settings streamlines your simulations, making it easier to analyze and present your findings. I encourage you to explore these features in your own projects and see how they can help in your modeling workflows. Thank you for your attention and I'm happy to take any questions you might have.

Ankit before we move on to the Q and A, let's go ahead and ask that poll question. We were curious to see how many in the audience already have access to Wolfram System Modeler. So we'll share that poll with everybody right now and that can maybe help inform your discussion a little bit.

Right?

Yes.

Okay. From the results, looks like most of the audience are new to system modeling functionality and that's and to address that I wanted to bring up okay, so can we close the poll? Jimmy okay, you can see my screen, right?

Yes.

Right. Okay. So as most of our users are new to System Modeler and in this presentation I delved into some intricate small details or the or the additional features that experienced users might find more useful for people who are new to System Modeler and want to explore to want to see how it is used in different other applications. In the last release cycle we have added two new courses that would enable you to see how System Modeler can be used in in regular application in different applications. And one of the courses this MOOC on Introduction to Electric Circuits which was released I think last week and here my colleague Leonardo introduces you to the to basic circuit design and analysis and also he goes, he he explains the operational amplifiers. He uses it to create music. So if you want to have an application where you can use System Modeler and learn the tool, I think this MOOC is a really good source but this is related to engineering. If on the other hand you are from a non engineering field and you you want to let's say model Social dynamics, then we had a study group on guiding principles for systems modeling and simulation where we primarily use the concept of System Dynamics by Jay Forrester and using that concept we have created different examples be it like on how the CO2 emissions are increasing or example on how to create a musical instrument. And LEO uses this operational amplifiers to create this instruments in this course. So if you are primarily from non engineering field, this is the right beginner course for you. Right? Let's see if we have any questions. Right. Okay, we don't. Looks like we do not have any questions right now. If you have any questions then the right place to ask about it would be the System Modeler community. Here you can see past questions or. Or you can see different applications which system model users have shared. Like for example how a battery management system for electric vehicle is designed. So along with that there are various other examples that you can find here. So if you come up with a question, please feel free to post it there and we will be happy to answer. Right.

Thanks Ankit. Right, and I think Ankit will be staying online with us too. So if you're. Sometimes it does take a while for people to get those questions typed and submitted. And so we're continuing to monitor that Q and A and the chat field. So feel free to put your questions in as we continue along. But thank you Ankit for that interesting presentation on System Modeler. I will pass things over to Ricardo. I see he's ready to go. Thank you Ricardo. And we're talking next about external languages integration.

Okay, I'm here. Hello everyone. So welcome to my presentation. And so I'm just mentioning that last week I've done another presentation on the same topic where I was going more in detail about what External Evaluate is and how to use it. I'm the lead developer of External Evaluate and basically I know a lot of it, what it can be used for, what it can't be used for. And so feel free to check the documentation is what I am saying today. You are not really following or you want to go into more details because in that presentation I was going into everything that basically externally Evaluate can do. Today we are focusing more on what's new on 14.1. Basically we have done two addition. One is improvement to External Object, which I'm going to talk in a little bit. And then a new symbol that is called External operation, which is something that can be a little bit intimidating but is something that is very powerful and can allow for very dynamic, really highly dynamic behavior or can be used to optimize back and forth between Wolfram System and the external language. So let's get started and see what we have done new for external objects. So unfortunately, until now, even if External Object was introduced a long time ago, the idea of External Object, the implementation probably was never there, meaning that External Object before this release was a MERV wrapper. It's something that the evaluator was returning to you that was giving some information about the object that was living on the other side. But unfortunately you couldn't do anything with it. And what we have done in this release is basically we managed to pull out a big internal refactor and thanks to that we were able to to make External Object much more useful. The most important thing that you can do with External Object, obviously is the ability to send them back to the evaluator. So you get back an external object, which seems to be something that is trivial, but unfortunately it was not. Now you can basically get back External Object and you can send them back to the evaluator and the evaluator will take care of the system, will take care of refreshing the object from the evaluator and use it and give back to you a new result, a new external object, or whatever you were doing. The other thing is the ability to run arbitrary operation, especially attribute, gather and running methods of an object, if there are any. This first slide is going to be a quick overview of what you can do with External Object and then I will go in the next slide in more detail about how the API actually works. So. Another thing that I want to talk about, you have seen this small loading when I executed this line of code is the Evaluator specification. So this is something that we actually introduced in version 14. It's not new in this version, but just as a reminder, normally into the evaluator part you will put it depends from the evaluator. Obviously on SQL is going to be different, but most of the time is going to be just the part of the binary that you're using. So something like user being Python or whatever is the location of your Python interpreter on your machine. You can basically put it in here, but on version 14, specifically for Python, you can put an association in here and there are multiple keys. One is the Python runtime that you want. So for example 3.11, 3.12, the main important one is the dependencies key. The dependency is going to be a list of packages that you want to use during the lifetime of this session. And what I'm saying here is that I want to use the chess module, which is just some random Python packages. That is amusing. It allows you to basically work with a ch, visualize it, et cetera, et cetera and the nice thing about this is that I'm basically not specifying a physical Python distribution on my machine. I only have a declaration of what I need for my external session and in general is going to be a list of dependency and the Python version that you want to use. Once you have done that, you put it inside the association and the system will do several things for you. First of all, it will download Python installation and Python distribution. If you don't have one of the version that you specified. After that is done, a virtual environment is going to be graded that is completely isolated from the Python distribution that you have. And inside the virtual environment we are going to go ahead and install the dependency that you specify so that by the moment the session is started, everything is going to be installed on your machine. Then the session prologue is basically just a prologue that I'm saying import Chess. So that is available for me. But in this line of code there is quite a lot of software engineering in some sense. Actually I would argue that could be better than the built in Python experience, because in Python the good practice is that every time you run a program you will need to create a virtual environment isolated from your Python installation. Sometimes you don't do it because you're lazy, etc, etc, but you will really need to do that. It is a good practice and it will prevent you from bugs, etc. But it's a process that is time consuming. This is not. Basically you just need to write this declaration and we are going ahead and doing all this magic for you. And the other nice thing is that you can simply take this code, move it to another machine and. And in theory will work. I would say in theory, because depending on what libraries you're using, people that have been dealing with Pytorch and other things know what I'm talking about. Sometimes they are just random crashes when you install libraries on different architecture, et cetera. And so things can still go wrong depending on what you're installing. But basically if you're using pure Python libraries or Python that are supported on multiple architecture, you should be more than okay. So okay, this is done, everything is installed, the virtual environment is created. Now we can move to external objects. First thing I'm doing is to create a chessboard. When I'm creating a chessboard and doing chess board, this is just something you need to read the docs for. What is going on is that normally when you do external evaluate of 2 +2 we return the number 4 because there is an equivalent between the number 4 in Wolfram language and the number 4 in Python, they are both integers. So there is 1, 1, 1 mapping. And we try to do that when it's possible. Date, times dates, rational complex number, they all have a representation in word from language and we move that. But we don't have a representation for our chessboard and we probably never have one. But we in those cases, when you have a custom class, what is going on is that we are going to return to you an external object. What you can do with an external object. Well, first of all, I'm sending it back to another function of the same package. This is a function that you run it and it returns to you some SVG code. Let's go ahead and import it directly so that we can visualize what the board looks like when it's created. So this is the initial configuration. Another thing that I can do is that I can ask the board legal moves. Legal moves is going to be an attribute that is documented into this library. So this is equivalent of running board legalmoves. And this is returning to me a lot of moves. Now moves are against some custom object into this library. And so again, I'm getting back external objects. So this is going to be a list of external objects. Now I can use work from language, for example, to get one of them. I would have done that in Python as well. But, you know, just for the sake of showing the fact that you can take objects, do some computation and then do something else in Python, I'm using random choice. Then I'm computing the UCI string for that move. I don't claim to know what it is on UCI, but it's probably just a pointer to the chess configuration. Then I can push this move to the board and when I'm importing the board again, I should see a different configuration now where this piece was moved. Now you can create a bitrari complex program. And this is just to show you that the board object really is like a pointer to this Python system, sorry, to the Python evaluator, that is mutable. And so I'm doing mutation. And every time then I'm running methods, I'm still pointing out to the same object. And so with this method you can do mutation, you can call methods, et cetera, et cetera, et cetera. And so that makes external object incredibly more useful than what it would was before. So let's go down and see what the actual API for external object is to do that I'm going to show something that probably the audience is going to be more familiar with. The datetime module. The datetime module is basically the standard way to interact with date times. Now when I'm doing that, starting the session and then I'm evaluating date time. Now imagine I am into the Python repeating, I import datetime and then I write down date time. Date time is going to be. Now at this point is an object that represents the module. I can prove that by by running the rep in Python. This is how you call function in external evaluate. You put the name of the function on the left hand side and then the arguments on the right hand side. And so what I'm asking here is the equivalent of running rep of that time in Python. And this shows you that this is actually a module that lives somewhere. As I was saying, like the system is installing all this thing into your machine somewhere. And at the end this summer we'll have an actual date time module. And so what I can do with it when I'm doing datetime date, this is equivalent of doing datetime date in Python. Now this is returning a function because datetime date is a class in Python and the contract is in this documented under the Python external evaluate interface. The class is callable and so everything that is callable is going to be a function. And when I say callable I mean the actual built in function callable in Python. So we call callable over the class. A class is something that you can call because it's a class and can be initialized and so is returning to your function. Since this is a function. Now I can send in some data like 2012, 1010 and I get back a wolf from language state object. This is because as I was saying before, externally valid is doing its best to convert known types into Wolfram language types by default. But you can override this behavior by specifying the newly a new thing that is called return type. Sorry, under return type there is a new type that is called external object and you can always force something to be returned to you as an external object. Now one caveat for that is that if you do that, you are basically doing one operation. This is one external evaluate call and this other one is going to be another external evaluate call. What will happen is that while this works, because in this case chances are that when you are accessing the data tripod out of daytime, this is never going to fail because this is. This is not something that got to do anything with file system or network operation or something. But Python is a really dynamic language. Let's say that I either misspelled the date operation or this operation is something that is doing a network operation that is failing, etc. This can return a failure to me. And so when I do that now I will get back involved from a failure and then I will be calling the failure with those things. And that that's going to be just some random weird expression that I'm getting back. So what you want to do is to chain them in one shot. When you do that, you access the data attribute and then you call it in one time. And to do that you use the same syntax for function application that we were doing before. Then you're doing this operation in one shot. And so if this operation fails, this one calling the method will never succeed. So I can misspell this thing. And so now I get back a failure that tells me that can be inspected, there is a stack trace in it, etc. Etc. That at this point the operation failed, this one never gets executed. And so when possible, always chain multiple operations together. Now another way to do the same is to use external function. This is a new syntax. In 14.1 you put the date, time, the object in the first argument of external function and the second argument is the method name. So now this is the same, this is getting the method and then calling it. And this is still done in one shot. So this is one operation that can only fail after the function application. Or you can specify a different return type by using an association. Here you put command goes to date, return type goes to string. Now when I run it, instead of getting back a date object, I get back a string. As I was saying, I can force the return to be external object so that I always get better and external object. This is useful because sometimes you want to access properties of the data of the object that we don't expose. For example, in the, in the Wolfram language options over it. I don't know that. For example, in the, in the daytime that there might be some time zone information you want to, you want to extract. You cannot really do it. You can do it in work from language, but it's going to be different in Python. And there are other properties that you might want to use. The other use case is that for example, you request a gigantic numeric array and transferring the numeric array is going to be. Is an expensive operation. And when you, when you do external object on the other end, you are only returning a pointer. And so instead of transferring back an external numeric array that you are not going to use in Wolfram language, you can simply return an external object and then send it Back when you need it. And this is going to be a very lightweight operation instead of another way to do the same. So calling a function, so calling a method and cast it into a string is by using the symbol string. Or you can use the symbol external object. And this is a shortcut for doing the same. Another thing that you can do is you can chain multiple operations together. So for example, you can do datetime.datetime now and this is the syntax to do it. This is the datum we are starting with. Then we're getting one attribute, then we're getting the function now, and then we are calling the function now with no argument. And this is returning to us. Now what now is doing. So this is more or less the external Object API. Again, if you're interested in it, go and check it out in the other session because I was going into more details, but this is more or less what is new in 14.1 for external object. Now moving to external operation. External operation. As I was saying at the beginning, we managed to pull it off this refactor thanks to external operation. There is something that is internal mostly, but it's something that can also be used and it is documented. It is probably something in you almost never need to use. Probably it is going to be. You will need to use it when you. You either have performance problem, you are doing a lot of back and forth between the Python evaluator and the warframe evaluator and you want to optimize that. You want to reduce the number of code that are done to the bare minimum or is something that you want to use when you want to implement some very highly dynamic property or for examp for when the user is giving you a specification and you wanted to compile it into something that Python understands. So it's really something for advanced use case. But just to show what this is about, when you are running external evaluate of rep, what you're doing is that we take the string rep and we evaluate in Python and these will basically execute is doing the Python eval operation and returns to us the built in function rep. Now this is the short term to do that. The full operation is what we call now external operation which is going to run the evaluation on the other side over the string rep. So this is the full form of it and you need to to think about external operation like a remote procedure call. It is basically a well defined operation that is implemented over the evaluator that we know nothing about on the workflow side, but we basically have this abstract representation. These Symbolic representation for this remote procedure call that we send to the evaluator. And so basically when you are asking, when you're writing this, you really are writing, you really are writing the following. Internally, this syntax where you're doing rep goes to two. That is equivalent of calling a function with the number two in terms of external operation. Is this beast here is going to be, first we evaluate the string rep, then we are calling the result of it with the number two. And again, there are all kinds of operations. There is an import operation that we added, there are other operations that we have to augment, and they are all different for every evaluator. In most of the case they are the same. For example, the evaluation is shared between Node Js and Python and Ruby and other things. But for example, the import operation is something for Python is going to be a required operation for node, etc. Etc. And so what we are doing here, for example, is importing the date time, getting the attribute date, and then calling the result with 2012, 1010, which is something similar to what we have done before. And internally we are really converting the operation to this guy and doing it in one shot. So this is going to be a complex remote procedure call that we package up in terms of external operation and we send them to the other side. And so, for example, when you do a date time, this is another way to do the import stuff only by using operations. Now inside this object there is the same stuff that it was before. I can actually do that again and stream here in order to see, this is the daytime module, just to show you that when you are doing date today, and then you're calling it, and then you're casting it to a string in this small piece of code, what you are actually doing is the following. We are getting the attribute date, then we are getting the attribute today, then we are calling today with no argument, and then we are casting it into a string. So basically those are operations that can be composed together, that can be nested, and you can write arbitrary complex programs with it. And this is a way to basically package up multiple operations that previously you needed a lot of back and forth on the evaluator. Now it can be specified in this symbolic intermediate code. And then again, this is something that probably most of you will never need to use, hopefully, because we really don't want you to write this thing. Also, for your own mental sanity, you can write this. And this is equivalent because as I was saying, we are packaged up in multiple operations, that this is done in one shot. This is one big operation that is sent to the evaluator. So it's pretty efficient, like this one. But again, in some cases, when you want to interact with a module and expose certain functionality in a way that the user is asking for, external operation might be a good way to implement those kind of things. So I think I am mostly done Take a look at the. As I was saying, take a look at my previous thoughts. Take a look at the documentation for external operation, External object. Also Wolfram client for Python is an open source technology that it's worth taking a look if you are interested in in. In Python specifically because it exposes communication between interaction between warframe language and Python. It's an open source library we are using internally, but that is used internally for. For this system. But it can also be used by you in order to call a kernel from Python. But we are using it on the other way around. We are using it to call Python and transform things into expression. The old implementation is here. And again I think it's time to answer questions. So let's see if we have anything today. 2. Can the notebook show a matplotlib figure in line? Right? Yes. I think when you do show, it should actually show and. And display a new window. When, when you're running? Yeah, when you're running show, I think it opens up a window with the. With the image the same way Python is doing. But there is a way to basically export the image using pill and when you return a pill image, automatically we display the image. So basically if you are able to convert any kind of image into a pillow image, then basically we are automatically converting it into an image for you. Okay, let's see this other question. Sorry if I missed it, but if you don't specify Python version, does it get the latest version and if I evaluate the network that is it reuse that version or do I get an update version? Correct? Okay, so yeah, good question. So basically there is a Python runtime that you can specify. So let me go back to my example here. I'm writing this, but you should be writing this, you should be writing. You should pin down the version and you should pin down also the versions here. This is because basically we create the environment on your machine at that moment. PIP is doing some resolution. If you don't specify the Python runtime, we are getting the newest Python runtime we have at that moment. I think at the moment is 3.11 on the system. We are going to upgrade to 3.12, possibly in a packet update or something. For now, we are only returning at 3.11. But basically you are correct. Every time you run this code, you get back the latest Python runtime that is available. So this can create a bug. And this is part of the. So when you really want to write something like this that is portable, it's always a good practice to pin down the Python runtime and to pin down the dependencies as well. Because what can happen, for example, now numpy move to 2.0, something that happened, I don't know, like a couple of months ago. And so if you have code that was simply saying numpy at the moment where you got the installation, once it's installed, if you never touch this dependency thing again, you get what you get. You get what was installed on your machine at that moment. When someone says executed, they are going to do the resolution again, they're going to get newer version, etc. Etc. If you don't specify versions explicitly. And so, and so, yes, you will need to, you really need to be in down the version in order to make sure that your software will keep working after a few months when major versions are released, versions are broken and yeah, all kinds of things are happening. Another way to do that, the proper way to do it is to basically run pip freeze and make sure that basically you got the full resolution under this expression. And we are working on making this thing simpler so that basically you can freeze the specification and then share the freeze version. But yeah, yeah.

Ricardo, I'm not sure if we kind of got your audio or your webcam frozen there. Is that happening for others too?

Yes, yes, I'm seeing the same.

Okay, well, no worries. I think that was actually the last, the last question that we had. Are you back?

Yes, yes.

Oh, there you are.

Okay.

Yeah, maybe the one switching off the webcam so that we are seeing some better. How long I was out?

Yeah, just. Just a couple seconds. You answered that okay. Okay, yeah.

Okay, great. Okay. And so. Okay, no, if there are no other question, I thank you everyone. And that was all. Feel free to check the other webinar. I did. And yes, reach me out, especially on the GitHub repo. If you have any other question or something, feel free to open them. And yeah, thank you for joining us today.

Thank you, Ricardo. We appreciate your time very much. Let me. I guess we are right up at the end of our time. Thanks to everybody for your participation today and for our staff behind the scenes. Let me just remind you about our upcoming webinars in the series next week we will be looking at visualizing and computing, astronomical and chemical data and historical geography. We have Jose Martin Garcia and Jason Biggs coming to Share with Us 3D visualization of biomolecules, astronomical graphs and computation. And then this making historical geography computable, which is really, really a fun and interesting new area. And after that, on the 4th, we'll be covering new and powerful components for AI and LLM. So we look forward to those developers joining us. Then a survey will come up on your browser. It allows you to request a certificate of attendance for today's event. And we look forward to seeing you next week. Thanks, everybody. We'll see you then.