State of the Front End

My name is John Foltz, I'm director of User Interface Technology at Wolfram Research. Thank you for coming. And what I'm going to be talking about today is just it's kind of an overview of the front end for 2023. It's a bit of a potpourri of things. So there's aside from definitely features about the front end, there's not totally connected thread through these things but I do highlight with just with a little dingbat icon there the slides that reference features that some of them are in 13.3 already, some of them are in 14, which new to 14, which I believe is available in pre release form or should be soon. And then there are a few things which are not. They're in 14, but the design isn't finished yet. I'm going to cover those too because there's some really interesting stuff that is going to be is going to be front and center in the spring release that we do. But you can already see pieces of it today and I thought people would be interested in looking at that. So let's get started. Okay, so one of the really interesting things that happened in 13.3 was the package editor got an update. So we introduced in 13.1, I think we introduced a toolbar for default notebooks, a very updated modern toolbar. And we ended up doing the same thing for the package editor. It's not as dramatic. Let's see. Nope, that's the wrong window. Here we go. This is a package editor in this case. I just happen to have opened a copy of Gitlink, a package that I happen to have worked on with Ludy Andrea and let me just bump the magnification there slightly so you can see across the top and sorry, it's going to be a little small on my screen, but there is a new toolbar here. The buttons on the left are useful but straightforward for running the whole package or for interrupting for doing cell styles and some selection manipulations, extending selection, commenting, uncommenting. Same thing that you would see for the code sections in the main toolbar. But on the right here we have something very interesting going on. One of the things that can be kind of a challenge for the package editor is navigating your code. And so we have a code navigator up here. And so if I click in it, if I do nothing but click in it, I get basically a full outline. Once again, I apologize for the size here, but I get a full outline that I can click through and I can open and the kinds of things that it has. It has all of the sections, subsections, so on cells it has. It has various definitions of global variables. The global variable has a little X next to it, functions which have a little F next to them and various metadata that you might have about the functions as well, which I think I'm not going to find immediately. So you can navigate. That's interesting. Gitlink's a pretty large package. Navigation is. I don't know, maybe I might want to take a look at the test section or something like that. But really what I'd want to do is be able to search. Let's do that. I'm going to search for the git got a little confused there. I'm going to search for the Git add function and boom, there we go. Just as easy as that. I have option settings, that's one of the metadata bits. You could also get used messages or something like that. I have option settings for Git Add. I have all of the various function definitions. And so let's say I'm interested in the first one. I click and boom, here I am right here. And I've got my definition right there. And this updates super fast. If I make a change this say change this to Git Add XYZ and let's go ahead and do a search again. And there we go. GitAddXYZ. It's already updated. If you have large packages that you're editing and you're trying to keep on top of, certainly as I do, I'm bouncing around between functions. Sometimes I just want to see their signatures. Like I might look here, just be able to see them in the preview. But sometimes I need to maintain code by going back and forth between different parts of the package. Having the structure with the sections and subsections helps, but if you have a lot of functions, you just have to be able to navigate. And so this is really useful for doing that. And it also can surface things that are difficult to find. Like I have 4 git add definitions, but I need to find where the options are and so on. So if you write packages in the notebook front end, this is a really useful tool if you don't write packages in the notebook front end. And maybe code navigation is one of the reasons why this is something to consider. So that is the package. Just some reminders about the package editor in general. Package editor is opening up plain text files. It preserves the plain text ness, but it's giving you the notebook structure. We have a VS code plugin which is really handy for being Able to do things like go to definition of functions. But if you want to be able to work in in the notebook front end, I think that we have a much improved mechanism for doing that now. Okay, Okay. Code editing. So this is something Package editor change, by the way, was in 13.3. This is something that's also in 13.3 and it is the. Oops. So the idea is that normally if you type in regular Wolfram language code and you type in full symbols and you get completions like you would expect based on what you've typed so far. And that's fine, nothing wrong with that. But we now have fuzzy matching for completions too. This is very popular in programming editors where you can and often sort of the way to access it is it tends to key if you have multi word symbols, it tends to key on letters that are the beginning of each of the words and the symbols. In case of Wolf language is very often done using camel case. So if I type in. And I don't even have to Type in lowercase PL3 or CPL3 for complex plot 3D or you know, oh, I intended contour plot 3D. Well, it's right there. And this makes typing symbols particularly long symbols, which there are a few in Wolfram language. And this makes typing long symbols a lot easier. Or if you don't remember exactly what the words are, you can kind of skip a few words and get them in there. So when I'm doing notebook programming there is a lot of long functions notebook programming. So if I wanted to do notebook read of evaluation notebook EVNO and everything just works. It's a lot less typing. I press tab to accept the completions and quite often you can navigate with arrow keys through the completions. But quite often when I'm typing, I type in just enough that my preferred function will pop up to the top. This works for user defined symbols as well. Now you have to actually create them in the kernel. So if we have going to do an evaluation here and now I can do. And there we go. And it's quickly figured it out. Let's do something a little more complicated. Let's say something like. And if I'm using camel case in my own symbols, that's going to help make that an actual notebook. So that makes a little more sense. Okay, so now I can do impno. I could have done even a bit less, but now I've got my completion. These completions are also context sensitive. And so if you. Let's just make a quick plot here. And now we Want to do something like. Something like plottheme pltt and it knows because I'm in the option position to be prioritizing things that would fuzzy match to options here. For this particular set of characters, there's only one and I press tab and there we go and it goes right into the completion for plot theme. And so then here I can pick my poison. I can do plotstyle, plts, say dashed and plp plot points. You know, how am I coming up with these things? It's just, you know, once again, it's just picking up, you know, a couple letters of each of the camel case until you find the thing that you want and boom, there we go. So now I've produced myself an astonishingly ugly plot. I apologize for that. Now this is something else that should be said. I've been doing all lowercase, but you can hint it by typing uppercase as well. If you type in an uppercase, the uppercase letters will only match to letters in the resulting symbol. So if I were to do. Let's see. Well, I am in was enough to get to Important Notebook as the first choice, but it gave me kind of several other fuzzy match things too. But if I type in say a capital in there, now there's only one hit and that's Important Notebook. Okay. This can be controlled in the preferences dialog and the settings right here. If you like it, you leave it on. So that's the fuzzy matching for auto completions. Also there's list only the case sensitive matches for auto completions. So if you like the idea of fuzzy matching, but you don't want it to be case sensitive sensitive, that's a thing that you can control as well. Okay. Okay. So something else in the. In the code editing genre is you can. This is something that we added to 13.3 as well. Is. And it's a small thing. You might not even realize that it's happening. We've had a bracket flashing for a long time when you type in matching brackets that auto match. But if I delete that auto match and then I type in something there and you see that little flash on the left that's been there forever. What we have now is. Here we go is editing or flashing for ampersand. So type. The Ampersand does that. Why would we do that? Because Ampersand is kind of tricky. What does the ampersand cover? Sometimes it's not totally obvious. And here's. This is actually an example of a bug. I've typed in an option and the option I want to have a function on the right pure function. But I typed the ampersand and it's telling me, oh, wait, this entire rule is actually part of the pure function. And did you intend that? Did you not intend it? Front end doesn't have an opinion about that, but it's letting you know what the scope is. And so now if I instead open the paren there and retype and now it flashes the expected thing. Here's just a couple of bits of code that I've written in the last few days randomly. So here's an example where it's flashing quite a large section. Now, it only flashes if there's a slot ish thing in there. And of course, that's because you wouldn't want it flashing if you were doing something like that, where you were really trying to type double ampersand. But it turns out that this has kind of an added benefit. Here is a thing, something I actually was typing yesterday, some bit of code that I was typing. I have here a function, and it's my intent for this entire thing to be a function which is mapped over this. But in fact, when I type the ampersand here, nothing flashes. And so, whoa, I intended a function and nothing flashed. That was assigned to me as well, because, look, the slot was outside the scope. The ampersand only reaches as far as the equal sign. And so the lack of a flash actually was useful here as well. So now if I add my parens and I retype that ampersand just so you can see, now it correctly shows the whole thing. And of course I got my syntax coloring. This is now green instead of pink or magenta or whatever it is as well. Okay, a couple other smallish things, nested action menus thing we added for 13.3. It's pretty straightforward in terms of implementation. Look it up in the documentation. It's basically a menu name with the rule and list of items. Or there could be some menus as well. This is a pretty trivial thing from the examples. Here is an amusing. An amusing example that I put together where it just literally reads the menus from the system and then does them. And you can see one of the main reasons I did this was just to show kind of, you know, you can have arbitrary nesting levels. So if you do control programming, that's definitely something that you want to be aware of. Want to talk a little bit about pixel shader effects as well? If you watched my talk last year, you'll know that I talked about drop shadowing. I think I may have Even used this example, the drop shadowing as a form of highlighting works in graphics. That is a pixel shading effect. We also have blurring. And here's a similar example. And there we go. Here's an example where I've just. Where I've blurred everything except for one particular thing. 13.3 we added haloing. So I put together a manipulate just so you can kind of get the idea. So here's essentially an empty haloing here. So the halo has a glow at an edge. And so here is. Well, it's a radius, but I think of it as a glow. The documentation doesn't use the word glow. Documentation uses word radius. So the. So there's kind of a hard edge which is the first color and a soft edge which is the second color. And so here I have the hard edge and then actually in this case I made the hard edge negative. And so then things move inside, but then I can move the hard edge so that it flips to the outside. And so that's haloing. What would you want to do with it? Well, it's pretty easy to make outline type of effects. Works well with text. Useful for emphasis. So here's. This might be a better choice than drop shadow. And. Definitely it produces sort of some neon type effects here. And so if that's the sort of effect that you're looking for, then this is a thing that you can do. Okay. And nothing to show on this slide, just some bullet points. Wayland. If you use X11, our X11 front end, you know that Wayland is a thing on the Linux platform and it is. X11 has stagnated for decades and Wayland is its replacement. It's not without its own controversy and it has some limitations. But we've been working very hard on dealing with Wayland specific issues and making sure that the front end can behave as well under Wayland as can be expected. There are some limitations imposed which are kind of curious and handcuff us in certain ways. But we've been spending much of this release cycle working on Wayland and we're considering Wayland supported now. Okay, so now we're on to the experimental stuff. Notebook assets so iconize. This is something that has been around forever. The idea of embedding data in a notebook. And so this is just, you know, it's just this evaluated thing, you know, now embedded here. And I'm not going to use that one. I'll just use the one I already copied and pasted earlier. So here's, you know, you just copy and Paste it, treat it as a thing, and there we go. It's a blob of data. It's as if all 100 random numbers were there. Asset allows you to go, but this has to be in the input cell. And Asset allows you to go further, attaching an arbitrary expression to a notebook. So first of all, we have no assets in this notebook. Right now I'm going to add this is just a copy. This guy up here, I'm going to add that. And now it's giving me an asset object. But everything's here in the experimental context because things may change between now when we release in spring. But this is something you can play with in 14. And now there's my full list of assets. And now I can get the value, there's my value and I'm just going to get the same numbers there. And there we go. And so the idea is now you're attaching something once to the notebook instead of maybe copying and pasting a bunch of times as an iconized blob. One early experiment we've been doing with this, and forgive this little thing, this little bit of code is necessary in the pre release. It won't be necessary in the final 14 is I can basically have images that are backed by, rather than being backed by image data, they're actually backed by essentially a copy of the file itself. So here's the thing that I have jpeg, just a random jpeg and I'm going to import it and image has this undocumented. But this is part of our experiment that we're playing with, the ability to bring it in, work with it as an asset. If we look behind the scenes, there's no image data here. This is a very lean thing and that's because the image data lives in the asset. It has a reference to the asset object, but it's not actually image data. It's the JPEG file itself that is in the assets. Now we can see we have two assets. One of them was my named asset and the other one is this anonymous asset that was created by image. So that's an experiment and it's very good about the size. It's pulling it in as a JPEG rather than image data. So it can keep things quite compressed and in basically original form. Ok, let's see. So. And there we go. Okay, so where are assets going? Ok, so certainly some of the obvious examples, attaching moving audio files, having those be out of band with notebooks has been a problem. Embedding original data files rather than the Post Process Data Creating Multi Notebook projects so imagine a book that's split into separate notebooks, but it's delivered in one file and then and then also being able to attach or ship code as part of your notebook. So these are certainly use cases we have planned and we're looking into other use cases as well, but these are some of the prominent ones. I've run myself pretty close to time and so I'm not going to go into a full demonstration here, but I should say Notebook Assets. There's going to be a talk about it immediately after my talk. Please go to that. Ian Hoschnicki is giving that the other thing that I wanted to highlight is Diffing. We're working on a thing called Diff Tools. It's in the Packlet repository right now. So actually you can really just look at what's in the Packlet repository. It's called Diff Tools. It's published by Wolfram, by the Wolfram publisher, and installing it is as easy as doing this. I already have it installed, so it takes me no time. But if you do this in your session and this works in 13.3 even. But it is our intent to publish or to design this functionality to be built into a future version. Hopefully the version that we're going to release sometime in late spring of 24, we're working very hard on it. If you've been watching our live streams the if you've been watching our live streams, Diff Tools has been actively reviewed in our live streams. The asset functionality hasn't shown up there yet, but I imagine that it will at some point. And Lou d' Andrea is going to be giving a full talk on Diff Tools on Friday and I just want to end advertising some talks. Ian Hochnicki is going next. So if you're interested in kind of the power of notebooks with attachments with assets, you can do that right now. Tomorrow our graphics team is going to be giving talks at 2, 30 and 3, And you two is doing a deep dive on the shader language stuff and James is giving a note of our general graphics functionality and then we have a whole wave of talks on Friday. And here's yeah, I'm not going to go through the full list, but you get an idea. There's a lot of cool stuff I didn't get a chance to go over as well, and especially some really new and interesting things coming on iOS if you're an iOS user, so that's kind of fun. And this is not principally a notebook front End Talk. But it's 3D game development that takes place entirely inside notebooks. The games themselves running inside notebooks. So there's some very. And that's kind of a showcase of how all of our technology gets used. So I'm going to leave that on screen and I'm going to see if there's any questions I can hand off. Oh, I should say the whole front end team has an office hour at 11am Central time on Friday. So if you have any questions I probably won't get a chance to deal with much today. If you have any questions, please bring them to the office hour. Would love to talk to people who have short or long questions. Would love to have conversations with you. Okay, so I'm just going to go through some questions here. Okay. So there are some questions about the asset functionality. Yes. There's some issue of, you know, can assets effectively be shared, reused in multiple places inside a notebook, but referencing the same asset? Yes, absolutely. Assets versus tagging rules. Putting things in options is putting huge data in options really can cause some performance issues. And I wouldn't want to be putting a 900 kilobyte file into an option if I could avoid it. Or megabytes and assets are intended to scale, potentially to be quite large. Okay. And I think I'm going to stop there. I'm sorry, there are a bunch of other questions I'm not going to get a chance to cover, but I want to make sure people get a chance to go to other talks. So thank you for your time. Have a great day. Enjoy the Wolf and Tech Conference.