Cell Tray Widgets

Hi everyone. I can't see you. My name is Jason Abernathy and I'm a front end developer with Wolfram Research. I'm mostly dealing with the Linux front end, so if you have any issues with Wayland in the next version, send me an email or send it to.

Bugs.Wolfram.Com.

Yeah, but that's the only thing we're going to talk about. I'm here to talk about a new interface we've developed for making cell level user interface elements and we call them cell tray widgets. So I'll get started on that. You guys can read the abstract in your free time. So why cell widgets? We kind of have a little bit of a real estate problem in the front end when it comes to adding controls for presenting information about cells or manipulating. We do our best to hide that problem by making sure that we only have things present in your cell bracket, like one at a time. I had to try kind of hard to find a case where I could get two things to appear at once and it was difficult to reproduce for this presentation. So I just took a screenshot here. We've got kind of, I don't know if you can see my mouse, but we've got a cell bracket on the right side of that image and we've got a cell error icon there. And then we've also got the minimize predictive interface there.

At the same time.

What happens when you click there is kind of tricky. If you're like just on the edge of the error icon you can get the error icon thing to pop up and otherwise you get the interface, the.

Predictive interface popping up there.

So yeah, we'd like to put a lot of things there. It's easier to have more responsive interface or an easier to use one if.

You, you can have context specific controls up when you need them. So.

And also so, you know, aside from the real estate problem, you know, if we want to let users like you guys be able to extend this interface and develop your own controls, we have to make that a little bit easier. As a front end developer, you know, as a developer you get access to like the source code and you can add your own hooks in for events where you want these, you know, user.

Interface things to pop up.

And when you want to dismiss them, you know, you can kind of see the secret behind the curtain. But you know, if you're just a Wolfram language developer, you don't have access to all that. So we need to improve, make it.

Easier to extend and add to these kind of Interface elements.

So that was another problem to be solved this and we came up with a olution called the Celltree widget interface. And it's kind of a. It's a toolkit that encapsulates those things so you can create those user interface elements and instead of having to handle and add those things and choose when they appear, you can kind of let the front end position and, and create them and destroy them as needed. So it really takes a lot of.

That work away from the.

Interface developer.

And hands it to Brendan.

Ok, so what do these things look like?

Let me see if I can get some to pop up here. Don't want to evaluate all initialization cells.

Here we go. So we have some kind of. In the front end. Now they're not all activated. This is kind of like a gear widget that Kevin, the host of this talk, is he developed. If you use Wolfram Cloud, you're probably, probably familiar with this. We've got one in the desktop printed now we've got another widget that edits cell tags and another one that does call out for the, the SKU interface. And whereas before it might have been difficult to have all these things appearing at the same time, now you can see that it's possible to have them all up at once and the front end kind of manages the placement of those things. So, yeah, so that's what they look like, these widgets. There's a little bit more about them here. So they're kind of like attached cells in a way. So they are cells, except they're cells that have names. So each widget has its own name and the properties of the widget can be modified by they have properties and those properties are kind of assigned using key value notation inside an association. So yeah, so they're kind of like attached cells with like more customization and so let's. I'll talk a little bit about how you use them. There's kind of like two paradigms, if you will, for how they can be.

Kind of like grouped.

There's kind of like two basic workflows. So one is kind of like an automatic one where you use, I call it the automatic workflow. So you kind of like tell the front end like this is where you know what the widget is and user conditions under which appear for like basic, you know, cell styles. So like all input cells should have like, for example, a gear icon or all output cells should have like an evaluation kind of widget kind of thing. But there's also an interface where it's more like the traditional kind of manual, you know, I want a widget on this cell here kind of interface. So and that one, instead of just saying I want all input cells to have a widget, you would say like, I want this particular cell to have this, a different kind of widget. So two kind of workflows to get this to work. And I think they kind of would work together depending on what kind of, you know, widgets or user interface kind of design you're up to. You know, there's a little bit for, there's something for everybody. So if you're doing like, you know, like you're spell checking a document, you might want to say, oh, I'm going to attach my spell checking widget to all of the cells that have a spelling error, for example, or if I'm.

Doing code analysis, then all my cells that have like a code error, I'd attach a code correction widget or something to.

So that's where you might want to use a manual one. Whereas for automatic you might just want.

To say like all cells have a gear widget kind of thing.

So I'll go through kind of quickly how to use each one of those workflows or give a brief overview of how they would be used. So for the automatic one, so this is where you want to say, like all output cells have a gear widget. We have a new cell level option. It's called Cell Tray Widgets, aptly named. And this is where you would throw in the kind of widgets that should apply to a given cell and each one of those cell specifications. Inside the option value you can have a condition property that can explain when.

The widget should show up.

So we haven't really talked much about how this would look done any code yet. So here's a little bit of code. So we've got a cell here. We've got the celltrade widget option value. So it takes an association, and each key in the association will be the name of the widget and then itself has a value which is an association, which is all of the properties for that widget. So it's kind of like a nested association structure which as it turns out.

Later will be useful.

So we have a predefined gear menu cell widget, and if I print the cell out, you'll see that it comes automatically with this gear menu attached to it right there. So that's kind of like using the option value, how you would get these things to show up. And so I mentioned that there's a Condition property for the widget. It decides whether or not you actually want the widget to be present. Which might sound kind of silly. You might be like, well, wouldn't you just place the, you know, place the key there if you want the widget to be present and then remove the key if you want it to not be there?

Well.

I'll show you, I think in the next slide why that's useful, but let's just show how condition works. So if it's false, it's not there. It's true if it's there, you know, but. So why would that be useful? Well, we can get with cell styles to where you might want to keep the base level options for certain cell types. Associations kind of participate in inheritance where if you have, if you have a value with a key, then that and then another association with, you know, with the same key it the windows styles are combined, it'll try to merge those two association values as well. So it's kind of like an override mechanism. So if you define a widget in kind of like a base style. So if I call my style and it has a, has a widget with a particular condition and some other properties, but you want to turn it off and inherited style, you can just set the condition of false, you don't have to worry about. And then, you know, later on you might want to set it to true again and then you don't have to worry about blowing away all of your inherited values. So it's kind of the way that cell styles kind of combine in the front end. You can do that with associations as well. And that lets you kind of use this condition to, you know, say like for, you know, my base notebook, all output cells will have a gear menu, but maybe, you know, in the presentation view I want all of my, you.

Know, gear menu widgets to turn off, for example.

So that's one paradigm kind of for this automatic loading and unloading of widgets.

That it's pretty useful there.

And then is if that wasn't useful enough, the condition option or property value. You can also take Dynamics. So, so we have the symbol X, we set it to true and then we create a cell widget, you know, a gear menu widget. We can kind of, we can choose when it appears and doesn't appear by.

Clicking on the toggler we created for that symbol.

So if you want to have like say a toolbar at the top of your, you know, in like a doc cell, for example, that enables and disables widgets, you know, that's kind of how.

You would get that to work.

Okay, so that's kind of, that's kind of the idea of getting the widgets to show up automatically. And for the manual method, this is kind of more like the attached cell interface. So it's more developer user driven where you evaluate a function to say attach you know, this widget right now and.

I want you to attach this particular element or this particular cell object.

So right now the function that does that is in the FE private context. So I'm just going to kind of for this presentation, wrap it with top level function. But if you were going to do this in practice, you probably use this for now, full FE evaluate run in the end evaluator this private front end.

Function.

And the signature is really similar to the attach cell function. So the first argument will be the cell that you want to have the widget attached to. And then the second one will be just be like the name of the widget, it's a predefined one. So here we're saying attach the gear menu to the cell that was just evaluated. Actually you can see how this widget here is placed to the left of the existing suggestion R1. So you can already see how this is an improvement, whereas before they might be overlapping. Also that second argument, you can provide.

Properties for the, for the widget that you're attaching.

So we don't have any, haven't shown you any properties yet aside from condition, which doesn't work for this kind of manual interface.

So this is just build the ignored. You see it pop up there?

Okay, so you know, attaching the widget might be useful if you do it manually. Well, how would you remove it? Well, you can access to the widget from the return value of the function. So it gives you a cell object which is kind of like a handle to the widget and then you can use that anywhere that a cell object.

Is, is accepted by, you know, for a function argument.

So I could change the background to orange, for example, or I could just.

Say get rid of it and it goes away.

Another way of doing automatic deletion of manually attached widgets is using removal conditions and accepts all the same arguments that it accepts. So, you know, we'll attach the widget manually and we'll say give me a gear menu. But if you click anywhere outside the.

Parent cell disappears, I click down there and it's gone.

So that's kind of the two ways of using attached cells. Either kind of, you know, let the front end kind of manually Add them and remove them, or you can kind of manually add them yourself to specific.

Cells and kind of give the.

Either give it conditions, go in and delete itself, or manually delete it yourself. Oh, more on that. So if you want to, if you're nervous about, you know, re attaching the same widget over and over again, it's okay. It's idempotent by default, which means you can evaluate it multiple times and it won't add another one unless you say.

I give me lots of them. So.

Okay, so that was kind of how you use them. How do you actually describe what's in a widget and what it does? You know, where's the sauce? So to speak. And, you know, we had a couple ways of using them. So we've got three different ways of defining the content. And, you know, each one is useful in its own way. You can either produce widgets and store them in notebooks that the front end will load when the front end starts. You can register widgets with an evaluator function, or you can override the content.

For a particular widget.

So for the first method, when you've got notebooks containing widgets, we got this directory called Private pathswidgets. And the front end, actually, not just when it loads, but whenever this option value changes, it will throw away whatever it's got cached for widgets and it'll.

Reload them from the notebooks in these directories. And.

What do those notebooks actually look like? Well, they just contain cells. Each cell will just have some content in it. And then the important thing is that it'll look for cells with a single cell tag, and then that cell tag will be interpreted as the widget. And so I've got. Can you see that if I open this window over, the existing one does show up. Okay, Kevin's saying no. Okay, what happens? Okay, so I was going to show you a notebook containing widgets. Maybe at the end of the presentation I'll change my share so it shows up share entire screen. Yeah, I could do that.

Let's see.

Or you know what, maybe I'll just share one.

Right click. New share. There we go.

Okay, so this is the notebook that opened up, and this is a couple.

Of the widgets that Kevin has developed here.

So they just look like cells. They're just regular cells of content in them. And the front end will load them up and then as they're needed, it will fully read. It'll just read the stubs in, and then as they're needed, it'll fully load.

The elements and freeze. All right, let's switch the screen back. Okay.

Okay. So the second way of telling the front end what these things are is annually registering them. So if you want to have a widget that's only valid for one session of the front end, or you're, you know, maybe you're dynamically generating widgets on the fly, you know that, you know, you can't just store statically in a notebook. So we've got this register widget function here. Again, it's high behind fe private might throw that in a global context one day, but for now, this is where it is.

And.

So this is how you use it. So you register a widget, you give it a rule, it's got a name.

And it points to the content there.

And the content here should be something that you would find inside a cell. So it should contain, if anyone's familiar with the term boxes, that's kind of.

What it's looking for there.

Or it can be a full cell expression. So like cell, you know, like something like that. It also accepts strings. So strings, cells or box expressions.

That's the kind of thing that the front end will recognize as being a cell.

And so that's what I've got there.

I've got a string.

And then once the front end knows about this, this new widget, you can.

Add it just like you could. Did I do that? There we go.

So, yeah, it's my face widget. It's just a smiley face. And now it's just sitting right there. So this one will override. So if you register widgets manually, they override what's in the notebooks. If you don't like the regular gear icon, you can roll your own and just kind of like use the.

Use the same name.

And then the front end would load those instead. And then there's one more way of specifying the content, and that's with the content property. So say you have a widget, you just want to change the content of it, then you can just provide the content property. And then again you pass in some.

Sort of like box structure there.

So here we go.

Here I've got it.

I'm saying I'm going to do the face widget this time. I want to inherit all of the properties except for the content. I'm going to use this shock one. So that works for the single target, the manual widget adding mechanism, as well.

As the cell tree widget option value there.

Ok, we got a couple other properties that are interesting. So we've got different types of widgets. We have focus widgets that only appear when the. When the cell that they are attached to has focus. And what that means in the desktop front end is when you mouse over the cell. I think maybe, I don't know, it might be a good idea to say focus also happens when your selection is it as well. But right now it just mouse over. So we're saying the gear menu is a focus one, so only pop up when the cell is focused. And the reason it's below the smiley one is because focus widgets kind of have their own row. So if we had more than one.

Focus widget, it would be beside the gear icon there.

You can also sort them. So if you say, hey, my widget's the most important, you know, put it in a place where everyone can see it.

We have sorting value property value too.

So the lower numbers are closer.

I'll brag there.

Okay, so in the last couple of minutes here, I'm just going to show you a quick how to for developing a widget. The basic idea is that you want to keep your minimized or inactive state small and then just kind of pop out something bigger when someone interacts with.

You like a mouse over or by clicking on it.

And then if you've got controls in your widget evaluation, cell should give you the widget itself. So if you want to change the content or the background or whatever, use that. If you want to get the cell that you're attached to, use parent cell. And then one thing to note is that if you're going to use.

If.

You'Re going to pass properties to widgets, associations will evaluate their rules. So if you want to hold it.

Use rule delay for your association element.

Ok, so how about a cell that shows the modification time? A widget that shows the modification time. So I threw in a little bit of an appearance here. I made a little clock with three dots there saying, hey, there's maybe some more information here. And this is cell change times is an option to cells and it gives a list of when the cell has been modified. We can grab that value and massage it around and find the latest time. And then once we have that latest time, we can kind of convert it.

To a human readable format.

Otherwise it'll just show the minimized icon. And then how I decided to handle the minimize state and the maximize state is just really simply with the mouse over behavior. So if the mouse is not over it, show like that little clock icon. But if the mouse is over it and show that human, human readable format. Okay, so that's what my widget's going to do. So I've got my pain selector here. I've just thrown it kind of agonized into my two boxes form.

So it's going to make some, a.

Nice box expression for the content of my widget. I'm going to call it my edit time widget name which is probably not sounds like you can edit the time, but it just shows the edit time and yeah, print that off and there it is there and I mouse over it and I get modification time which looks, looks wrong. That's like 10 o' clock at night.

It's not 10 o' clock in Canada.

In any part of Canada right now.

So probably a bug there somewhere.

But you can see if I change.

Here it changes the modification time so it's dynamic. So 37 seconds now 42.

So anyways, you know, kind of a lot to cover there but if you want to go back and read the presentation you can kind of see a.

Little bit more in depth into how that was created.

But yeah, so that's the end of the presentation. If you have any questions, let me know or suggestions, just feel free to send them in and yeah, look forward to seeing what people do with the interface.

Thanks.