Implementation Story: The Notebook Toolbar

Okay. Hi everyone, I'm Noah, I work in the UX and UI teams here at warframe. And I'm talking about the Notebook toolbar and how he implemented it. So it's actually implemented entirely in top level Wolfram language code. So this is how we did it. So I'll give a brief overview of the toolbar and then sort of up front we knew that there were going to be kind of two main implementation challenges, you know, trying to implement it in Wolfram language. And so I'll review, you know, what features we have available, what boxes, what might help us, and then how we can use those to solve those challenges. Okay, so yeah, just a quick overview of the toolbar. As I said at the top, the UX of the toolbar. In other words, you know, what we put in there and why that was covers in a different talk. So if you're interested in that sort of look, look that up on the, on the schedule. But yeah, so here is, you know, here is the toolbar here. And so if I, if I narrow this window then you can see that the controls, they're kind of being lighted away and then if I click here, they're in this little pop up. And so these, these really were the two challenges. First of all, just how do you implement, how do you create this overflow system, how do you implement that? And secondly, how do you align these pop ups with the buttons in the doc cell? Because they're two different content areas, so it's actually not trivial. And then one other thing I'll point out quickly that will be relevant for later is that you can see, so if I'm in like a text, sorry an input cell or evaluatable cell, we have some controls here. And then if I'm a tech, and then if I'm in a text cell, they change to like text formatting controls. So anyway, that will be relevant later. Okay, so the first issue is overflow. So let's first sort of, you know, define what we want. So discrete elision of a row of items. And so for example, I've got some messier objects here. And so as I resize this container, they're disappearing sort of one by one. Right? And that's what we want. And then we also need to know which items have been elided so that we can put them into the overflow menu. Right? So if I click on this, then we've got the items that were to the right of that. And the issue is that there's no pre existing overflow box that does this so we need to implement this ourselves. So, okay, what tools do we have available? So we have things like panes and pane selectors. And you know, you can use a pane to sort of constrict content and wrap content, an expression. And then pane selector allows you to kind of dynamically show one of several panes based on some state variable. We then also have the initialization option to dynamic module. And so when that box is first rendered on screen, that code will fire. And then sort of related, there's dynamic wrapper, which is when that box is on screen, it's updating things constantly behind it. We then also have current value, absolute current value and mouse position. And basically these are ways of querying various states of the front end. And these are very powerful, as John mentioned the other day. And then finally we have attached cells and this is going to turn out to be quite important. So I'll just go into a little bit of detail here. And so the idea is that you can take a cell and you can attach it to a parent, and the parent might be another cell or a box within a cell, or indeed just the notebook. And then we have arguments for controlling like the alignment, the positioning of the two of the attached cell with relation to its parent. And so, for example, here I'm attaching this cell to the evaluation cell on the left. And so you can see that it's just attached over the evaluation cell and returned the cell objects conveniently. And then here I'm attaching it instead to the right of the actual notebook. And so you can see here that it's not actually, it's not attached to any cell. It really is pinned to the right of the notebook. Okay, so bearing in mind those. Sure. So they can't be attached to doc cells. They, they can be attached to boxes within Doc cells, but that was new in 13.131.13:1. Yeah, so they can be attached to boxes inside DOC cells. But that was Union 13.1. And you've touched on the alignment issue, which is that when you can't attach it to the doc cells, you have to attach it to the notebook. Where'd you put it? So that. Yeah, so going back to overflow, what you might think to do is, ok, well, if you knew the widths of all the individual items, then you were tracking the window width. Well, you could work out by adding them up sort of which one should be hidden at a certain time. And you could do that with a pane selector and just dynamically switch them with like Zero width spaces or something. And so if you're interested, the details and implementation are in this section. So download the presentation and check that out. And basically what you find is that because there are lots of dynamics in this, as in explicit dynamics that we wrote in this solution, those create inefficiencies, right? And so for example, there's, you know, first of all there's a dynamic wrapper which is going to fire every time the windows resize. And then there's a dynamic in each item to know when to switch it. And so the moral of this story is, can we do this without using dynamic? We minimize the numbers of dynamics we have to use. And so as I've said here, you know, we want to exploit boxes that don't rely on dynamics to achieve their effects. So okay, in terms of elision, forgetting the actual sort of overflow menu at the moment, but just in terms of elision, line breaking would work perfectly for that. So we can use pane and then the various options to make sure that we're only seeing the right bit to just view the first line of items. So in other words, they will line break away from the end. And if we can just view the first row, then we've kind of solved that problem. And we can do that by just setting the right sort of image size of the pane and the alignment and scroll position. And so this is what I'm doing here. So I've got a pane around this row of items. These are the options I just talked about. So the line breaking, scroll positions, et cetera. And. Oh, sorry. So I should have mentioned that the front end tries to do line breaking in a good way for sort of codes and math. And so line breaking adjustments has to be used to kind of get the desired line breaking. In other words, one thing off the end at a time. And so if I evaluate this then, and put it in this resizable container, then we have these items and I've highlighted the pane so you can actually see what it's doing. So at the moment I've just, I haven't constricted it to the first line. And so you can see that they're line breaking away one by one. And then if I do constrict this to the first line, then you can see that they're disappearing one by one and you don't see anything else. Okay, so that's good. But we, yeah, So the scroll position. So if you think about it, if you, that there are two lines and you can only see one of them. So if you put the mouse over and then you scrolled. Exactly. Yes. You're freezing the scroll position. Yeah. So this doesn't solve the overflow problem though. And so. Okay, how do we think about this? Well, instead of a row of items, start with a row. Let's start with a row of overflow buttons, right? So let's say I've got five items. So I'm going to start with five overflow buttons. Okay. Then I'm going to cover the overflow buttons with a row of items. And you don't need the last item to begin with, but I'll come to that later. So, okay, so this is the row of items. These are the row of overflow buttons. I'm going to cover the overflow buttons with the items. And now I'm connecting the nth item to the n +1 overflow button. So what does that mean? So in other words, overflow button five, I'm connecting item four, right? And then item three is connected to overflow button three, etc. Etc. And so if you think about this, what happens is when I resize this 5, overflow 5 is going to break away, but because 4 is attached to it, it's going to take it with it and reveal the next overflow button. Okay. And so if you look at the top row, it's. It's almost sort of what we want. And then of course, the final thing is, well, you don't need an overflow button on the end of the row, right? Because there's nothing to the right of it to put in the overflow button. So you just replace the last overflow button, in other words, fifth one here, with the last item. And so then this is the structure that we have. And I'll explode this vertically so we can see what's going on. And so if I kind of just half hide this and then go through that again. So I'm going to drag this in. Five is going to line, break away and take four with it. And then that's going to reveal the first overflow button, and then so on and so forth. And so if I completely put that back, that's the behavior that we wanted. Right. Okay, so how do we actually construct this? Well, we can do it with attached cell, because we want things that are sort of hovering over other things and attached to other things. Right. And so the items are the things that need to cover the overflow buttons, which means that the overflow buttons are the parents and the items are attached to them. And so we can Use this by using the initialization on the dynamic module, right? So this is an overflow button. And the initialization that we want is to attach the item that's to the left of it right to it. And so that's what these alignment options are doing. Ok, so download the presentation, look at the implementation details, which I don't have time to cover now, but essentially, if you run, you know, if you run this section, this code, this code, then I'm defining a function called construct, and that's building our sort of our construct, our little toolbar thing. And then again, I've put it in this resizable container. Okay, so let's see what happens. So I'm going to resize this. And yes, they're breaking away and we have the overflow menu, as expected. But the problem is, is that attached cells hover over everything. And so although the parents have indeed been hidden, the attached cells are still there, and so they're still visible. And so, you know, this almost works. And it contains no dynamic updating after, initially, if, after the initializations fire, there's no kind of dynamic tracking of anything. But we can still see those attached cells even when their parents are hidden. So we could increase the line spacing as a hack, because if you think about it, right, this is line breaking onto a new line, right? So if you just increase this to some enormous number, so it was always off the bottom of the notebook, that would work, but it's a bit of a hack, and that's actually what I've parameterized out here. So this argument is, is the line spacing between these, these lines. But, and so, for example, you know, okay, if I put in 100, then this appears to work, right, because all those attached cells are way off the bottom of the screen somewhere. But that is a hack. However, if the containing interface is known not to exceed a certain height, so for example, you might be in a palette that you know is always going to be a certain height or never bigger than a certain height, then this is a perfectly reasonable solution. And of course, the toolbar is only one line tool, so the line spacing solution works just fine. As I kind of alluded to at the beginning, the docked cell area really is a separate content area. So it will sort of clip attached cells. They won't bleed into the notebook content area. So this, this is fine. So, okay, so what am I doing here? So I'm making a notebook and then I'm adding a dot cell and then I'm. And in that dot cell is the Construct we just made. And then all I'm going to do is just dynamically control the height of the cell so we can kind of see what's going on. So if I run this. Ah, I didn't run the code section. Oops. Right, let's try that again, shall we? Great. Ok, so if I resize the window, then great, this is working. You know, this is working as expected. And then if I increase the size of this cell now we can see those, those attached cells that were hidden. So comparing this to the Notebook toolbar, and I apologize, it's a bit small, but this is, this is how the Notebook toolbar is working. So you know, I'll align some items here and then if I change, if I do what I did previously, if I change the, essentially the height of the toolbar cell, we can see that actually all those controls are just hiding there. And it's. And remember also that all of the items, they're just attached cells. So it's surprisingly easy to break the Notebook toolbar. If you just enumerate all the cells in a notebook that are attached and then delete them. You can see that they were. All those controls are actually just attached cells and you can see the overflow buttons behind them. Okay, so because we, because we've got the solution that doesn't contain any explicit dynamics, we've kind of freed ourselves up to use dynamics elsewhere so we can have some, we can do some heavier lifting elsewhere. And so what you might want to do, as I showed at the beginning, was dynamically hide and show some items based on a certain context. So for example, you know whether the insertion point is in a text cell or in an input cell. And so the way you do this, you take your toolbar here and then you put it in this dynamic wrapper that's going to update, that's going to track some state variable. And then you can just use pane selectors to swap out an item for a 0 width spacer. Right? So if this state is like X, then you'll see the item. If it's Y, then the item is hidden by a zero spacer. And then similarly with the overflow buttons, so there's just, you have to just be a little careful when you're doing this. So suppose we Want to hide item 3. Now what you might think is that, okay, I want to hide item 3 and therefore I also need to hide the, the overflow button underneath it. So in other words, overflow three. So if I hide both of those, if I collapse them to Zero width panes. Okay? And then see what happens. It doesn't quite work. So five breaks away, takes four with it. Four breaks away. And if I just kind of put this back here, we're left with just an end because we're left with that zero width overflow button on the end. So what instead what you have to do is you have to ensure that you always have these pairs overlapping. You have to maintain the structure. And so if I want to hide item three, I have to hide the thing it's attached to, so I have to hide overflow 4 as well. In other words, the thing to the right of it. And so if I do that, so this is going to go to zero width and this is going to go to zero width. Now if we see what happens, everything works. So five breaks away, takes four. Yes, these are on the end, but they're zero width. You can't see them. And so we're just left way. Oops. And so we're just left with the next overflow button which peels off as before. Okay, so that's, that's, that issue solved. So second is pop up alignment. And so the reason why this is a pain as we were talking about, is that the dot cell area and the notebook body area are completely separate content areas. And so for example, that has, you know, an effect on, that has consequences for attached cells. So if I attach this cell here to the doc cell of this notebook, right, you can see that it is completely confined to that doc cell and it doesn't sort of bleed over into the notebook content area. And if I do the same thing to the notebook content area. So if I attach this, this blue cell to this area here, then we see the same behavior. So therefore you must attach pop ups to the notebook body. Now of course, this is only relevant if you want sort of rich pop ups that you want to put in an attached cell. If you just want action menus as we were using before, then obviously that's fine. So what this means is that, yeah, these attached cell pop ups, they must be attached at notebook body so that they're visible. But then you've kind of lost your alignment points, right? So pop up gets attached to the evaluation notebook, but then how do you align it with the button that it's meant to be sort of spawning from? Okay, so again there's sort of a more straightforward but perhaps but suboptimal solution which is, well again if you, if you, if you knew all the item widths, you could just pre calculate everything, right? So you could pre calculate all the horizontal positions of each control. But this has, I mean, this actually has sort of several disadvantages. One is that it might not quite be the same on each operating system. You also have to know the overall offset of the entire construct because you need to add that to all those spacings. So it's not great. So what can we do? We can use mouse position differences. So if the. Whoops. Okay, so let's say this is, you know, this is a control here in the docs cell. If we know the mouse position when the mouse is clicked relative to the notebook, which is this blue line, and if we know the mouse position relative to the item, which is this red line, then of course, if we subtract the red line from the blue line, we get the left edge of this, of this button. And so we've now got an offset that we can attach that we can place that pop up. Okay, so how do we get these mouse positions? Well, you can just. There's this mouse position event handler absolute. And so you can just wrap your control in sort of an event handler that doesn't really do anything. And so that's what I've done here. And so you can see that, you know, when the mouse is outside of the event handler, it's just none. Then inside it's giving the mouse position relative to the event handler. And then to get the mouse position relative to the window, it's view. Absolutely. Okay, so you can then kind of generalize this to two dimensions because of course mouse position is returning a two dimensional coordinate. And so if you subtract again, just subtract the event handler absolute from view absolute, then you can get the bottom left hand corner. And so that's what's happening here. So the top arrow is view absolute, the bottom arrow is event handler absolute, and the difference of them gives the blue arrow which is pointing to the bottom left corner of the event handler. And of course this works wherever this is on screen. Right. So you can just kind of locate this wherever it is. And then finally you can also get the size because of course you may want to attach to the right edge rather than the left edge. And so if you just take the quotient of the absolute coordinates over the scaled coordinates, that's going to give you the size of this, of this event handler, which then you could add to this coordinate here. Yes, if you want to implement this yourself, there is just some things to look out for. So have a look at that section. Okay, so we've solved both problems. So in summary, we identified sort of the two main challenges that there were with the toolbar sort of up front and then which were overflow and the pop up alignment. We reviewed the, you know, what features we had, what tools we had and realized that, okay, so we need to reduce the numbers of dynamics to make a nice efficient interface. And then we achieved that by using the sort of layering of attached cell method. And then for the pop up alignment, what we really found actually was just a general way to locate and size items when the mouse is over them, as long as they're absent event handlers. And we did that with, you know, various combinations of different relative mouse positions. Yes. This is a really fun project to work on and I just want to thank the team. So Kevin, for a ton of implementation, Lou Agawan implementation, John, Brett and Path for UX as well. Any question? Ah, so if I just go back to that. Where was it? Here we go. So it was here, right? Yeah. So the idea is that this, this is literally just the row of items and the line breaking off one by one and then we're only doing the top row. And so that's, that's fine for getting the elision behavior, but the problem is we also want the overflow menu. And so how do you know which items have line broken away and so how do you know which ones to put in the, in the overflow menu? And so you could do that by like tracking the window width, for example. So you could kind of make like a combination of two approaches. But then you still have a dynamic that's constantly running. It's better, but it's still not great. And so that's why the layering just does everything for you. Exactly. Yes. Yeah, yeah, yes. And so the key thing there is the line breaking adjustments. You need to set it to make sure that it's line breaking away as words, words off a line of text one by one. Sure. So they will their parent, although their parent is hidden, their parent hasn't been destroyed, actually. I mean, John would be able to give a better answer here, I feel, but I'm not sure. Okay, cheers.