Live CEOing Ep 760: Language Design in the Wolfram Language [TreeOutline, BitFlip, and more]

Okay. Hello, everyone. Welcome to another Wolfram language design review, part of our incremental language development series. And our first item is tree outline.

Okay. So, yeah, if you open up focus of 0.5 first. Right. So we've looked at this design before and you know that that was kind of representative of the previous design. And your comment was essentially, you know, it's quite precious was the word you used essentially. You know, too much ink on the page, really. And so the focus of this iteration has been to try and sort of simplify. So if you open up simplest state, you can probably skip over the other two. Yeah. Okay. So here, yeah, as I say, tree mock is just meant to represent tree with an appearance option set to opener. So here is the new sort of design. As you can see, it's kind of a lot cleaner. We've got rid of sort of, you know, all of the branch lines, and now we just have these sort of vertical scope lines. Yeah. Leaves are bulleted with those, those pale squares, as you can see. Openers, obviously. Sorry. Inner nodes. It is. Yes.

Slow, isn't it?

Right, so this is a Wolfram language mockup of something that would require some degree of C support, I would imagine. But yes. So, yeah, so that's the fully expanded tree of the thing above. Obviously, it would open in the fully closed state.

So what are these numbers down here?

The numbers correspond to, you know, the path to the node that's currently being hovered over.

And so what is this? So first of all, I like this, but how about some green just because. To liven it up a bit.

Sure, yeah. I should say that this hasn't graphic design. Haven't seen this yet. So, yeah, certainly open to all of those sorts of suggestions.

Okay. But I think. Okay, so then an ordinary tree. I mean, if I take the same tree and I just display it in our usual tree form. Right, Okay. So now, does this have any kind of rollover?

So you mean. I don't believe. So. I imagine someone from the trees team could probably comment sort of better than me on that. I don't think it does.

I mean, is it obvious that we want this or is it because, you know, it's kind of thing where. Where you're showing the, you know, the webbing inside, so to speak, by showing those numbers. And it may be that the tree, you know, is presenting something, and the fact that that's the 13 branch is just totally confusing and irrelevant.

Sure. I think, and Ian, correct me if I'm wrong, but I think that another option was potentially, we could show the names of the nodes there instead. Is that right?

But why aren't we. Why are we showing something different here than we're showing here, Ian?

Well, so the numbers that are being shown on the outline mockup is for consistency with data set. Data set also shows the position that you're hovering over.

I know, but why.

In fact, if you scroll down, I. I have a section on this and I have an example data set, so we can see it. Oh, there we go. It's the next one. Hover effect.

I know this is consistent with data set, but I'm just. I don't understand why this is. Has anything to do with data set. I mean, this is a tree. There's no way of. Of showing a data set as a tree, is there? Yeah, maybe there should be. I mean, there isn't right now.

I think there is. Give me a second, I'll figure it out.

You know, somebody else speak up here because, I mean, many uses of this tree. I just think that this is confusing to see these numbers here.

Yeah. If you give a data set to expression tree, with the second argument data set, you get a tree version of that data set.

Okay. An inspiringly large number of examples here, but whatever. Okay, Okay, now how do I get it in the new form?

I don't think it would. Yes. Although I don't think that's going to work.

But I'd like to see a more complicated data set. That's my point. I think it's just killed it totally, Absolutely dead. Okay. I don't believe in the sanitation thing. Okay. I think it's just going to confuse people. What. What is it thought to be useful? Ian, speak up. Is it useful?

Yes, it, for example, when you drill down into its subtree, you would know what position you're at navigating the tree.

Okay. Which you wouldn't know. So you claim you need that, just as you do in dataset, to know where you are.

Yes.

Well, I mean, let's look at the examples because you also do have the path written out at the top of the tree as well as you do in datasets.

Yeah, Right.

I think the two together help you. You see where you're at and then you see where you're going.

Right.

Right click on the double chevron where you are. And so we can see you've got the path at the top now with.

The, with the tree. Okay, but if this was a. You can have associations. You could have named branches and trees. True.

Yes.

Yes.

So then what does this look like? Because In Dataset doesn't look like a list with numbers in it. Right.

It's a list of keys.

No, I understand that in Data set it does not displayed this way. It's displayed with little greater than signs between the elements here. Am I making sense?

Yes.

This down here might be displayed this way, but this is not. Okay, Okay. I mean the consistent hover thing for this is just a null list, isn't it?

Right.

Well, in which case, I mean, it's sort of better to just not show anything as I'm doing now.

That's fine, so long as the spacing is okay underneath.

Yeah, it's just using an attached cell, so it doesn't, you know, it's not taking up any space.

That's good.

Although I mentioned again, this is just a mock up that I've put together. This is not, I don't think at all representative of the actual implementation of what this would be. So, yeah, so you've just been exploring this, but basically, yes, if you click on those double right chevrons that appear when you're hovering over a leaf or an inner nodes, then it will drill down to that node or that leaf and then you can sort of step back up baseline position. That's kind of detail we probably don't need to go into. Now there is kind of one interesting question in there that maybe we could circle back to. But yeah, so then this is just to give you an example with kind of, you know, other content.

What is this font here?

I think I'm just picking it up from the output style. But obviously we could define our own style for this.

I mean, this is a. Ian, this is. How would this look right here? Oh, I need the ge.

Oh, you need the width. Yeah. No, no, no. Okay.

You know, I have to say that this font is a bit ugly here.

Tree uses standard form for its format type.

Do you think that's right?

I think so.

It's really ugly here in text. I mean, what looks like text.

Consistent with what? Like edge labels and vertex labels and graph use just standard.

It's different from graph and it's because these aren't usually labels, they're arbitrary Wolfram language expressions. You know, we're not expecting text to be the default thing. In all cases. When you use Tree, you're expecting expressions. So I think standard form is the right thing.

It'd be nice to make sure that we can have some global thing here that switches all of them so you don't have to put text in every single one.

I believe format type Format. Format type goes to traditional form.

That's in tree mock, though. Don't evaluate that just in case we freeze up Again, they do it in the tree.

Sorry.

Yeah.

That looks much nicer. All right. I mean, the problem is people always leave these things with defaults. I think not for this meeting, but that's a discussion we should have, is whether to change that. Okay.

Okay. So here again, there's some consistency with data set. This is, you know, how do we allide, you know, long lists of items.

And.

Yeah, this is not an interactive mock.

And the fact that these aren't labeled properly is just because this is a mock up. Is that right? Because these would normally just actually have names like Cordelia and so on.

Correct. Right. Yeah, it's just. Yeah. For the sake of putting something there. But yes, the fact is, you know, data set there, you can see like, you know, 61 total, 27 total, etc. And so I've just kind of copied that here.

Nice.

The. Yeah. And then if you have, you know, if the current route, because of course you can drill down so you know, the root or the node that is serving as the root has too many items to display at once. Too many leaves, too many children. Display at once. And if you scroll down, you can see that we display a scroll bar for the, you know, sort of for the. For the branch coming from what is serving currently as the root. Again, copying the start.

Ethan is commenting on our livestream that our graph label layout is still bad after all these years. I don't know that you need to. When labels are arbitrary shapes, you need to relay out the nodes. This is probably the wrong group to talk about that in. I'm not sure what's wrong with this particular one.

No, this is known. There's a family of tidy layout algorithms for trees that should be implemented.

Well, that seems important. Is that on your plate?

Yes.

Seems important to me. This looks good. I have one comment just from the whole beginning of this is, you know, the only aspect of this design that feels a little bit still precious are those dots.

Okay, so I went back and forwards on this. So the. Okay, so all the children of one, right. They're all at the same level in the tree and so they all should have the same indentation. And if you don't put something there. Right, so in other words, if you. So, so, you know, nodes with children are bulleted with those.

What about. What about. Is Jeremy here, by the way?

No, no, he's not.

He needs to be, because this is a design story. Right, we have.

Billy is here.

I think I'm here.

Okay. I think this would look better if instead of a dot, it was a little tiny fragment of the vertical. See what I'm saying?

So kind of like bend out from the main branch and then onto this.

No, no, no, I'm just saying. I'm just saying that just as this is spanning, see this vertical, you know, line. There's a span from here. There's a span from here. I'm suggesting that instead of a, Instead of a different graphic motif here of a, Of a bullet, that instead, it's just a fragment of that. Understand what I'm saying?

A vertical ball.

It's just a.

Just a bar. But the bars don't connect.

Yeah, they don't connect. Now, is that inconsistent or silly from a point of view of what the tree means?

I was going to say, would it be confusing or weird that it's on the same sort of level as 13 below it and that the lines might somehow indicate that it's somehow associated with 13, whereas there, the dots sort of show that they're kind of.

It is at the same level.

Well, I mean, we agree that it's at the same level. The question is, is it associated with.

It's not going to look that way. I don't think. Just no more than this line here looks associated with this line here. I think I would make this design look cleaner also. Let's go green, as they say, for these lines. Okay. If you can make a mock up, Billy, that would be great.

Okay. Hello, can you hear me?

We can now.

Sorry, my connection dropped, but I'm back. Right. So yes, XML is obviously a case we want to deal with slightly differently. I think that one is a live mockup. The first one. Yes. And so, you know, we're not displaying the XML object of document nor the declaration. We're just starting with the root of the actual content. And then all of that other stuff gets put into a tooltip. But I've got a section on that below. But here I have a few alternate layouts for just an XML tree itself. So we could also have the closing tags and then if you scroll down one more, we can show a preview of what's actually in a closed tag. Right. Because I mean, it's not particularly useful in that case to just see sort of, you know, sale, sale, sale. So I'm showing the, you know, like the first. A preview of the first element in sail.

I like it. I also think we should have the closing tags here.

Okay.

I think it doesn't waste much space and it makes the whole thing less confusing. Now, how does it know to go blue for this here? It's just looking at XML element. It's saying that's a special case.

Right. And this is specifically for appearance, goes to quote xml.

How, how will we get that? What will default to that?

So I think I have a note about that just above, you know, it would be set by the appropriate thing. So for example, I would imagine that again, someone could probably comment on this better than I can. But you know, there are functions that it would. That would sort of obviously set that appearance option.

So import. I mean this thing here, input, comma, tree would probably go to this.

Yeah.

Okay. And what can I copy paste from here?

Well, I mean that's, that's a more general question, which is that contents of these openers actually be sort of acceptable editable, have some group of interaction. And I sort of wonder whether, I mean, I guess John would sort of, you know, maybe has an idea of how this might be or whether that's kind of, you know, possible.

John, do you want to comment on that? Please.

Sorry, I missed the prologue to that.

Whether these things physically.

Yeah.

Whether. Sorry, you were talking over one another and I couldn't. This kind of tree structure, could it be made editable.

Or at least selectable?

I don't know. Okay, how would I get back from this tree display to the raw XML elements? Would normal do that or what would. What would do that?

Tree expression.

I see, yeah.

That'S a tree mock, remember? So it's not going to behave properly in that sense.

I was going to say font wise, given this is a special rendering, we should make these angle brackets a little less angley, I'm saying.

Sure, yeah, yeah, makes sense.

Especially given that we've got so many other angley, you know, equilateral triangles and things like that going on here. Make sense? Yeah, okay. Yeah, I think adding editability would be a significant challenge. Okay, well, let's not worry about that right now. Okay. I recommend here, maybe this goes to a smaller font or something, this content.

Sure, yeah, I can mock that up. So here's the actual annotation. So I was saying previously that we don't include certain parts of the tree in the tree outline. So any metadata which relate to the entire XML document and not like a specific node, they'll be displayed in a tooltip next to the root. And so that's an example XML document, if you scroll down, that's what its tooltip would look like. And Then if you scroll down even further, if you then hover over the eye next to note in the actual tree. Sorry, in the cell below. Yeah, there we go. You can see that sort of. That's what it would look like.

And you would display the I in the actual thing.

Yeah.

Ethan on our live stream is asking about interactive JSON viewer and doing the same thing with JSON. I don't see any reason why that shouldn't be. Couldn't be made to work.

Yeah, I assume we'll need like eventually in need like an appearance goes to JSON, appearance goes to file system.

But what does JSON typically import as?

Well, we have two forms. There's JSON and there's raw JSON. Raw JSON.

But the import form is lists of rules, isn't it?

Yes, but if you do raw JSON, then it's nested lists and associations. So associations for objects and lists for arrays.

I see.

I think it's slightly cleaner than.

But unlike xml where we have an explicit XML element, we don't have a JSON element. We don't think we need one. Or do we?

I think we do not need one. JSON is basically lists and associations.

But. So you're saying that we could take that and we can do this tree appearance. How do we tree ify and do this tree appearance on that? You understand what I'm saying?

For example, I think if you did like import a JSON file comma tree, that would set the appearance option to JSON as well as construct the tree. Similarly, expression tree could set the appearance option.

Okay.

Yeah. So this is basically the same as above, but there's metadata that's relating to a specific node. And so you can see that now there are two of those I icons, you know, one for the. Oh, that should have worked. You actually in the notebook. Is the notebook selected?

There we go.

Yeah.

So the thing I don't like, I mean with this, if we're really just doing cruddy formatting here with standard form, why are we doing text here? Why are we processing this? But we're not processing other things. Do you understand my question?

Yeah, I'm thinking. So firstly, I mean the style of tooltips is by default, I know this sort of thing. So just, you know, just as an. As a consistency with other tooltips. Secondly, there's more of an argument here for making things compact, I guess I.

Don'T understand why Internal subset, how does that come from the text that's up there just says doctor Type. I'm confused. What Is that internal subset thing going? Is that just an implementation thing right now?

So I may have got that wrong. Ian, can you comment if that's. I may have just got that wrong. From the XML tree above, perhaps.

I know there's a few. It's the XML object as opposed to XML element and the XML object ref page documents. These different.

I don't think it makes sense. You're showing something with this whole elaborate tree structure. Don't have internal subset, arrow whatever. Because it's not. It's not even close to that. That's not what it is. I mean, you know, if this was a. If this thing imported as a rule, for example, that's not what you should be displaying here. Does that make sense? Hey, we got it. We got to liven up here, right?

I guess the idea being that if this is meta information and we. We want it to be displayed, it should be displayed without a bunch of XML tags on it, right?

Well, it should be displayed in whatever. Look, it's very consistent with what we're doing for the main content, but. Right, yeah.

Is.

Is the metadata always going to be fairly textual for xml or do we have to worry about the. The problem that Ian said earlier, where there's arbitrary expressions that are potentially going to be showing up?

I think it'll always be text for xml.

Okay. Anyway, so that needs to get fixed. Right? All right. What else do we have to look at here? You want to look at something about this baseline thing.

Just that there's a slightly bizarre case where. Right, so it's the top thing. Right. Should the baseline of the tree always be the baseline of what the current root is? And it just means that if you don't do that, you get that kind of. Or if you do do that, rather you get sort of the path when you've drilled down, sort of displaying above. Right.

So that's an example.

The top example says aa. Right. And then it's got the tree and then it's got yy. I just wanted to check that. You don't think that looks slightly odd, that the baseline, when you drill down is, you know, has stayed with what the current route is, and therefore you kind of get this odd path.

The path is a thing with a decorator above. I don't think that's a problem. I think it's quite nice, actually. I mean, in a case like this, you know, it's a little weird that you have a borderless tree, you know, construct here that this thing is just hanging, but it's probably in actual Use cases. It's probably going to be fairly easy to understand and if you don't like it, put a frame on it. I mean it probably be good for the tree thing to have a frame arrow true option.

Sure.

Okay.

Yeah.

And then I guess the only other question I had was right. So I, you know, I'm proposing here that we have tree.

Okay. So I'm sorry, sorry. Billy just mocked up. But he didn't change it to green, which is a shame. And he also changed the form of these openers. Was that intentional, Billy?

Yeah, but there is, it is green just because it's not coming through. Um, and yes, I did change those, but.

Oh yeah, it is slightly green. Yeah, I think it could be greener.

Personally, I think the filled in triangles are just. It's just a little easier to see, you know, the things you can click on and what's been opened and what's been closed.

Yeah, I kind of agree. I kind of agree. Maybe there's an intermediate case here, but I. Billy, why did you change that to the outline?

I was thinking more of like, you know, when trying to interact to see that the change happens as well. So if you like hover over it, maybe it fills in, but it could also kind of hover the other way.

I see it filled in and then.

You get the lighter hover. So it would look like.

Do we have a hover effect right now in this?

Yes, it goes blue.

I mean it could clearly go bright green.

Sure, sure.

I mean, given that we're going green as trees, you know. Well, what do people think about these verticals? I think they look better than these. I think they're more integrated.

Yeah, they definitely have a little more weight to them.

Well, I mean they just make more sense. This is just a different design element that we're not seeing anywhere else here. Okay, let's assume this is going to work. This looks good. Let's implement it. Go ahead.

Well, yes.

What do you want?

I don't know because it's. Okay, so where one is open there. Right. And you've also got one three open that has a solid line. So you can clearly see what its children are with these, you know, with these vertical lines above and below. It does make it ever so slightly less easy to kind of just parse what's a child of one three. Because that line continues like 14 is.

Not part of that three.

Nevermind. Right. Yeah, but it almost looks like it is.

Well, okay, but it maybe needs some other device which we're not going to solve here. I Think it looks better than having bullets?

Sure.

I understand that. That has to be solved. Maybe these things aren't quite as tall as this. Okay. But they're integrated with the form of these things.

Sure, sure. Makes sense.

Okay. Should we move on?

Okay. Thank you. Yeah.

This is very nice. This is very nice stuff.

Thanks.

All right, next item. Highlight text.

Yeah. So this is a. Originally a function repository function that Rick put together. I figured it makes some sense since we have John lurking in the background. He can either opine or groan at the idea of inserting style markups in text based on patterns. That's the general purpose of this particular function.

But what is the result? What is the thing that it.

The current result is a frame box that has a bunch of style rules.

But what is it? What is the thing that you get? I mean, so is this something where highlighted text, or whatever it's supposed to be called? I don't understand, because there are many text annotators, including text elements and so on. How does this relate to those?

Right.

I mean, I think this is most closely related to what text element is doing in terms of you're getting back a decorated object that is not obviously just a string. It is not a. A row with a bunch of styles in it.

Okay. I mean, we got to be consistent, right? We've got. We've got a certain number of these. These kinds of things and back, I guess.

Well, do you want to just do, you know, input form on one of those outputs and so we know what the hell we're talking about.

Yeah.

I've not seen this. Well, maybe I have seen this function before, but it's been quite a while.

Because this was made at. For being a resource function, I couldn't really define another separate object that had, like, formatted definition. So if I remember correctly, it's just a string with inline standard form stuff. But if I were doing this separately as like a standalone thing, there would be like, you know, a highlighted text type of wrapper or something that does something different rather than just baking stuff into the string.

Look, I want to know how is this consistent with text elements and the other kinds of annotations for strings? Because we've got a whole bunch of annotations for strings we're trying to do. Do we.

I mean, I don't. I don't think we want to implement this necessarily as the function exists here, but I think there's some interesting elements of it, particularly, you know, the idea of taking strings and string annotations and doing stylistic markup based on patterns. You know, there's, there's an option included. The last example there that Rick put of this trimming threshold that's really useful when working with large sections of text. You know, I don't.

Also, it's not clear to me that there's any relationship here to text element. I mean, text element is a semantic markup. This is a purely, you know, how.

Does this positional markup, how does this relate to snippet?

Yeah, I mean, the, the trimming threshold is, is effectively using what snippet does, but we're highlighting it around, you know, in this case, it's trying to highlight this particular hurrah instance within a large body of text. And what it does is it compresses around that detected element. Snippet gives n number of lines. This is either, you know, you could potentially have lines or characters, but say, you know, I want this auto compressed around some element that I'm looking for. So we allied content before and after it to, you know, 200 characters. You could cut it down to 15 or something, and you'd see the highlighted sections, but you'd have less context.

Right.

Right. I don't, I don't think we want to add yet another markup on text that is just for highlighting. But I think, you know, looking at, as itay said text element is primarily concerned with semantics. And I don't know that we have a general markup for text that is done around style and where that integrates. Maybe it's just not particularly useful to go through and say, okay, I want to find instances of this particular phrase in a body of text and just highlight it or change the styling.

We are running out of time here. So let's just see what, what things that we know. Look, this, this idea that we're highlighting by pattern is something we don't have for expressions either. Right, Right. What you do is you go in and you. For expressions, you go in and you say map at some part or some such other thing. Right. So here, that's kind of inconvenient. And I don't know whether this is useful. I really just don't know. I mean, like, I can imagine wanting to highlight section headers or something that are defined by some kind of pattern. Like, you know, star. A star at the beginning of a line or something. You know what? I really would like us to look holistically at all the things like text. All the things like that are, you know, annotations of strings.

Okay.

Does that make sense? I would like to look at that. Yes. I would like to look at that, but this is not that.

Okay.

This is a very small one off function, but looking through.

I like it.

We have a number of bit functions.

I like it. I think it's a useful function. The only question is which. Which direction does 3 mean? 3 from the left, right?

Yes. Yeah, we're counting. We're counting from our zero bit.

No, no, no, no. That's three from the right. Then three from the right. So which is it? I don't think it's correct to do three from the.

I mean, it's. I don't think from the left is necessarily well defined when we're talking about integers.

Yeah, I mean, if you look at the example down below, this, you know, we. You can use this principle if we're counting from the right to flip bits that aren't set in the integer.

I see.

I can set.

That makes sense.

The tenth bit.

Well, so we could have a minus thing that. That starts from the first non0bit. Am I making sense? Right.

I want to count. I want to count to the right from the first non0bit.

I think this is. Okay, Let's do it. Well, except bit set and so on. Those count from the right. Is that correct?

Yeah, I mean, this is. This is literally using bit set.

I see that.

XOR of bitset effectively.

Yeah, fine, let's do it.

This is a utility function. I think Rick originally wrote a function repository version of it, but it effectively lets you inject a hold into a function that isn't normally held. The way that it does it is by using a pure function that's hold all complete and injecting it unevaluated. But you.

I'm sorry, what is this trying to do? So, function evaluation.

So that print held function there. That this hold argument of print is basically a version of print that holds its arguments prior to printing.

Okay, but so, so this thing here would be. I mean, forget. This is a super confusing way to do it. I mean, this here doesn't work. Okay.

Because this didn't run the definition. Right.

Yeah. This is an interesting wrapper. I mean, this is a. This is a good idea. It's a good idea.

Yeah. I think it solves a problem of their various specialized cases where you want to have a held version of another function. We don't want to introduce a bunch of hold variants to functions.

I like this idea. We should have had this idea before. So this passes.

So if you want to look at the definition, you can.

No, no, no, no, no, no. I just want to know this Passes unevaluated there. So if I were to say identity here. Okay. So it'll pass to the identity function one plus one without. So it's the equivalent of identity, of unevaluated, of one plus one. Is that correct?

Yes.

This is a nice thing. This is useful.

It'd be hold arguments or held. I think it should be held arguments.

I would think so, too.

Okay.

Yeah.

Held.

Let me think about withhold arguments.

No, no, because this is like. Is this an applied function? That's what I'm thinking. Right. I mean, it's kind of like reverse applied would be. You know, this is hold applied, isn't it? Right.

Yep.

That sounds.

Or is it hold applied? I mean, is it the. Is it the direct analog of reverse applied? That is it. There's some sort of commutative diagram. Ish thing here. I mean, it's kind of like this is. It's inserting a reverse here just as this is essentially unevaluated applied. Is that correct?

Right, right. Unevaluated applied would be directly analog. Right. It's not actually inserting a hold anywhere.

The only problem with that is it sounds bad.

I don't know why.

It just.

Yeah, I understand. I understand. But it is conceptually unevaluated.

That is technically probably the more correct name. But it just. Yeah, I just don't like it.

Well, maybe held applied at the middle ground.

I'm thinking held applied. Held applied is a little weird because it's two past participles. Yes. And reason. We didn't use reversed applied. How about just held? Sorry, Sorry. What were you suggesting? Just held. I don't think that's right. I don't think that's right. Held of F. I don't think that's right because it's not the F that's held.

Right.

Holding.

We don't. We don't have anything else that sort of temporarily sets attributes to the thing that it's being applied to. Right. I mean, this doesn't work, but it's effectively similar.

Well, gosh, we need a name here. I'm not sure what it should be, but we are out of time here, so. To be continued.

Okay.

Circle back to the name. But.

Okay. All right, see you guys soon. Thanks a lot.

Thanks all.

Bye for now.