Live CEOing Ep 690: Language Design in Wolfram Language [Kernel Configuration UI]

Okay, hello everyone. Welcome to a Wolfram language design review. About the well consolidated kernel configurator, how to connect to kernels from your front end. Okay, so Roman, take it away.

Yes. So at the moment we have two mechanisms for configuring kernels. There is the kernel configuration options which manages the kernels that the front end knows about. And then there is the parallel kernel configuration. Parallel and sorry, that's in the preferences. So now this used to be a small old fashioned kernel configuration panel in here, but that was no longer maintainable with the new kernel configuration based system. So at the moment the configuration is through command line or service discovery only. And so I wanted to reintroduce a graphical way of configuring kernels and that should work for all types of kernels in including front end evaluators. So the proposal is to have a new tab here with kernel configuration and then we can probably get rid of the parallel tab because the only thing that's left in the parallel tab will be that general preferences, which we can.

Okay, let me make a comment at the same time here. That Internet and mail thing and it turns out it's the Wolfram Alpha connection. You know, that's kind of crazy. I mean, I probably should say, I don't know, it's. And where's the cloud connectivity? I mean, okay, this was supposed to be part of the system services project which somehow, you know, is dead on the vine. Can anybody turn Todd or anybody else comment on system services?

I can comment on system services. Well, first, the project is definitely hibernating. We have a system, the language API for setting and getting these preferences. But there's little value in that because we don't have any compartment based services at the moment.

Right.

We do have.

Well, mail was supposed to be the.

First one, Mail was the first one and we could add it for mail. But Mail already has configuration ability.

Right.

And we don't need to introduce system service.

But how do we do geoservers?

Well, at the moment there's no geoserver other than our own, there's no reason for.

No, I understand that, but if somebody had a privatized geoserver, how would they deal with it?

Well, if they had a privacy, I'm not exactly sure. There's probably some sort of dollar geoserver there is that they can set. But at the moment I'm told by the geoserver folks that that is not likely any. That our geoserver is highly specialized thing. At the moment I think geoserver is a Ways off. As an example, the next most obvious example, Packlet servers. And we have a design for how we could do Packlet servers with the Services API. But I don't think we need to go into that discussion here.

Right, all right, fine, fine, fine, fine, fine. I kind of. Yeah, okay. I mean, I kind of think, just in terms of refactoring this, I think it's incredibly obscure that, you know, I think we could have more things. John might want to comment on this. We could have more things on the left column here, like mail separated from Cloud separated from Wolfram Alpha and so on. Rather than putting together things which will not be seen as conceptually similar.

I, I mean, I'm fine with adding more tabs. That's not a problem as far as cloud is concerned. I mean, the problem is any, anything interesting one might want to do with the cloud is just not implemented in the front end yet. So there is nothing to configure here.

I see. Other than kernel variables. Right, okay.

But, but, but those kernel variables, you know, they're. How they influence front end behavior is, you know, it's a bit tricky and I, I wouldn't want to expose that.

In a way that.

Yeah, fine.

All right.

Let's understand what the goal is. I mean, to me, one of the things drives me crazy is, you know, I want to be running something in parallel on, for example, a very common thing for me is run it in parallel. But I want it on machines that are not my own desktop machine. I want it on the other machines that I have. Sometimes I want it also on my desktop machine because quite kind of a beefy machine. Sometimes I'd like to pull in, you know, something in aws, for example, I would like some way where I can easily pick out those things and say, parallel evaluate across that particular collection. Is this going to give me that?

Yes, I mean that workflow is part of the design that. Okay, terrific.

All right, let's, let's dive into the design.

Okay, so what, what this new design will support is editing kernel configurations in a graphical way.

In addition, which we already have for the, for which we've had for like.

35 years, 40 for the front end kernels.

Yes, right.

But we want to modernize, especially the SSH support for the front end kernels anyway to make use of connection tunneling and so on. So it's a good, good. And, and I use the same functionality for parallelism anyway, so, so that will be a lot of shared code.

Okay, so let me make another comment. Sometimes, You know, somebody Sends me a notebook and they left it connected to their kernel, which is a kernel I do not have. Okay. And so then when I try and run it, it complains. Might it not be a good idea when you open a notebook that is connected to a kernel that you don't have, that it puts up a dialog saying that and says, do you want to realign this to just be using the default kernel?

So that would be a kind of a modern, more modern notebook kernel item here.

No, no, no. What I'm saying is, do you understand the scenario? Somebody.

Yeah.

I. I understand it. And not a dialogue, but yes, and, you know, something in a doc style or something. It's, you know, let's not do dialogues. But, but I get your point, and it's a reasonable one.

I mean, and if you hit Shift enter and it says, and it brings up the thing saying that kernel doesn't exist. There's got to be some way, maybe in that dialogue, of pressing a button that says, just reset this notebook to use my default kernel.

Yeah, well, I think the. Well, there are two choices here. One is, you know, that we can just use your default kernel, but we probably don't want to switch the notebook to use the default kernel unless we tell the user that's what we're doing because then that fact will get saved into the notebook.

Fair point. Okay, fine. So, I mean, it needs a little bit of ui.

I can figure this out. I'm not. I don't think we need to design it here. I can figure this out and I'm not worried about coming up with a sensible solution.

Okay, fine. Onward.

Okay, so the other parts of this new stuff will be choosing which kernels to use for parallelism, for front end, for remote local evaluate, and all of this is backed by persistence. So you could store a list of kernels in your cloud account and then easily access.

That's nice.

So that.

Okay, but then, so, so just so I understand, there'll be an easy way to give a name to a group of kernels so that I can just say parallel evaluate and then give the name. Is that correct?

Name for a group of kernels is not yet in this proposal, but it is a very good idea because at the moment I just have to select sets of kernels. But it should be possible to give them a name.

Yes, right, but so in particular then in the kernel I can just say parallel evaluate onto that group of. Of kernels.

Well, I mean, the.

Yes.

At the moment, the way it works is you define a variable whose value is a list of kernels and you can initialize that variable persistently. So I don't think we need an extra layer for names of kernels.

No, I don't agree. I don't agree. I'm sorry, I don't agree. Let me tell you my use case again. Okay. You know, when I'm doing a big crunchy computation, if it's overnight, for example, I wanted to use all my machines, including my desktop machines. If it's during the day when I'm using my desktop machine, I don't want it to use my desktop machine. I wanted to use the non, you know, my other machines. You see what I'm saying?

Yes.

So I want to be able to have two names there that I can use. I also might want, or similarly you.

Might want a group you're willing to pay money for or a group you're not willing to pay money for. That seems a reasonable scenario as well.

Yes, exactly. I mean, I would love to have something which is, which is named to be a thing, which is some AWS kernel that I know I'm paying money for. Another case, which maybe we've covered separately, is the representative from every computer.

Yes, that already exists.

Okay. Okay. All right, let's move on.

Okay, so there is not much in terms of new symbols, but there is a lot of UX UI design. So let me show you a functional prototype. So I've obviously not spent any time making it beautiful because the people.

Well, who's here? Who is going to deal with the beautification? Do we have somebody from graphic design or UX here? If that's what we're primarily talking about, I will go snag somebody. Okay. Okay, let's see.

So the code is. It requires the prototype currently and there's also some extra code which I have not put into the repository because that's the thing where most likely UI design will make changes.

Okay, fine. Just show us what we need to look at.

Okay, so the first new functionality would be that kernel editor. So if you have currently you create kernel configurations with option settings. And now there is.

This is a symbol that we introduced a couple of years ago. Kernel configuration.

Yes, that is not new. So kernel editor, whatever its name will be, is the new thing. And so this will bring up a data driven user interface for all the changeable public properties. And that depends obviously on the kernel type. This is for a local kernel, so it will have the append in out name.

All right, what's our model for this? What other things do we have that Bring up. I mean, I have to admit, I don't think we've been like, molecule Draw, for example, is an example like this. Am I making sense? John, can you talk about that? You know, what things do we have?

I'm not understanding the connection to molecule Draw at all. And frankly, I haven't actually used Molecule Draw much, so.

Okay. Nor have I, unfortunately. But, but I mean, I'm.

I'm.

What it does is you've got a thing in your session and it brings up a dialogue separately like this. So I'm saying, I mean, this is the thing where you are, you know, where the workflow is. You start with something, it brings up a dialogue, you change the thing and it goes back again. What. What else do we have that does that?

Well, that's actually.

I mean, input does that in some sense.

Yes. I mean the. I, I thought about this and so what, what I came up with is where is. So, so what do. Yes, we have, we have two functions that prompt for input and return. Block and return the thing.

Yeah.

Then we have three more that insert, that bring up menu, special characters, insert color, insert file path. But these are really the only ones.

But those are front end things. They're not things which are initiated. We have file browse. Is that correct? Could you type it in and see what it does?

File. Do we have file browse? No.

What do we call it?

I think we only have the insert, right? Do we have to insert file?

No, no, no. We have a. There's a kernel function which brings up a file browser and I should use it. Well.

System dialog inputs.

Oh yeah.

But there's also a control.

So system dialog, insert file.

This one open. No, no, no, no, no. System dialog input.

Just type a quote, the string file open camel case.

What is it doing?

It's not a good sign.

Is it? On another monitor?

Maybe.

It should have popped open the open dialog and then waits for the results. That's why that, that's why it's hanging is because it's waiting for the results.

So there is. Oh, there it is. Yes. Okay.

There it is.

Okay. Okay. Yes. So these are the kind of things we have. We have very few of those. And so do we need or want one for. Specifically for kernels. I mean, the nice thing would be you could do something like locally valid and then you could have that thing. It would prompt you for a kernel and you could then change the. Some of the properties of that kernel. I would like to use another version. Okay. And then it would immediately use that kernel and run an evaluation on it. But that would be a possible use case for this, for this function.

Well, I mean, I think it should be called kernel configuration Edit or something like that. Or kernel. I mean, I don't know. It's going to bring up.

It brings up a kernel configuration editor. Yes. So.

But I don't think it should be called Editor. I think it should be called Edit. It's an action.

Well, yeah, I mean, how do we call these things that actually bring up something and wait for.

Well, we don't have very many of them. The.

The block kind of configuration dialogue.

Yeah, there's message dialogue and there's another one of the dialogues. I forget which it is. Well, we could put a ton of.

I don't. I don't know, let's keep going dialogue.

Yeah, we have a few dialogues like that.

Well, what do you think this should be called, John?

I think Edit is a reasonable choice.

So we. Kernel configuration Edit.

Yeah, why not kernel configuration dialogue?

Because it's obscure. Because, I mean, the fact is it's an edit, isn't it? I mean, it's going to return.

Also, not all dialogues, not all things with the name dialogue follow that pattern. Create dialogue is. Well, maybe all the things of the form noun dialogue do that.

Yeah. Okay, but hold on, why is this not an alternative would be, you see the kernel configuration blob, right? And maybe you press a button on that and it brings up the editor.

But then you first have to put something on the screen where you can push the button. Preclude this particular case. And the idea, the other proposal here is that if there is no argument, then it will prompt you for the type of kernel and it will then give you a chance to edit, to enter all the information for that new kernel. So maybe kernel configuration edit. This is probably a good name.

Okay, provisionally, yes. Okay, let's keep going.

And then one small detail that requires some care of the ui. I just wanted to show you what the idea is at the moment when you enter an option for new kernel configuration that is wrong because this is not a valid host name. But then when done like this, it gives you the expected message. But in the kernel editor, I follow the idea of something you see a lot in web forms. For example, if I here enter something that is not a host name, then at the moment I just have this pop up.

That is not one of our forms. You didn't custom write this form, right? I mean, this is just one of our forms.

This is just this. This is input string or Input, input, input, field and the rest is just some glue around it.

I mean, why isn't it using a form function? Why isn't it just using our standard form function? Because it has a standard method for dealing with. Oh, you know, I mean, that's a whole giant system.

So I should look at form functions.

Yes, but, but did you custom write this as a.

Well, it's all data driven, it's all generated from, from, from. I mean.

No, I understand, but I mean, look at form function. It should just be exactly this and, and then we know it works on all the platforms and we know it has good error behavior and so on because it has a mechanism. If you enter something that's the wrong thing. And then we also don't have to redesign the, you know, we don't need a lot of design because it's already been.

Mm, okay, I'll look at that then as soon as possible hand it over to UI design so they can design it.

Well, Right, but. Okay, all right, fine.

Okay, now we have one more terminology decision.

Okay, but wait a minute. Kernel configuration edit. Yes, that name is not sitting that well with me. I just want to make that clear. I'm not really a fan of that name, but it's the best I can think of right now. Can we see the dialogue again, please? What the hell is fe? Kernel arguments. Yeah, what kind of a name is that?

Yes, I mean, I know I need to change those. I mean, the problem is it gets a bit too. So the idea is that the arguments that are supplied to when launching that kernel, that they depend on the use case. So the front end kernel will have different arguments from the remote evaluate kernel, from the parallel sub kernels.

Okay, but, so hold on, hold on.

Hold on, hold on, hold on.

These are basically keys in an association. True or not true?

No, these are, these are elements that will go onto the command line with which you launch.

I know, but in terms of the kernel configuration function inside these are keys.

Yes, these are all strings.

Okay, so then those keys can just be strings that are quite long.

Yes, okay, yes, makes sense.

And I mean here I think we've many, many times we have sort of, you know, English ified things for purpose of a dialogue. Am I making sense? So instead of having a camel cased, you know, remote evaluation, kernel arguments or something, we just write that out as separate words. I'm not sure what the right thing to do is, but, but, but, you know, either it's just a long name there, Remote evaluation, whatever it is.

I mean, some of these Have. Have tool tips with. With some more explanation of what they do. But clearly these. These are bit cryptic. Yes.

I'm sorry, Stephen, did you say that.

We use camel case or we. No, I'm saying that we have examples where underneath it's a camel case variable, but in the description, in a dialog or something else.

Oh, oh, oh.

Okay, okay. I'm sorry, I'm sorry, I thought you were making statements about the underlying association. Okay.

No, no, I should. I should simply add another data item. That would be the display name for these. For these options.

Yeah, I think so.

But I mean, I also like the tooltips here.

But, you know, in out name. That's really cryptic. It's like. Is it? No, I mean, we should make these things more explicit. There's nothing wrong with making more explicit include in out labels.

I haven't. I haven't spent time on the design of the.

Okay, but I mean, it should be like included.

It uses the RAW option names.

Okay, fine, I understand, but those should be changed. I mean, to be. Those should be changed. So it's like include in out names and things like that.

Yes.

Kernel command string. I don't know. Is it. Is it dollar command line? That. That's the thing from dollar command line, right?

Which one?

The kernel command.

That's just the first. That's just the binary without the I.

I'm aware, I'm aware, but it's a. Maybe that's the right name for it. Okay. I mean, I don't know what.

This one already exists. So that. But these. These are. These down here are new ones, so they can very easily be changed.

Okay. It's kind of upsetting that you didn't use the form function mechanism, because it's a big mechanism and it allows you to do all kinds of things with contingent pieces of forms. Like here you could check things in. I mean, presumably you don't need all of those different kinds of arguments if the usage wasn't there. Right. If you remove parallel subconnel as a.

Usage right up above, then that would.

That doesn't make. It doesn't. But it should. But it could with form functions.

Okay. Yes. No, I have never used form functions, so that's the reason Roland and I.

Talked about using form functions to mock this up, but his goal was to mock this up in. In the quickest way he could. And so it. Whatever, straight notebook programming was his choice.

But fine. Okay. All right. So. All right, so then somewhere there is a manifest of what this did. So is that in the kernel configuration? Blob or what? In other words, what, like the comment. Where do I see the comment?

Yes, I mean the comment is something I just introduced. I haven't any usage for it, but it could be for you to annotate. This is a kernel that costs that many dollars per hour.

I understand that, but where is it going to show up?

No, I mean some of that will show up in the. I mean these will also. These are just the ones from the current version.

So. But then in the menu there's a chooser presumably for. For. I mean, the menus should only have in them, if I'm not mistaken.

Yes, I mean the, the. I mean here at the moment, like Notebooks kernel. So that's. That's how this. Yes, and so that name is what. What is at the moment, it's underused. But the. So that I wanted to show you this. This is the name and so this is what you should make.

All right. I'm trying to understand the front end obviously needs in the front end dialog system, you know, front end menu system, something that says where to connect this front end for its kernel. For notebook kernels. Okay. When it comes to remote evaluation and parallel evaluation, it is a mere courtesy of that the front end would provide any way to configure that. Am I making sense?

Yes. I mean, the front end is not involved in that.

No, I understand that, but. But in the past has the front end. Okay. Are you intending that the front end as a courtesy provides the ability to GUI specify those, you know, things for parallel kernels and so on?

Yes. I mean, this is part of what we're talking about today.

Okay, fine. But then where in the menus of the front end will that appear? Well, is that appearing in the one kernel configuration options and that's going to.

Handle both Evaluate the kernel configuration options. These. I mean this. This will just lead to the new tab in the system preferences.

Okay.

And then parallel kernel and parallel configuration will go away. That will also lead to the same. The same panel.

What about parallel kernel status?

That needs to be worked. I mean, it worked well, but it's not scalable for larger number of kernels.

It's hopeless. I mean, it just kills my.

Yeah, I mean it is. If you enable remote monitoring, then if you have 100 kernels, then for every evaluation it will do 100 requests. And so it just doesn't scale.

Is there any way out of that? I mean, we should probably remove it then.

I mean, one thing to do is to. For example, first thing to group the kernels by machine. And so you have just one summary information for all the kernels in the machine. And then I mean it's really up to you to decide do you need fine grained telemetry for the kernels? Then you have to accept the fact that it's slow or do you just need. Okay, let me make a practice properties that the master kernel can figure out without querying the sub kernels.

Okay?

That's what the Roman.

Roman. I'll tell you what I need, what I want. Okay? If this kernel configuration status worked properly, I would like it very much, very useful if you're doing a, you know, if you're doing a computation. So a very common type of thing I'm doing is various kinds of search type computations where there might be a straggler, you know, and I'd like to be able to see that kind of thing.

Yes, I mean there's, I mean there are two things. This is the general one and these on the right hand side, these are the expensive ones which, which query the remote kernels.

No, I know.

The other thing would be an extension of the current progress dialog. So it would be something. If you do a parallel map, it would give you information specifically about what the kernels are doing with regards to the progress of that parallel map. So it would have to know what operation it's doing to give you more meaningful feedback.

Well, I don't understand. I mean, as far as I'm concerned, it would be very useful to just see, I've got 100 kernels out there, 25 of them are still busy, 75 of them are finished. So if all it did was when it finishes, it puts up a flag and says I'm finished. And that would show up in the status box. In other words, if it hasn't had the I'm finished flag, it just says it's busy.

Yes, and the master kernel can find that out without even querying the subturnals. So that green icon here should be if I.

Okay, fine, fine, fine, fine, fine, fine.

Then, then. See, the local ones are the cheap ones and the remote ones are the expensive ones.

Okay, okay, hold on, hold on, hold on. You're telling me that there is a default configuration of this dialog that would actually just work for my hundred kernels and not burn the computers all up? Right.

You would have to disable the collection of the performance data.

No, but that's not showing me the most important thing.

Yes, yes. So we need something in between. So we need as much performance data that can be Collected without pinging. Yes.

Right. That's what we need. That's the default.

Okay. That I can do that. Yes. I mean, it was never an issue with 10 years ago, 20 years ago when you had four kernels or so. But. But now it is. Yes.

Okay, so I mean, let's make that happen. I mean, it should be rather straightforward. Just have something where the default. So then that is a meaningful dialogue. And it doesn't just crash and burn. Okay.

Yes. Okay. So there's one small terminology you have seen before. These three terms. One shot evaluation, parallel, sub kernel, front end evaluator. These are the three supported use cases for kernels.

And what are the functions that do that? One short evaluation isn't that Local evaluate.

Remote evaluate. Local evaluate. And in the future it will also support session kernels. So you can keep kernel open.

And then it's not one shot, is it?

Yes. So that name will be wrong in the future. And they are kind of long and I'm wondering whether we just shoot them short to have evaluation kernels, parallel kernels and front end.

I don't believe in that. I don't believe in that. I think it's extremely obscure. It's hard enough for people to understand.

This will only be used in the context of a kernel. So I don't think we need more context.

I. I understand that. I understand that, but. But the fact is. Well, okay, hold on, let me think about this. Maybe front end should be called notebook.

Yes. That's what. What they are currently called. Right? It's the notebooks kernel. Yes.

What do other people think about this? It's a bit obscure. Local evaluate. What's the conceptual difference between a remote evaluate and a parallel evaluate? The point with remote evaluate is it's a single evaluation on a single kernel. Yes. Parallel is a correlated evaluation on many kernels.

Yes. And you cannot address individual kernels out of these. So it's. You don't even address the kernels at all. You say parallel map, then it does its thing.

I mean single evaluation, which goes with your one shot. Anybody else got a suggestion here?

What specific phrases are we trying to replace or.

Well, the things we've just been talking about. One shot evaluation.

I don't like one shot.

No, no, no, no. It's not right. So the question is, what is right? So this supports local evaluate and remote evaluate. But I don't know how you build a session. How do you build a session for a local moment?

You cannot yet. But that's another thing that we need to design. We need to Find the proper term for starting a session. We have.

Well, we've got external sessions.

Yeah, that's external evaluation. That's an external evaluate realm. A session.

A session. Not a. Not a single evaluation. Yes, but easy.

External evaluate. Look, look, normally external evaluate is the thing for other languages and we could apply that to our language as well and call it an external evaluation or something.

Yeah, that's one possibility. So I just wonder whether it's the best one or whether we should have something a bit more. I mean.

No, no, look, look, look, look, look, look. If what we call it is, if we say this is what's confusing. We have local evaluate, we have remote evaluate. Okay, let me ask a question. I've got many cores on my local machine. Do I get to address one of them as a local evaluate? You know, in other words, just like for remote evaluate, or do I just address a machine?

Do I address it because the cores are. Now they. I mean, okay, some modern processors have these extra slow cores in addition to the normal ones. But I mean, there is. I mean, the same is a question for parallel evaluation. Sometimes when you really want to dedicate a machine to parallel computation, it makes sense to pin each kernel process to a particular core. And there are, there are operating system means to do that. But that's below the level of our language.

Okay, but, but, but I'm trying to understand in the case where, I mean, if you want a local evaluate which is session based, what on earth is that going to look like? And we have a pattern which is start external session.

Yes. So at the moment, what happens when you do a local or remote evaluate? It launches a kernel, then does the evaluation and closes the kernel. So what I want to do is to have separate function for launching the kernel. Then you could use that launch kernel, which will look almost.

I get it. I guess it should be called use.

That as argument to.

Okay, so I'm suggesting something. I suggest we call it Start Evaluation session. Okay, and that's the thing that, that then returns some kind of session object which you can then feed on, that.

Just returns a kernel object.

Okay, but then you can feed that to remote evaluate.

Local evaluate.

What's the difference then between local evaluate and remote evaluate?

The only difference currently is the default kernel they use. So you could have. You have a default kernel to use for local evaluate, and you have one that points to your favorite machine for remote evaluate. That's the only difference. It's really the same function. So Start Evaluate session would take a kernel configuration as argument and returns kernel conference Config and returns a kernel object as result, which represents a running kernel and you can then use that for local or remote evaluation.

Yeah.

Yes.

It's a shame that we have that we're distinguishing now between local and remote evaluation. Maybe we just need a sub evaluate or something.

Separate evaluate. Evaluate is not.

What's that?

Evaluate is unfortunately not available.

Oh, no, we wouldn't want it anyway because it's a different. It's a. It's like an independent evaluate.

Yes.

You know what I'm saying. It's not a parallel evaluate, because parallel evaluate means many things in parallel.

Could external evaluate be overloaded to provide this or is it completely different?

No, it's almost the same. It's just that we have these. Start external session and returns external session. And the shame is that we have the sort of separate names and session string based.

Right. I mean, it implies serialization.

Right.

Which we don't need.

No, the. In the mechanism by which valuation occurs is completely up to the particular implementation for that language.

So that line, that usage could be extended to have session as the first argument and any language expression is the second argument.

Sorry, I. I was not paying attention for a second. Could you.

I mean, we are thinking about whether we could abuse external evaluate to. To double duty as local or remote evaluate.

Yeah, I don't think so, because I think it's got an expression as a second argument rather than a command. It's a mess.

Yes.

I don't think we should do that. I think we've got more services that we can provide for our own language than.

I mean, it's not. It's not urgent, but maybe somebody will sometime, some point, have a good idea for a new name for that.

Separate evaluate. Yeah. Private evaluate. That's probably not right. Well, could you write these down, please? Separate evaluate. I don't really like that.

But.

But Private evaluate. You know, what this really is like is you're on a web browser and you go into, you know, incognito mode or something and you do something. And in other words, it's a sub evaluation that doesn't mess up your main state. Do you see what I'm saying?

Yes. And it also does not have access to completely separate.

So that's why it should be called separate evaluate or something. But it's kind of like separated evaluate or something. I don't know. That's not quite the right word. But. But.

Yeah, we can bring this up in one of the EC meetings again, but after people have had a chance to think about it, probably trying to get to the second main point of this.

Yeah, let's get to it.

Okay, so the second main point is obviously that new tab in the system Preferences. So without. Lot of. So this has too many tabs here.

These are by the way. By the way, just another name. Disjoint. Evaluate. It's a bit weird, it's not quite right yet, but we're getting. Okay, all right, let's go to the second thing.

So these tabs at the moment, they are not actually.

I've got another name. I've got another name. I've got another name. Branch. Evaluate.

Branch.

Still not quite right, but it's getting there.

Okay, so yes, so that new thing will. This first tab on the left there is this list of all the kernels you have ever touched. And that can include your favorite. That can include persistent data from your cloud account and so on. And so it's just a browser for all the kernels that you've ever configured and you can edit those. That will bring up the thing that we've seen before, that editor for that individual kernel configuration you can duplicate.

What does pool mean? What does pool mean?

Pool.

Where is it?

Pool.

Oh, that's for WSTP server kernels. The WTP server maintains kernel named kernel pools. And you can choose. You might have a pool from kernel with a gpu. There's another pool for parallel kernels and so on. And there might be a pool for version 13.2 kernels, another pool for version 13.3 kernels and so on. So you can choose which pool to request a kernel from. Okay, that's terminology of wst.

No, no, I understand that. Okay, so this thing of things on the left, I mean mine might be 30 things long, including ones from five years ago. Is that right or is that not right?

Yeah, I mean it keeps. It keeps it persistently until you. You hit the reset button. Also, also, it could be long, but that, that's why there are the filters. So I could say I want only to see only the parallel sub kernels or I want to see all usages, but I want to see on the SSH kernels. So SSH kernels. So you can. There are. There are these filters and obviously it should scroll when. When the list gets long.

What is lwg? Lightweight grid.

Lightweight grid.

Yes.

Okay. Okay, so what happens if I hit go to the parallel kernels.

This one?

Yeah.

Okay, so this is where you specify which ones you want to use by default. So that's your dollar. Default kernels. So these the ones you specify here. Are the ones that are used when you say launch kernels without an argument. And here we could add these named sets of kernels.

Okay, so that is. So I'm trying to get some. Okay, Kernel Configuration Editor is the cesspool with everything in it. Is that right?

So, yeah, that contains everything you can edit. And this one, I mean, another way would be to have checkboxes here, but. So that's something that we need to decide with UI design whether it's. Whether we should overload this. But it's probably already very crowded and at the moment I've chosen just for the.

Okay, hold on, we're zooming out for a second. Okay, hold on, hold on. This kernel preferences thing, first thing is it has the inventory of all possible kernels. Is that correct? Yes, yes, that's what this is. So if I have 100 kernels, I have a bunch of different machines, I have kernels on them, where does that show up? Does that show it on the left sidebar here?

Yeah, I mean, these are the kernel configurations. So you would only have one for all the kernels on that particular machine.

Right. Okay, but I mean, like, like in my case, I have. I don't know what, I don't know, six, seven machines, eight machines. I don't know how many machines.

Yeah, then you would have. Then you would have eight entries with SSH in front for those machines.

Okay. And I mean, to me, the details of what the host name is and all that kind of thing, I might very well want just a human name for that.

Yeah, I mean, that. That is, for example, if I edit this one, this is. I can change the name.

Okay.

And so now it will. That's the name. So far, the name has always, always been there, but so far I haven't actually used it for anything sens. So the name at the moment is rather pointless, but because I have all these different persistence settings, I need a way to unique.

Okay, but so the backing of this whole preferences thing is some expression. Is that true? So, like, like, for example, in my case, I could get somebody to make me an appropriate. Okay, let's say I want. Let's say I ask somebody, make me one of these for Azure. Okay, make me a thing with my account settings and all that kind of thing. So I can just plug it in and use Azure. How do I do?

Yeah, you could. I mean, there is, at the moment, there is no. I haven't mocked this up, but it should be possible to enter a cloud. Cloud object that contains that information or cloud.

Or just enter the expression that contains that information, somebody sent it to me by email, let's say.

Yes, there should be a way to load these. Yes, that's a good point.

Now, one of the tricky things is it might need to interact with my keychain for contains account information. I might want that on my keychain, mightn't I?

Yes. Now that is relevant for the SSH kernels. And SSH has its own key management system.

So how does that work? I mean, if I. If somebody sends me. Oh, this is a remote kernel you could use.

Yes. Now there is a way to. Where is it? Let's see, you can specify the SSH key here in the kernel configuration.

Okay, but is that in plain text or is that.

No, the key is the key. That's the. That's the private key which is usually protected by a password. That's a file on your system that contains the key.

Okay. Can it be appointed keychain to a system credential?

Yes.

So can it be a symbolic.

SSH uses its own system that is usually also connected to your keychain. So you would have to load that key just once and it would be remembered in your login session.

Okay.

MacOS does that automatically.

Okay, okay. Okay. So. All right, well, let's just see to the end of your agenda because.

Okay, so. So this is the tab that allows you to select which kernels to use for parallelism. And this is where we could have these named sets. Now, this tab, which is probably not even necessary, would allow you to choose which kernels that should show up in the Notebooks kernel evaluation menu. We probably want to list all of the ones that are capable of being Notebooks kernels.

I don't think so. I think it's good to have this.

Okay. Yes.

I mean, I'm telling you, I've got stupid cesspool stuff that's from, you know, 10 years ago sitting around there, but I don't necessarily want to kill it. Let me give you an example. Okay.

Okay. You could just turn it on and off. Yes.

Yeah, right. I mean, I'm going to hang out in my office, you know, in Illinois or something. I go there once a year or something like that. I want to reconfigure this because I've got a different machine there. Yes, blah, blah, blah. Okay, good.

Then the next step that just allows you to choose this dollar. Default local kernel, default remote kernel already exists. So obviously it should be possible to persistently choose.

Wait a minute. But wait a minute, wait a minute, wait, wait. That. That's what we're calling. And someone has suggested. Zab suggested Partitioned evaluate as another name for the separate Evaluate this or extrinsic evaluate. Doug suggests that, but I think that's a little bit too weird. But that's got to mirror. You know, it's got to be basically, you know, and this. In these tabs, they've got to be graphically designed differently. So one of them is kernel Configuration. You know, it's basically that's. I don't understand. That's a configuration editor for all kernels. But why. I see.

Yes, you. So you have your 100 different special use case kernels and say, okay, which one do you want to use by default for local evaluate? And then you will can select it.

I know, but that's just like saying which ones do you want to use by default for parallel evaluation? It shouldn't be called default kernels because the one specifically for these, you know, partitioned evaluate or something. You know what I'm saying? It should mirror the name that we're using for that.

Well, is it. Because it's. I mean another possibility would be down here. You could also have the default notebook kernel specified here. This would be consistent with what we are doing here.

Well, are you sure it's the right thing to do? I think it'd be a better thing to do to just say parallel kernels and then what's the default in the Parallel Kernels tab? Front end kernels. What's the default in the front end kernels tab? Yes, that's rather than separately bank switching the defaults, which doesn't seem. I mean, if we're going to do that, the defaults should come first right after the kernel configuration editor.

Okay, so the parallel Kernels tab will. That already contains that default. As I said before, this is. This design is well, kernels to include in dollar default kernels to use for parallel computing.

Is that the sole function of the Parallel Kernels tab? I don't think so. I think it's the thing.

Yes.

Yeah, okay, but it's the thing that should give you the map of. These are the named groups of parallel kernels.

Yes, so because named groups is something new.

Okay, fine, I understand that but then.

For the front end, I agree that default, front Notebook evaluator and default should go here. And then there's the question where they.

Okay, and then.

Right.

Okay, fine.

And then we are left with the default for local Evaluate and Remote Evaluate. And I'm not sure they deserve their own tab, but maybe.

Well, I mean, I think.

We do.

Have Billy in here now if you want to discuss any of that with him. Yes, but we've kind of run out of time now.

The other one is something new that I haven't worked on yet, but it. I think it would be nice to have an overview of all the kernels that are running.

But that's similar to the kernel. The parallel kernel status option.

Yes, but not only for parallel kernels. If you have local session kernels and front end kernels running, then it should be.

Why not just have a kernel status dialog that shows all of them?

Well, that's. I think what this should return.

Yeah, I understand that, but you might as well show it in. I mean, like for example, it will be useful if I've got parallel kernels running. I mean, you know, there are wacky things that go on. Like for example, each parallel kernel could. Could call its own local evaluate. And you wouldn't see that here.

No.

And there is no way I could ever.

Right, but. So this is the things attached to the current kernel, correct?

Yes, just like everything here.

Right. So what I'm suggesting is that we just have one status.

Yeah, the power kernel status menu.

Turn into a channel running kernel status or something.

Yes, that's what this is intended to be.

Okay. Okay, I'm gonna have to go and very, very soon. So what else is that to go through here?

No, I mean this is just a detailed discussion of the various tabs which we have already looked at. So I think we had a couple of naming questions that we settled that we still have to settle. And then I need to think about those named kernel groups and then we should hand this over to design as soon as possible.

I think so. And.

Now the.

Okay, now I assume that the cloud has nothing to say about this cloud.

Is not involved other than as a storage for persistent information.

Right. Because you can't run.

Yeah, no, no. You can use cloud kernels as. Locally as remote evaluate kernels.

No, I understand that, but you can't run at this time. There is no way to run. Okay. There's two things that I think, you know, for example, from the cloud, you can't call an AWS kernel, Correct?

That is correct, yes.

Even though that was a project that famously was supposed to get done like seven years ago.

You're talking about the. About the private cloud or.

No, I'm talking about the ability.

For.

One to, you know, say I want to run this computation. I mean, you know, the thing we've been talking about for years is I've got this computation I want to. For 15 minutes. I want to run it on 2,000 kernels.

Yes. Now, from. I mean, we now have the technology thanks to Jesse's work. Right. So. And in the desktop, this. Once all of this is in place, then we have everything we need to support that workflow about the cloud.

Okay. So let's not worry so much about the cloud there. We can support that on the desktop. That's super easy to give you an.

Idea of the timing and the amount of work involved. I'm pretty sure that if this goes as planned, that without. Except for the front end kernels, this can be put into the next version. The front end kernels obviously requires a lot of cooperation with the front end because, I mean, I don't. I think we can. I can support the current way the front end launches local kernels, but the way it launches SSH kernels and WSTP server kernels requires quite a bit of work because the front end will actually have to use the system kernel to. To help with a new way of. Of establishing the connections.

Right. I. I really unfortunately need to go to the Southern meeting, but I think we have covered a good collection of this. I think we should, you know, time to go through, you know, the form, function, thing design and not a moment too soon. This stuff, I've wanted it for a long, long, long time. Is really lwg. Is that really a transport thing? Are we really going to do it that way? As an lwg LWG colon. Is that what we're going to do? Is that.

Well, that is. That is the way it already works. Now that's the. But yes, I mean, we. These, all these, these, you can just make up your own there. You don't even have to register those. I mean, it's just.

Yeah, I understand. All right, looking good. Okay, I really need to run, but good stuff. Thank you all thanks to folks on the live stream. See all that. Sa.