Live CEOing Ep 837: Language Design in Wolfram Language [Engine Connectivity Engineering]

Okay. Hi everyone. We are looking at engine connectivity engineering, which is lots of software engineering kinds of things. So let's start off parallel tools.

Yes. So there are two topics that I concentrated on for the next version. One is fast kernel launching, where I can give it.

Thank goodness.

The other is parallel table with a deep parallelization of tables. Good discussion of some of the issues on how to specify what to do exactly.

Okay. Fast kernel launching.

Okay.

Is there anything to really show here? I mean, or is it just like it runs faster?

No, I mean, I can show you why it is faster. So I have a couple of very diverse kernel configurations. Local, remote, WSDP server, ssh, local kernels.

I believe you, but use kernel forking. Is that a documented feature? Now that.

That will be in the next. I mean it's already in the code, but because there are some licensing issues, it's not yet documented.

Okay, what are the. For kernel configuration, what are the other things that can go in that second argument? Is it an association generally, and this is just a cheapo syntax that uses that string.

Any number of documented options.

But I mean the real options which would be symbols or are they something which lives in the association and should be strings.

They are strings. I mean internally it will all turn into an association. But I mean for the.

Wait a minute. Can the user. If we're documenting it as an association, then it's fine for them to be strings. If we're not. If we're documenting them as options, then we got to talk about that.

Okay. I mean we can. It's not. It's nothing new. I mean that has been in the code for about three versions.

Well, can you show me what the list of. So the second argument of kernel configuration can be an association, true or not true?

No, this is not true.

Well, that has to be fixed if those are string options.

So these are the. All the. Yes.

Okay, then. Then there's nothing more to say about it. I mean.

Okay, good, that's fine.

And then the. Then the syntax that doesn't involve an association is a. Is a. Is a kind of a courtesy.

It's an option option syntax.

Yes, right, but. But it's a courtesy syntax. It is not the primary documented syntax. The primary document, the moment it is. I understand. So I think it should be changed.

You want to change that?

Yes, I think it should be changed and I want to continue to work. But we don't do string options generally. We do strings and associations.

Yes, in a more long term way. I'm trying to extend the URL syntax So that everything that would require an option or association could be specified in the URL. And so it is in a single place that has advantages for persistent storage.

Okay. I mean, why isn't this like, why isn't this set up in analogy to the way that we do LLM configurations and other things where the actual configuration thing is somehow stored as metadata? The whole symbolic configuration object is stored as metadata.

I would have to look at this. It's obviously younger than this, this code.

Here it is. So why don't you look at it?

Okay.

Does anybody in this group know how it works? Go ahead, please.

Oh, I, I just worry that, that, about that question mark thing and how it might clash with, you know, real, like this is clearly like some kind of pseudo URL thing and I wonder.

If it clashes with a real URL. I agree, I, I don't think it's a good idea.

I mean, I mean, the question mark is this the standard?

I know, but the point that, that Carlo is raising is, you know, it could be the case that you need an option to SSH that is eaten by some innards of some, you know, authentication system. You see what I'm saying? And we don't want to muddle that up with a use kernel forking option which is eaten by the destination, you know, stuff in our code.

But, you know, there are standard ways of doing things like that. Like for example, in database Reference, the. There is a standard magic string which is URL like, but it's not really a URL because it uses a different scheme, which is not a real URL scheme. So you can do basically whatever you want. But I wonder if SSH like that question mark looks like the thing you do for URL parameters. And I wonder, you know, if SSH supports that in any form or way.

But we don't just have ssh, it's not the only protocol. So in general there will be protocols that do not support any given thing.

Yeah, I mean, like this one, which we made up ourselves.

Right. Well, that we can make support anything we want because we, you know, it's our protocol. But, but I. Look, my suggestion is do it like LLM configuration. I don't completely remember how that all works, but fundamentally it is storing, you know, and it's similar also to the way that external evaluation registration works. Right. Which has many options as well. Am I making sense? Maybe somebody here, maybe Ricardo, can speak to that one. Ricardo?

Yes, I'm here, but I don't have a good suggestion here. I don't understand what is Question mark.

Four doing.

It'S starting four kernels. I don't think it's the right way to do it.

Yeah, I mean, I chose that because.

Just a better way to do it would be to add something like kernel equals to four.

Yeah, because this is the most often used one. So I wanted it to be short, but.

Fine. But I think that it should be available as part of the association. That comes as a second argument.

It is. I mean you can alternatively you can always say kernel count.

Can you specify the port?

Yeah, I mean that is a standard. That is a standard with colonial. Yes. I mean that's standard URL syntax and that is also standard URL syntax. And anything after that is whatever, I can do whatever I want after that.

No, you can't. I think it's question mark. And then it's not exactly whatever, is it? Ricardo, do you remember?

I mean it's either a list of query arguments or it can actually be a single thing after question mark.

Oh, we also have functional URL parse that are supposed to deal with URLs according to the standard syntax for URLs.

But why?

This is not an option, just kernel count goes to four. Seems to be a little bit cleaner to me.

Yes.

Yeah, but I want, I want a single thing that I can put into lists and into, into, into persistent storage.

Why, why not put the whole symbolic object into persistent storage?

That always leads to loading the whole framework when you start the kernel, when you have a persistent configuration which is not always desirable. And so if I might say so, then this doesn't happen.

Okay, you know what? That's a hack, you know. Okay, so look, don't we have a tiny piece of code? If we don't have it, we could Write it in 5 seconds. Probably an LLM could write it for us. That parses the trivial format of kernel configuration. Blah, blah blah, association, you know, string blah blah blah, arrow blah blah blah. That's what I think we should do and not try and hack it.

Yeah, I mean the one, one idea is that when I, when I enter something like that, I could always have a URL form that is equivalent and then I could, I could just generate that behind the scenes.

Whatever form you want. I mean, whatever you want. To me, this is, you know, if you want to have some el cheapo string that you can use that you can parse inside the kernel launcher. Right. You can store along with kernel configuration, you can store a hidden option or not so hidden option. That is the string version of kernel configuration. I would Argue that you might be better off just reading the ASCII of kernel configuration in the kernel launcher. Okay. Because the format here is going to be very standard and you can set it up so that whatever is entering kernel configurations canonicalizes them. Right. And then it's a trivial piece of code to just parse the ascii. But if for some reason you don't want to parse the ascii, just turn this into, in the metadata, turn this into something which you can readily parse and you know, which could be. I don't care if it's URL string. I wouldn't, I would have thought it was not a good idea to have it be a URL string. I would have thought you should have, you know, triple colons or something separating the fields or something like that, or new lines or something separating the fields. If you want something that's easier to parse than this. But I don't think this is very hard to parse. Makes sense.

What we have done for database reference, which looks similar, it's a URL and then some option is that basically you write that guy and if it is a string, it evaluates to an association. Probably you can do the same and then store the whole association. Why the association should auto load anything if it is just an association?

Yeah. Okay, guys, you can solve this. This is elementary software engineering. You guys are a hundred times capable of doing this. We don't need to go over this in the speech.

Okay, so let's see whether. So what will happen? And it may.

Let me see it.

I don't, I don't, I don't know how quickly the. The REST display. Okay, so. So in the first stage starts to launch and then, and now it's already receiving all connections and you can see update for all of them in parallel. And now it's setting up the kernels and now it's done. So this was 126 kernels.

Well done. It's going to make my life a lot easier.

Yes, I mean, I have some slow machines here, I have some machines on the other side of the Atlantic and so on. So.

Right, okay, let me, let me. This is great. It's going to improve my quality of life. But I need one more thing, which is I need a way to launch. See, I often have things which use GPUs, use external programs, et cetera, et cetera, et cetera. Error. And I would like to be able to say, for example, in my parallel evaluate, I'd like to be able to say. Instead of one Kernel per machine. I'd like to be able to say local or something like that in the parallel evaluate so that I can evaluate it on my. How many kernels I have on my local machine, which I know has whatever crazy thing configured or. I want to be able to evaluate it on a. Okay, yes.

I mean, this, this, this just goes by type, right? So it doesn't know that. It's just, it's those kernels that you specified as local.

What do you mean? Those ones you specify as local. So I don't understand that.

Well, I mean, you, for example, you could also have an SSH back into localhost and it would not be considered local.

Fine. So, but, but I got further thing to say, okay. It would be really good if somehow we had a. Capabilities, you know, array for the kernel for each kernel configuration so that, for example, I could say something like, you know, GPU or some such other thing, and it would only run on my machines with GPUs.

Yes. So I mean, this is the parallel environments idea that we discussed last time, where we decided that we would support several concurrent parallel environments. And so you could have one with only local kernels and one with your GPU kernels and things like that, right? That's already noted. Yes.

Okay. All right. But not done. Okay, I've got one more thing to raise. Apparently there's an issue with system credential and ssh, which has been biting me, and it's some new feature of some Mac operating system, I think. Let me explain.

Yeah, I mean, there was a. There was a problem on Linux, right? I mean, there was a. There was a library. It's to do with the keychain. I don't think it has anything to do with you, Roman.

I don't agree that it has nothing to do with Roman. I think that because it's the mechanism used to launch remote kernels with ssh. See, in other words. Let me explain. As I understand it, the issue is that programs can no longer retrieve things from the keychain on a Mac. Okay? Only users with GUIs can retrieve things from the keychain. That's my understanding. Does anybody else want to correct me on that?

Well, I think Harman should talk to Roger Dooley after the end and see.

What we can do.

How many more. I think it's mostly a machine configuration issue. So how many more Mac OS versions before the machines become, become completely useless?

I don't know. Some ways are getting better, but. But who knows? Okay.

Okay. I mean, it's, it's I don't think.

This is not one of. This is not an SW special. I'm the only person in the world.

I mean, I don't. When I use ssh, I don't manage the keys. I let the keychain or the SSH agent do it.

I understand, but what I'm saying is that something has gone wrong because we have a system credential mechanism in, you know, which interacts with the keychain. And I think that's what Roger is using on my machines to manage the SSH keys, and it broke.

Okay, so you might have to go back to using the keychain itself.

Oh.

I mean. I mean, I don't think you.

I mean, you might have to use some SSH encrypted file thing.

Yeah, okay. Yes. So, I mean, I can help debug this and find out.

And then we'll need to generate a workflow for other people to say how to make this work. Okay.

Okay, good. So, I mean, another small thing is I don't need to show it here is that I now have an explicit note in the. In the progress display when the kernels have been launched, but are receiving large chunks of shared of distributed definitions, because that would also sometimes look like a hangman. It's just a user error to distribute gigabytes of data to kernels.

So question when things are coming back after. After the progress monitor says all computations are finished. Okay. There can also be a large data transfer, I think that is unprogressed, if you understand what I'm saying.

Yes, I see what you mean. Yes, I need to look at that. It would be nice. Yes. If that progress display remained up until.

Transferring data or something.

Yes, yes. That would help in avoiding tech support calls.

Right. Okay, excellent. All right. To be. To continue here. So, yes, I want to say that a general direction that we absolutely need to go in is, you know, I'm sitting here, I say I want to run this in parallel across a thousand machines. Okay. I really want to be able to do that by, you know, assume I have an AWS account or better assume that I'm just paying service credits to us. I want to just be able to do it. So I'm saying yes, yes.

I mean, this idea has been around several times. I mean, we're well aware.

I want to see it actually done.

Yes, yes. So if we can get some more help in our department, then it will happen because some of the key people who were critical for that are no longer available.

Okay, well, fine. I think there's an opportunity there. We'll talk about that offline. But that's a. To me, that's a project. I hope it's in gpl.

It is, yes.

Okay. But it's an important project. It's really a thing that is a.

I mean, we have. Coming up in one or two weeks, we have another admin meeting where we can talk about these kind of things.

Okay.

Okay. So parallel table. That is something that you ask for.

Indeed I did.

So let me explain why the current code, the code is as it is now. At the moment, it only parallelizes the first dimension of a table.

Yeah. Which is not useful.

And the way it does it is that it doesn't evaluate the table on the master kernel. It symbolically dissects the table in as many pieces as there are kernels, and it sends these compact representations along.

Right.

So that is, if you have a long table, this is the most efficient way to do it. On the other hand, I could argue that if you do have a long table, it's probably not one that you can parallelize efficiently, because you always have a long result. It's completely different with sum and product, which just collapses the result. And so for sum and product, this method is most likely the most efficient one.

I don't understand. Look, look, look, look. Let's assume in a table, let's assume that there are fixed limits for the table, for a start. Okay. We'll deal with the variable limits in a minute. Let's not muddy ourselves up with the corner case. Okay. Let's say fixed limits. See what I'm saying?

Yes.

Even if they're variable limits, it is trivial to determine what the complete set of values of the parameters of the table are. Right. Trivial to determine that up front. Right. Given that whatever shape it is, you know. Yes. If they're small subtables, you don't want to break them up. You know, if there are a large number of crunchy tables with the parameter I, you don't want to break them up in the parameter j. But, you know, if there are. If the whole. I mean, come on, you can. You can make up a heuristic for this.

Yes.

Is that what we're talking about? Because that's what we, you know. Okay, step one. Understand the. The shape of the thing you're trying to parallelize. Right. Could be. I mean, here's what we know. We know, in the end, every individual I and J value can be separately evaluated. Right. But it could also be that. That you don't want to break up the J evaluations because, you know, because. Why do it? Because it doesn't make any sense. Because the number of kernels you have is comparable to the number of I values. And there's no reason it's not going to do any good to break up J values. Okay, so you got a bunch of heuristics like that to determine what to do mean, like if the number of kernels. It's almost certainly a good idea if the number of kernels is such that you can have I times J, you know, IMAX times jmax, more or less things on separate kernels, it's probably a good idea to do that. There may be a corner case where it isn't a good idea. And I strongly suggest that you unravel things in the triangular case of, you know, the limit of J being. Being dependent on I. Yeah, right. You should untangle it first. Right.

But I mean, of course, I mean, here. There could be anything here, right? Or any function.

I know. So untangle it.

Okay.

That evaluation can be done independent of the payload, right? It is an independent payload thing. It does not have. It is a, you know, it is a. It's like, what is it? Course of values recursion type thing. It's a thing where, you know, it's, you know, before the MU operator. You know what I mean?

Yeah, the.

It's, you know, it's primitive recursion all the way down or something. In other words, you, you know, how big. So you should get the shape. Okay. Then figure out a reasonable way to divide the shape. You know, we're assuming no side effects. And then you do your thing. Right. Based on how many kernels you have and whether, you know, it's. It's really pretty obvious. I mean, like, like if the thing is, you know, I goes up to 20, J goes up to 20, and you have 200 kernels, 200 cores. You know what to do.

Yes.

You. You don't. Right. And, you know, just figure out reasonable heuristics if you really want. I'm sure other people.

Steven, may I have one comment here with this? Basically, there are cases where. Where if you do, even if you have a lot of kernels, but if you distribute like single computation to a single kernel, you can lose a lot in SPE example, when your expression can be autocompiled by a table. So it's not quite. You need to have some weight in terms of the intensiveness of a single computation for the function.

Maybe you know that. I mean, maybe you do a computation and find out. I don't know.

Yeah, but all I'm saying is that there is another variable here, which is how intense is a single computation.

Yeah, I mean that, that is a general. I mean, if that, if the thing in the table is trivial, it's never worth paralyzing because it takes the kernel a lot less time to generate than to send the command and receive the result.

Yeah. So, but maybe there's a warning message. Maybe the progress monitor in that case can make it can indicate that you're probably wasting your time. I don't think this meeting. Please, guys, work together to figure out heuristics for this. I mean, I'm sure I could figure them out for you. I don't think I'm the right person to do it.

Okay, so let me show you what I'm experimenting with at the moment is the complete parallelization of tables. So you go all the way and in the innermost case, that single computation is then scheduled for parallel computation.

So I understand. So you're either doing. You've got these two cases, you want to know if you should use a method option. I'm trying to hustle us through because we got. I've got another meeting.

Yes, I mean, of course I could also stop anywhere in between. I could say if I have five dimensional table, I parallelize three of them and there the heuristics would be helpful to see.

If you write parallel table, you probably want it to be parallelized. If you didn't want it to be parallelized, you would have written table.

Yeah.

Yes, because table and parallel table can nest arbitrarily.

Yeah, right. You could have, I mean, you have an explicit parallel table or you could do parallelize of table and then it's less clear what to do because then I need to analyze.

And so you could say parallel table of table and then it will do the inner tables on a single.

Yes, of course. Here's an example. Yes. So yeah, that would, I mean, that would be for. For those cases where it gets it wrong. Right. So my question here is, I mean, we have a method option which is geared towards the old case where you how to hack up the individual computations. And so is there a good way to specify the depth to which you want to unravel the table? Is something like method depth, acceptable, incomprehensible.

Why isn't coarsest grained and finest grained?

I mean, coarsest grained could be construed as meaning first level at the table and finest grade, lowest level.

Why don't we say the method could be, you Know multiple things there. So. I would say table parallelization level or something. Table parallelization depth arrow N. Yes.

Do we need the parallelization here? Because it's already a method of parallel table.

Oh, table depth seems incomprehensible.

Okay.

But I'm thinking what we could do. Table flattening level. Table flattening depth.

It doesn't flatten. I mean it doesn't change the structure.

I mean I. I would call it table parallelization depth. I think you call it table depth. It's just really incomprehensible.

Okay. No, that, that's a. That's a reasonable. That's a reasonable name.

Okay.

Should it be something like maximum table polarization there?

Max.

Like what if one.

Yeah. Yes. I mean up to. Up to two or something. I mean we do have notation, right?

Color is making the point. If the table is only of depth one and you ask for depth two in the parallelization, what's it going to do?

It's not going to error, I hope, right?

No, I think it's implicitly max. I wouldn't bother to say max.

Yeah, I mean the default would be minus one.

I'm trying to hustle because we really. I have another meeting. We have to. Okay, so can we get through this please?

Okay. I mean that was really all about tables. I mean for two summon product may need different defaults because they are the result collapses.

Think it through.

Okay. And other than that we do not have any urgent items. We have an update on R link and external sessions with R open the tab. So Leonid has a demo if you are interested but.

Well, I mean you can just.

Leonid, you can just tell. Just update us here.

Yeah, instead of a demo, if we are short on time, I can just verbally just say briefly, what's the status there?

Good, what is it?

So basically I've done a number of infrastructural improvements over the last half a year and the biggest one that still remains to be done is to implement the multiple R link sessions. Because currently R only has one at a time can only support one session R link. So that's one big stumbling block for like to be able to build something for example on top of R link. And then the other thing is the ring virtual environments, they need to be made more robust and probably faster because right now it takes quite a long time to actually create a single virtual environment in R. So I'm going to look into what Ricardo recently discovered for Python, those new package managers that might also provide faster way of providing like provisioning our Virtual environments. And then the last thing.

Let me ask a question.

Go ahead.

But our language, why do we not need for other people using us, do they need a virtual environment? Should we provide such a thing or do they not need it because our engineering is kind of more modern and better and so on?

Well, virtual environments are typically needed to isolate whatever. Like let's say someone builds something on top of Ruby, it could be us, it could be someone else. But if they want to separate that from any other R related functionality that they have, be that their own R installations that they use for other purposes or whatnot, then we do need.

So I'm asking the question then, for Wolfram language, should we provide a virtual environment? Oh, for Wolfram language, yes, that's what I'm asking.

Yeah. Well, it has to be thought. I mean, we have to think about it. But I'm asking.

I would argue that it's not super useful because it's like we are. Everything we did is very monolithic. Like you don't expect people to need conflictive version of the same packlet. That's what they're good, right?

I mean, maybe the, maybe these virtual environments are dealing with a mess of these, you know, library rich, you know, sprawling systems.

I mean, lot of conflicting packages. So it's.

Yeah, which is not our issue.

If we, everything we have is matching.

We become massively successful and the packlet repo explodes, then it might become useful.

That's because when there's a mess, you're saying it's for packaging up the mess so people don't see it. But we don't have a mess to begin with.

Steven. I personally partly agree with Carlo, but not completely. Because if we, if we think from what we have right now, then I agree with Carlo. But if we think about the possibilities that we could have, for example, more granular code, you know, like the packages which are like WFRs, which are pretty small. And then if we also think that eventually many users will have their own and want to avoid all kind of conflicts automatically.

Okay, I understand. I got it, I got it. We do. This is not a problem we have right now. We've done a better job of, you know, keeping consistency. We just don't have to worry about this right now. Not an important thing for us right now.

I mean, it's not just we've done a better job. It's also that I think that in the packet repo, most of the things are written by Wolfram employees. So we like.

I think we've done a better cultural job. Yeah, that's what it comes down to. But anyway, it doesn't really matter. Let's move on. We got to move on to another meeting. So, Leonard, you had one more thing to say?

Yes. I also wanted to say that I really need to move our link to better error handling. And I'm going to use the new error handling functionality that I also am working on with the other people in the error handling team.

Good.

And this will serve both purposes. It will improve our link and it will improve our handling.

Good. And I want to hear about error handling at some point before, too. Very sure. Okay, quick, quick look at IDE support, and then we need to move.

Okay, there is. This is almost done. The new version. There are just two items. One is waiting for legal clearance, and the other is uploading the code to our GitHub account and to Microsoft.

Okay.

All right.

I don't. And that's great. I mean, it's great. But is there anything we need to talk about here?

No, just this update.

Excellent. All right, thank you all very much. See you all soon.

Okay.