Workshop: Hands-on Paclet Development

We've got Bob here, I'm Todd. Rick is over here. Right. Rick just walked in. Connor unfortunately can't be here and Ryan is our documentation expert. He is remote. Okay. So I think the plan here is that we don't have anything to present. We're just going to mill around and assist people in what they're doing and talk to people individually and we can do questions. Yeah. In fact, maybe we can start out with a few questions. The question was, and this is probably a common thing for people is I've got a package file. What do I do? How do I start to make a packlet? And so one question we asked was, are you intending to submit this to the packlet repository? And we decided the answer to this question would be yes. So let's assume that for most people the answer to this is going to be yes. And, and so Rick was saying, I mean the first thing to do is to do Bob's doing here on the screen. Right. File new. Are you going to show them the.

Oh yes, this could be. Or you could go to repository. It will be in the file menu in 13.2. But yeah, but you can just do Create Notebook of Packlet Resource that like I have here on the screen. Oh, here, I'll, I'll magnify this. All right, so just that Create Packlet of Packlet Resource which will open this dialog and then you'll want to choose where you're putting your packlet. So I don't know. This is Todd's computer. I don't know where I should put.

That's fine, that's fine. Right. Documents directory.

I can. Okay, so. Oh, there's already one.

My packet is going to be a name in my document.

Can I ask a question while we're going? I was hoping that, you know, following Bob's talk yesterday, like we can actually talk about some of the trade offs like you already started with, you know, what's step one? Can we talk about like the trade off, the pros and cons of why you, what you might be trading off when you decide to share locally with a small group versus sharing larger with the world.

Right.

I mean there's privacy and security, of course. Right.

Well, yeah, more than that. A little bit more than that, please.

You know.

Yeah, like you're not going to be able to get this back. You know, whatever you put in here, it's gone. But you might be trading that against the ability to sort of collaborate on GitHub with a gazillion other people.

Right.

So like I think that would be useful to talk about.

Yeah, yeah, that's a great idea. Do you want to like work with him, Rick, while we kind of take questions? That'd be good.

What I've got are two functions.

Well, here, let me talk about. So, right. So, yeah, so I mean, I very much want to encourage people if there's no reason not to put in the public repository, to put in the public repository and put the source in GitHub or some other public Git source control. That way collaborative development can happen via GitHub. We're not trying to compete with GitHub, we're not trying to create a new source control thing. We're just like a release process. So, yeah, that's definitely what I'd like to encourage. I think the reasons not to do that, I think is basically like, if the design of your packlet is very transient, you know, you're kind of prototyping, you might make a bunch of breaking changes, then you might kind of put out a 0.1 version, just cloud deploy it or send it around. And that way it's very limited how many people are going to use it. And you can tell them all, hey, I changed this, update your code. But if you put in the public repository, it's probably best to try to avoid like those big breaking changes because you don't know how many people might be using it.

Right. I feel like it's like it's not, it's not just like me. I feel like it'd be a good discussion.

Yeah.

Because there's for example, like the decision to clone or fork inside a GitHub versus a decision on the other, maybe on the other side where you're sharing it inside a community.

You know what I mean?

Like there's, there's quite a lot there.

Yeah, yeah, yeah.

Are you, are you going to do source control of the master node in GitHub or are you going to do some sort of like, you know, own all of it?

You know what I mean?

No, I'm not sure. So who's the you in that question?

If you choose GitHub, load it to GitHub as your distribution mechanism and then you allow it to be cloned or forked and let GitHub manage the forking and recombining of branches versus distribution within any other mechanism inside Wolfram, you're now going to own all of that, Right? You're gonna have to do like, are you gonna fork it? Are you gonna allow people to branch?

Right, right, right, right, right, yeah, yeah. Yeah, yeah, yeah, yeah, yeah, yeah. Right. He's just saying like, what approach do we recommend people do? Right, yeah. So if you do that, if you do GitHub, you accept some burden of sort of owning the repository where if you just made a packet file and put it and post it somewhere on community or in a cloud object, or then you're sort of washing your hands of it, I guess.

Possibilities. Right. If you stay in, let's say you choose consciously to go into the GitHub and allow it to branch. There could be examples where that draws a lot of energy and it becomes. It lives, it has its own life.

Right.

That you essentially don't control anymore. Largely.

People can't do whatever they want.

All right, you could do that too. You could control the license. Right. But, but again, like Linux didn't have a license. Not really. It became a whole nother thing unto itself. I had a hundred thousand different programs contributing. Like that could happen in a packlet world.

Could happen if you. Yeah. I mean if you just opened everything up and you authorized other people to merge, pull requests and that's, you know, you could, it could be very distributed and collaborative.

Interesting. Like, I don't think the sense that like many people here think in like a DevSecOps kind of way. Like it seems like it's. They think differently, but like it, it could be the beginning of allowing DevSecOps people to sort of make use of the Wolfram.

Yeah, I think. Yes, yes, absolutely. Yeah, that's definitely what we want. Yeah. Sort of, kind of be have people realize that we are a proper development environment that just like, you know, and that you can do all these products. It's not just a thing that lives on a guy in a notebook on someone's computer and they maybe email it around. Right. You know, it's a proper development environment in the modern world. Absolutely.

Like that choice, you made it like early on and it was sort of like an almost like built like when you selected MIT license, like the decisions around like what license do you choose can have dramatic out.

Absolutely.

What happens to your code?

Absolutely. Yeah, yeah, yeah. I'm definitely not a expert on what recommending licenses, but yeah, yeah, it's definitely something to think about if you're putting something out there that you want to maintain some sort of control over.

I was also thinking about the version. I mean again, your form, maybe it's early days still, but like the form, like how do you decide about minor version releases versus major version releases?

Yeah, so we're giving. Yeah, yeah, so we encourage people to use semantic versioning. So, you know, like a breaking change should be first digit thing. Right. So that if people have been using one point something and liking it, they know that like if they have 1.2, they probably are safe to go to 1.3, but they might not want to install 2.0. Right.

Is that spelling anywhere?

Yeah, yeah, yeah. So if you've published and it's already in the public repository and then you make changes and you go to publish again. I wish I could demo this, but it's like I would have to. It would take me most of the time to contrive an example. It'll pop up a dialog where it says do you want is this minor major or bug fix? Right. Something like that. And I think that's the terminology we use. And the bug fix is the third digit. We want you to do sort of the three digits and you know, little bug fixes that everybody really probably is going to want should be third digit. You know, incremental additions to your packlet should be second digit. And then like if there's a big breaking change there, you are fundamentally overhauling things and people will need to change their code in order to account for it. Then that's got to be first digit. That's the big one that we really want people to follow. Right. The difference between the second and third digit, there's definitely gray area there. Yeah, yeah.

One of the things I've noticed here, I'm trying to create a simple to the accurate resource notebook. I just downloaded that. It's the first time, but I've noticed. What I've noticed on the Mac is that I have my desktop.

I don't know if you're familiar with.

The Mac, but I have my desktop and my documents directory on the cloud using the cloud. So it's not on my.

Okay. And.

But, but Mathematica does not recognize those as being directories. So I get an error that says directory. It's not a valid directory. So it doesn't recognize desktop or documents as being a directory. It seems to know that it's on the cloud, even though the naming sequence is interesting.

Yeah.

Those two directories have a cloud path. So if you go to slash walls.

You'Ll find you can find the right.

Yeah.

To those two directories and you can give that path to that.

Just to say that the naming convention, you would expect that to function appropriately.

Yeah, yeah, yeah. That's not a case that I've run into because I, you know, I have them local. Sure.

I'VE got a regular directory. To me it's a.

So, yeah, so he's getting this message where he chose a directory like in this dialogue on the previous step, and it gave him a little red message underneath. It says it's not a valid packet directory. So what the director you choose either needs to be an existing empty directory, so with nothing in it, or it needs to be a valid packlet, which means it has a packlet info or packlet info wl file already. So it's got to be one of those two things. So if you, if you chose a directory with contents in it, that and those contents aren't a packlet, that's when you get that message. So you probably just want to make a new empty subdirectory in that and select that directory and then you'll probably be off and running.

Yeah.

Right.

Basically right now.

Right. Okay. Okay. So, right, Versions. So the question was how do we handle versions? And especially like having multiple versions concurrently and being able to choose. So. Right. So versioning packlets have a long history where versions are very important to it. And the way that packlet managers work is if. Users, look, a session can find a packlet, they're going to use the highest version they can find that's compatible. Right. So there are tools to enable and disable specific versions that we probably should build out. So that's from like an end user perspective. Right. So you can disable a specific version. There's a function packlet disable. I think that then you would fall back on using Go1 from a developer point of view. Yeah, yeah, we need to kind of. Yeah, you can also install a specific version, but it's likely that you would. It's easy to accidentally then upgrade later. Right. You have to be careful not to then accidentally install the update, which you can do. So you can say packlet install of here, let's see, let me cancel this. You can say packlet install of Wolfram packlet CICD. There's this. I can say like 1.1.0 if that's. If that's a version that exists. Okay, well, that version doesn't exist. I can do packlet find remote of this. See what's here. Okay.

0.33 star. It would have found the latest biggest number.

So this might give me a warning that says I have a newer. Okay, so that just did. So you can do that and then. But then you have to be careful to not, you know, ever do like the one argument form. A lot of people use like a one argument form in their code to like ensure that the pat that some version is installed. So like, that people just like put this in because if it's already installed, it's like a zero out. You know, it's really, it's. It doesn't cost you anything. So you have to make sure not to do these if you want to make sure that you're pegging it to a specific version. And then from the developer's point of view, we definitely plan on building out like administrative, like a publisher portal where you can have a bunch of versions and you can mark one as like beta that people won't get unless they ask for it. That's still in the works. That doesn't exist yet. Right now. If you publish it, it's published. But that's something we're planning on. Yeah, sure, go on.

Like actual software.

Yep, yep. Yeah. So. Yeah, yeah, I think that's probably, I mean, so you can, you can't run multiple versions of paclit concurrently in a session. You can have them installed, but you can't, you know, it's just a namespace where symbols have down values and if you load an old version, a new version, then the down values are all going to be there. Right. So you can't, you can't like sort of run them concurrently in the same session. That's never, never going to be a thing. But yes, so you can, you can, so you know, for, you can obviously you can have a development version that you're working on locally and you can load that with packet. Directory load is a great tool, but that, you know, that's not bringing it down from server, that's just running the source code that you have in your, like your local git repo. So you can do that. You can also, like a common idiom might be like have branches on GitHub where you do builds and you have packet files on GitHub and then your team can grab those from there. Right. So then other the general public when they do packlit install won't see it because it's not on the packlet repository, it's just on GitHub. So then you just need to point directly at the URL of the packlet file on the branch and you can use whatever sort of branching scheme you want to come up with. Just as long as the version of the packlet that you've built there is higher than what's on in the repository. So that's sort of a workflow that I think is common and could work. Yeah, you can also Cloud deploy packlets. So if you saw in my talk yesterday at the end, I didn't publish in the repository, I cloud deployed it and then anybody can use it, but they need the URL. So just packlin install the name isn't going to find it. But if you pack install the URL, we'll grab it if you know where it is. Again, if the version's higher, anyone will start using it. So that would be another alternative to just having releases on GitHub. So there's lots of ways to sort of create those test workflows. Yeah, but I mean. And you could create. I mean, if you're really invested, we can help you get a private cloud with a packlet repository on it to serve as your dev and test environment. We can do that. It's going to take some work and some money though, I think. Okay, yeah, yeah, okay, sure, sure.

Can you on line 29 change the dot 2 to dot star?

Would that work?

That's a good question, I think. Oh, nope, it won't. One thing you can do, you can use. So packlet objects are also resource objects now, if they're in the packlet repository. So I don't know if people are very familiar. And that. And that has this resource version that does support things like that. I think that should work. That wildcard. Just set this to something. Right. So that. That works. And you can even do things here like greater than. Yeah, I think things like this were.

Oop.

Yeah. So you can even do sort of like Wolfram syntax or like semantic versioning with between greater than that sort of stuff. And then, and then once you have the resource object, you can either you can do packlet object of the resource object, I think. There we go. You can do packlet install, I think, directly on the resource object. So, you know, the packlet repository is built on the resource system, just like the function repository and data repository. So under the hood everything has a resource object, which is not true of the sort of legacy packlet server that's been around forever. So you can, if you're familiar with resource objects, you can use all those tools. And then, and we've tried to make it nice to sort of switch back and forth between the packlet world and the resource world. Other questions or anybody want help with anything? Sure.

Parallel with each other.

I think that's fine. I think data packets are distributed as packlets, right? They're, they're, they're proper pack lists. They're just pack lists where the content happens to be like an MX file Or something. Right. So. So yeah, I think, I think that's good terminology. It's like, you know, and I could imagine like people talking about this is a style sheet packlet, you know, where basically all it's delivering is a style sheet or this is a documentation packlet. We're basically all just delivering. Yeah, I think that's good. I think that's good terminology. Yeah, yeah. Bradley or back Bradley.

If I have a packlet that is.

Using functions from say from 13:2, but I've declared the version for 13:1 does like the check button know that or see that?

Yeah, that's something that we are actually adding. So we have all these check buttons that do inspection and I don't, I think. Rick, is the, Is the inspection for symbol versions in 13.2 or is that still coming? Okay, it might not be in 13.2. It's coming though, where if you mark it as. If you mark your packlet as 12.2 plus and then you use confirm and enclose, we'll say, hey, confirm didn't exist until 12. 3. Maybe you should, you should increase. Right, right. So we're planning to kind of automate that. Yeah, yeah, yeah, yeah, yeah, I guess. Yeah, yeah. It might not be in. It might not be in yet though. But that's something we have at least, at least on a branch we have that. Sure, go ahead. Sure, sure, sure. If anybody wants to ask questions, just, you know, yell. There's a couple. Also there's a couple of sort of half, half baked functions in the function repository that can be helped, that can help get started. Let's see. What browser does Todd have here? Okay, let's see, let's see what. All right, So we got these. Paclitize and paclitize resource function. Paclitize. It kind of, it works. It's not the best, but it can kind of get you started where here. If it'll create a packet layout and it'll kind of copy content into it and that content can be files or it can be like definitions that are in the current session and it'll kind of get you started. That can be a helpful tool. We're going to add more of these. We're going to add more of these sort of like convenience functions to, to help you get started in the coming weeks. Probably. Anybody else got any questions or want look at anything, otherwise I'm gonna have to start telling dad jokes. Okay. Yeah. So you can just copy the input, copy the contents of the input cell and paste it over. Just normal copy and paste yeah, yeah, yeah. Sometimes, like, line breaks won't be good and you want to go in and modify those. If you happen to be using VS code with the extension. With the Wolfram extension. I don't know if you saw Brenton talk on Tuesday, I think. Ok, if you happen to be using VS code as your editor for WL files, there's a like format, automatic format selection that does a pretty good job of like getting all the line breaks and white space. Nice. So, yeah, just simple. Yeah, absolutely. Yeah. If it's. So let's. Here, let me see.

That.

Palette.

Significant.

Yeah. So if you're pasting things in, you probably want to paste the code like in this area within the private context. And then if it's a symbol that you're going to want end users to use, you probably want to declare here in the packlets context. Right.

So just for practice, I want two functions to be in separate WL files. Just.

Okay.

Just to see how multiple. Multiple files function.

Yeah. So there's, There's a. Right. So, yeah, the way this is set up, the default, it's like everything in one package file. But most packlets that you'll see in the world have lots of them. Right. In this kernel, they'll have a lot. And there's sort of a few different, like, paradigms that people use to control loading. A lot of times people will take this, Take this file, and basically all they'll have in it is like, you know, git. Okay. And that's. That's in GitHub actually could search for there. Yeah. Okay. Is it in the Wolfram? Ah, okay, hold on. Zoom is. Oh, okay. All right. Did I. I'm still sharing. Okay. What? I've exceeded a secondary rate limit. Todd, how much searching in GitHub have you been doing? Oh, conference is like the conference WI fi you think is. Camel case. Okay. Right. So here in the kernel directory, there's three files. This is basically exactly what you're saying. So there's two that are dedicated to specific functions that look like a lot like the ones we've been having. And then if I look at sample packet, all it's doing is calling git on those two. So it's like a. It's just a loader file. Right. So the, the. This is a very common approach where the WL file that shares the name of the packlet, all it does is load all the other WL files in the kernel directory. Right. So this is. You'll see this all over the place if you start looking around at Packlets and then each of the other files has the begin package, begin private and end package stuff.

So.

People already got notebook for their patent.

New notebook, maybe had some fixes in it or something.

How do they.

Yeah, yeah. So that, that I'll answer loudly for everybody. So we had a question about the definition notebooks and like how we keep definition notebooks updated. And this applies to packlet definition, function definition, data definition. So all of these notebooks under the hood are like templates that can be filled in by metadata. And when we push out and when there's an update to the template and you open your old notebook, the first time you try to do anything, use the check button, open the toolbars or anything, it's going to see that there's a newer version of the template available and it's going to ask you if you want to update to that version. And if it does that, it grabs all the metadata from the notebook, applies it to the new template and you'll get all the new features of the definition notebook. So that updating mechanism works. And similarly if you downgraded, so say in a year you make something in version 14, but then you try to bring the definition notebook onto your computer where you're still using 13.1 or something, it'll downgrade as well. That usually works. Obviously you'll lose the new features. Sometimes you'll get some. Yeah, I don't remember where we landed. Hey, where do we land on policy wise for auto updating in the repository? We allow it. No, we don't allow it. We don't. Just by policy, we don't allow it. Users have to actively update. Yeah, but we do plan on rolling out like a notification system at some point. Yeah, you can see that with function repository items. Now if, if you, if you're using a resource function and there's an update that's available, the next time you use it, you'll get like this blue box that says there's an update available. Steven even showed it on here. I can, I can if I.

I think that's true. I know it's true.

Right. The problem is that you don't like invoke the packlet actively in the same way that you do a function. So the time when we would prompt you is not clear enough. This is like simulating like I'm starting a new session. Right. So we have these, these, these are this sort of a new UX element where for resource functions it notifies either there's an update available and we'll probably try to work these in for packlets somehow or have some sort of notification, but it's not in place yet. That's still on the to do list. Yep.

I mean that's one way to get that behavior now.

Yeah, that's true. And just like in the loading, just like at packet load time, you could just have resource object of your packlet name and then this would happen. Yeah, that's a good sort of convention. So you could. So, so yeah. So in addition to resource function, you can do this with resource object. So let me do this again. So actually. And it'll probably work with. With a packet right now. Right. Well, from ICD. Yeah, we were just playing so 0.32.0 or something. So so then if I. If so then if I had in. In the actual code of my packlet, if I just put this so that it evaluated when the packet loaded, not like in the right hand side of a function, just at load time, then this is a message. Even though it's blue instead of red, it works just like messages. So this message would be issued to the user when the packlet loads and they would see. Yeah, so that could be used right away. Yeah. Although this will just update the research, it won't install the pack. The new version of the pack that it'll only. We need. We need to change that. Does it. Can I just. Why do I need to quit the session? I didn't actually install this, so I don't know. Yeah. Yeah. Oh, that doesn't work. That should have worked. Yeah, maybe I messed up my percents. That's. Well, that's a bug. That's not good. Is this a bad version? Oh, wait, hold on. Oh, I did.

Okay.

Oh, what happened there? I'm just being clumsy. That still says. How's that?

You're getting information from the.

No, it's the same one. This is the local. Yeah, I mean that'll tell me about the cloud object, but. Oh, that could be it. Let's try. Let's do this and let's do. All right. This I think. Okay, so now we got 31 and then we want to do this and if we say update now and then we do this, is it going to be now? See, you still have. Yeah, we need to do that. We need to make that work. Yeah, that's something we need to make work. That update button needs to update. Install the new backlit. Yeah. From like from your doc pages. Yeah, yeah. So if you need to have documentation installed or if you have documentation installed, you should like in docu. I know how to do it in the internal docu tools, but in documentation tools there's I think, a link section and it might even have like link to guide in it. Yeah. I don't know if Brian's still here. It does, yeah. There's link to guide and under more link tools there's link to system guide too, if that's interesting.

Yeah.

And I think it'll like. I think if you have the docs installed, it'll like list them all for you to choose from and you don't need to know the exact name or if you don't have docs installed, you probably need to know the exact name, which isn't hard to do if you can find the web page.

It.

Ever tried eating a clock? It's really time consuming, especially if you go for seconds. There you go.

By the way, two WL files. I mean you give an example. So I'm way behind this.

No, we're here for.

So you had an in the needs creating this overall WL file, do you have to put the entire. The whole context in? When you pull the. Use the needs in the. In the top level WL file you have multiple functions.

Okay, yeah.

Multiple WL functions.

Well, usually you'll call get not needs there. Oh yeah, yeah. The greater than. Greater than. Yeah, yeah, yeah. Okay. Yeah.

So is that the full context name that you have to use?

So the convention is that actually is like the relative file path with backticks instead of the file separators. And Todd can tell you how that can run. Like if you have a loader file. Like if you're. If the WL file that has the same name as the packlet is just calling git on a bunch of other package files. The way that you specify those is like the relative file path from the kernel or from the packet directory with the. Okay, but if the context has the same name as the file though, right? Yes, yes.

I'm looking at this here. It's in the same kernel. So can I do this or do I have to do the full. So I have these two WL files. This is going to be the.

Right like. So here's the example we were looking at earlier with, you know, the. Was asked about what makes a good packlet, the one that might get featured on the homepage or maybe like in an email or a blog or something. Mostly the important things are spelled out in a couple of documents linked from the website. This creating packlets one which is kind of more teaching and then a guidelines. Where's the guidelines linked. Where are the guidelines? There's a got to be a link to it here, right? Here it is guidelines. All right so we have this PACLIT repository slash guidelines. So this really tells you all of the conventions and guidelines and rules that we want you to do. A lot of these are not required or mandatory but you know these, these are sort of the the general rules you should try to follow especially and it's a lot of them are about like how to write good documentation is a big part of it. But see oh here's the answer to somebody asked about auto loading so or auto updating. I mean auto updating is is prohibited but yeah so this you know you have to choose a license, you have to write good documentation and you know the styles of the documentation should try to conform to Sir Wolfram's standards. So yeah that that's it. All right I think we're at time.