Documentation Tools/Authoring Documentation for Paclets

So, hi, my name is Brian Van Vertloo. I work for Wolfram Research. So I work a lot with our documentation systems, both with authoring and with building that documentation. So, but today I'm going to talk about authoring. I want to save building documentation programmatically for Connor tomorrow, so make sure you go see his talks. What I'm going to cover here, I want to cover a little bit of context and background for documentation and Wolfram language and for packlets. And then I'm hoping the meat of the time here I can spend on actually writing some documentation with you. Okay, so of course you've got pack, you've got some packlet code. What do you do now? You want to tell everyone how to use it? I'm going to talk about these three different kinds of documentation pages here in a minute. So for a brief history of packlet documentation for you, some of you, I'll say veterans, I was going to say old timers, but I thought that might be a little rude for some of you veterans who've been around here making packlets for. While it used to be we used to have this, this kind of dinosaur that lived inside of Wolfram Workbench that we also called documentation tools. It's very, very Spartan in its, in its presentation and it, you know, it worked, but you needed, you know, you needed the workbench to work alongside it to, you know, construct your, construct your documentation pages and construct the packlet. But now we have some documentation tools. We've reskinned it, we've restyled it, we've trimmed down a lot of the cruft, and we built it right into Mathematica. So not just the tool, not just the authoring tools, but also the, also the build code as well. So, ok, let's talk about some types of documentation pages. The probably the most important kind of documentation that you will be writing is your reference pages. They're also called symbol pages. They're called function pages. Here on the left we have what, what it will look like when you're writing it. And then this is just for context, what one might look like after you've built it. What these pages do is they will. So you've written a function. These will define how you use the function, what, what is its usage? It will talk about any options it has, what maybe those options, default values are. Most importantly, in some cases, you will have many, many examples, hopefully definitely include examples. Tell people how to use this. A lot of people skip past reference material. I've, in my experience A lot of people skip past the reference material and go straight to the examples. They want to see the function in action. If you don't include examples, people won't have that. These reference pages are very heavily structured. You know, you're intended to put certain pieces of information in certain places and the template that we'll get to in a little bit has, you know, has a place for everything and everything in its place. And if you want things to look right once they're built then you know, you gotta, gotta play by our rules in that one. Okay, the next type of documentation page. And if you were here for Bob's talk and I'm assuming you were, you know, he talked a couple, a little bit about these pages. We got guide pages. Guide pages are kind of the parent level of reference symbol pages, right? We've got, you know, where you would write a guide page to collect symbol pages into a coherent group of functionality. Maybe you have, in this case I'm showing here, one for importing and exporting. Right. Of course the built in system has you know, hundreds and hundreds of guide pages. Maybe your packlet will have one, maybe your packlet will have 10, maybe your packlet will have none. It's usually a good idea to have at least one, but there's nothing technically standing in the way of having zero, if that makes sense for your packlet. The third type of documentation page we're going to talk about are tech notes. If you've been around the Wolfram systems for a while, you might recognize these used to be called tutorials. I'm not going to really get into the details of why those were renamed but suffice to say these are called tech notes. Now these are for, you know, the, for the kind of narrative, you know, bookish level descriptions of how you might be using the functions in your packlet. You might have more in depth multi step examples in here. You might talk about the, you know, how, how or why you implemented certain things in certain ways. You know there are some, some very, very book type documents that are written as technos in, in the system. This is, it's very free flowing. You're kind of given, given the reins and you can write what, you can write what you want. So. Ok, now that we're through some background let's actually do some writing. I'm gonna switch to my other screen here. So for the purposes of our, or for this offering here, I've written an example packlet. It's very small, it has whole three symbols in it. Say hello is back for and encore. And then I've got Add two and add three are two more functions. And I just, you know, I have them here, I have them defined. Very very simple package. Very very simple package file. You'll notice here that I did include usage messages and I'll get to this in a second. It's if you have a usage message for your symbols in your package file, it will make something that I'm going to show you here a little bit easier. So let's go ahead. We can go up here and open Documentation Tools Palettes menu. And here we have it. Ok, now this Documentation Tools as this is what it will look like when you've opened it for the first time and without the help of the packlet repository submission notebook that we saw a bit before. So in this case, you see this button here says Add Packlet Documentation Tools doesn't automatically find packlets. It doesn't assume that you're going to be writing documentation for all the packlets that you have on your system. So you have to add them manually once you've added or once you have a packlet here you want to add. This could be an empty directory if you want and you can write, you could write documentation without there being any code. But in this case I'm going to use this WTC example packlet folder that I showed you the code form for. And I get this little window that pops up that just confirms this is the packlet's name, this is the packlet's directory, and this is the context that it automatically found from the packlet info file in that folder. So we click OK and then we see it shows up there. All right, so now any documentation that we write will be, will be placed automatically. It will automatically be saved in this WTC example packlet folder. So just a quick tour of some of the top level bits of documentation tools. We have this right here, this tab that's split up into three tabs. This is the F tab for function, this orange one here is the G tab for guide. And this T1 is the T tab for is the tab for Tech Notes. And sensibly the tools to build each kind of page are stored in each of their tabs there. So let's just go ahead and start with a new function page. And it's. Since I'm on my second screen, it's putting everything, all the dialogues on the first screen there. I'm not sure how well this is showing up, but we have, you know, new function page. It does tell you the packlet that you're putting here. I'm not sure if that's, if that's visible, but it's, it does say WTC example packlet to remind you that that is the packlet that you're using. So let's say we want. Okay, Brian. Okay, great. Okay, great, great, great. Thank you. So we're going to just make our function name say hello, let's, let's talk about this for a second. So ignore is something that you might do if you just wanted to, if you just wanted to open a blank template and you didn't want to save a file anywhere, you just wanted to say, you just wanted to just look at the template and see what it looked like. This, if this ignore would. You could leave this name. You could leave the name blank and click ignore. And then it wouldn't automatically save anything anywhere and it would just open up a blank unfilled out template. But for now I've typed in a function name. I'm gonna click ok and it again opened on the other screen and you'll see it open to the bottom and I'll talk about why in a second. But you can see that it did save it as say hello. It filled out the title, it filled out, you know, filled out the, not only the template here with the, with the arguments, but you'll see that it also even pulled in the usage message automatically for you. So usage, usage messages are a good thing to have anyway. But you know, putting them in your code before you start writing documentation saves you even this extra step which is, which is super handy. You'll see up here it does tell you which packlet you're dealing with. And then, so okay, let's, let's walk through a bit of the template before we start typing anything. Before we do this, I find it's useful to have a side by side page to see where, where things will end up once they're, once they're built. So this brown box you can, it's pretty obvious that that becomes this. These blue usage messages here, everything that is in here, everything you type in here maps to these details and options. So everything that goes underneath the usage box in the, in the offering template goes in here. That's tables, that's text. But by convention we don't have, by convention we don't have code in these. You know, if you want code, the examples are replaced for that. But let's close that back up. These, these are actually things that will go down in the footer. These are related. For related tech notes. These are for related links. Say if you had any. If you wanted to point to, you know, a function repository page or a data repository option or object or maybe your GitHub repository or something. It doesn't matter. But these related links are links to things either in Mathematica or on the web that don't fit in one of these other kinds of other kinds of links. See alsos are for related functions. So these are placeholders that will, you know, one click placeholder so you don't have to select the X's manually and you can say, you know, if you wanted to link to plot and if you want to add another one, you can just click the little plus button, click as many times as you want and just add more however you want to do that. Related guides, of course, is for related guide pages. Typically you would have at least the guide page that this function was listed on, if any listed here. But maybe there's more than one related guide. You can list them here. Separate cells Examples Initialization this is, this is a new. This is a new thing to this version to version 13.2. So in previous iterations of packlet documentation authoring, you would need to have a needs. You have to have one of these needs calls or any other initialization in each example group. And the reason for that is that examples by cell group are split are sandboxed to themselves. They're islands unto themselves. So if you for example, use the same, you know, if you use X in 30 different examples in your notebook, you won't have that the. Any. Any definitions to X polluting any other examples in your notebook. But the, you know, the, the. The drawback of having these isolated examples is that you would need the. You need this need statement because when things aren't loaded on your context path, then you know, all of a sudden the symbols that you're trying to use your packlet won't show up. Now I'm what I'm not sure about and I need to test is whether the the new needs syntax that Todd was talking about would would obviate the need for this. But I suppose then you would still need to fully qualify your pack or your functions, which may or may not be convenient for you. Now the way this works once, once your page is built is that this needs call will show up. But this. The. You know, the. Any code that you put in here will also be baked into every input. So you need to make sure that whatever you put here is not computationally expensive. If you're finding the first you know, 500,000 prime numbers in this examples initialization. Guess what? Every time you evaluate any example in your page, it's going to try and find the first 500,000 crimes. So it's just be wary that the examples initialization. With great power comes great responsibility. Right? And it's up to the author here to make sure that they're not putting in something crazy here that's going to completely bog down. Every example in the notebook needs calls are always very cheap though, at least after the first call. So that's, that's. So this is automatically filled in based on the packlet or based on the context that was defined for the packlet. And so then basic examples is basic examples. It gets its own section. Close that up and then the rest of the examples of course map one to one their scope generalizations and extensions options is, you know, has a sub section for each option and that, that makes sense. So let's talk about metadata here for just a quick second. So metadata is of course the data about the page that shows where you know what version it maybe was included in or modified in. And categorization is metadata that talks about what kind of page it is, you know, what packlet it belongs to and its uri. If you were, you know, once you've installed a packlet with documentation, you can in fact find its documentation in the help center, but not if it doesn't have a URI assigned to it. And then you also have, you can also provide keywords. These are search keywords because building your packlet will also include a search index where. So if you're searching for, you know, if I, if I searched for hello, this, this page may come up in, in the search results there keywords are typically one per line and then they're typically not capitalized except via or except for proper nouns. But you know, those are conventions. The search system tries to do the right thing based on whatever you put in there. And it will index things. Exactly. And also search things with stemming and other things in place. So it, as long as you get the right idea, your keywords listed here should, should be. Should work fine. Syntax templates. These are typically very advanced capabilities. These. I was considering maybe removing these as they might have been confusing. But I think they're, I think they're useful for a, for a built version of this page. I'm not sure if the, the packlet build tools do anything with this yet, but we're, we're hoping to have something in there eventually where you can provide and this is largely to affect the syntax coloring of your symbols. So when you, you know, if the, you get the coloring of missing, missing arguments or if you, you know the, if you have extra arguments or you know, even the, you know, the, the lighter blue for, for localized, localized variables and things like that. That, that's all defined in the syntax templates area. So right now the, the packlet tools won't do anything with that. But that's not forever. So you can fill these in and I'm not going to go into the syntax for what goes here. That should all be findable in the documentation for these pages, which I should add is findable at the bottom of every tab via the help. The help button will open up a documentation page that links to several other workflows and has a step by step for every, for every, for every button here. Okay, so we have talked about that. So let's say. So now we want to talk about. Well, let's actually write some of this, let's actually write some of this documentation here. So what, what do we want to say about say hello? Well, we should probably say what, what is name? Let's say name must be a string. I don't know if that's true or not. I think it is. Yeah. Ok. It is. So we see that. Oh ok. So you know, Bob mentioned earlier about template input. Template input. It's great. It's a great thing. Helps you format everything, links things for you. We see that but when we're, when we're talking about this we see that name is the variable. So we say template input and then does pop this up. It, it gives the option here to, if you want to link to a symbol named name. There is no symbol named name. But Documentation Tools is, you know, maybe you're writing this piece of documentation before you have written that page. So it offers to make the link for you if you want. But in this case since it couldn't find one, it defaults to just format the link and you hit okay and you see that it looks and it's nice and formatted like that and you. So maybe, maybe say hello has some options. It doesn't but you can see for the tools here, Details and Options has its own section. You can open that up and we can see some of the, some of the tools that will help you fill out this section of the documentation page of most note, here are these tables where you see the little graphical, little visualization here of a one column table, a two column table. You see a little break there and a three column table. Unless if we quick click that, we get the three column table and each. We could fill out each object here however we wanted. If we wanted to add a row, we just put the cursor in the table, click Add row and we added a row. And it will add the row after wherever you've put it. So, oops, now whatever that works. So we click Add row. It puts the row after wherever it was selected. So if you just want it one at the end, of course you can just put it at the end. Sorting. Your table will sort by this, you know, the string value of the first. The first row or first, sorry, the first column. Merging tables. If you had multiple tables that have the same number of arguments, you could, you know, select two tables. You click Merge table and it automatically merges them for you. And I want to talk about Options table here. So Options table, which, and I wish I had written my function here with, with an option in it. What Options Table does? Options Tables option What, Sorry, what Options table does, is it will look at the code and see are there any options for this function? And if there are, it will let you select. It will let you select from the list of options which options you want to show up in the table, and it will fill in the default values for you, and then it will leave a space for you to describe what the option actually is. Unfortunately, I don't have an option. I don't have a symbol here with an option, so. But that's. You can look at the documentation for documentation tools and see that in progress or in situ. So I'm going to briefly talk about the examples. There's not a lot, there's not a ton here because examples are generally just captions and code. But, you know, so that we simply have buttons for captions and code. The text is for the caption. This is a caption. And by convention, we always end our captions with colons. That's just our convention. They used to be periods back in the old Mathematica book. They are colons now. So that's what we do. But of course, if you type without pressing any buttons, then you get. Then you get code. So there is no button for insert code because you just type to get code. Delimiters are important not just for structure, not just for visually distinguishing examples from each other. But you'll see that delimiters also group. And what that means is, if you recall me talking before about examples being isolated from each other by the cell group, a delimiter is what we use to isolate examples from each Other. Okay, so links, links, make links. For the sake of time. I'm just going to skip links formatting. There are some. Sometimes you might want to have special formatting in your pages. Like literal input will make just code form without linking anything. Code in line will make a computer voice without. Without an. Without an inline cell, an italic input and things like that. That's kind of makes sense. Let me close some of these up. Let's talk about Futurize for just a second. The old documentation tools had futurize as well. But let's so futurize. What futurize does is maybe you have a section of your page or you have a whole page so that you don't want to actually build when you have your packlet. Maybe it's a page for functionality that doesn't exist or maybe it's a piece of your page that you don't want to be included in the final section. So we have. See we have selection and whole page. I'm going to select this table. If I click selection I see you see it turns this nice totally non alarming pink color and tells me who futurized it. That's me, Brian V. And when I did, you know when I futurized it. And then this cell will no longer be included in the build. In the built page if we did whole page then we get this banner at the top that says future. Now the whole page will be ignored when it comes to build time. The last thing I want to talk about with function pages is something under Tools and this is a nice little shortcut for when you have a number of function pages you want to make for your packlet under Tools is generate function pages and if we click that then we get on my other screen again then we see the packlet name and now we have some options. This will make. It will do the same thing that it did for new function page where it generates the. Generates the page with usage and the metadata filled in. But it will do it for every symbol in your package. So if you wanted to build them all you can if you've built some of them, if you've made some of the pages yourself and only want to build the missing ones like we have here. Like I wrote the say hello page. I don't think I saved it, but I did write it. Then we could say build only missing or if you click selected then you then pre populates this input field with all of your functions and you can just common delimited list of functions you can add or delete Whatever you want from the list. For now, I'm going to say build only missing. And then you can. It tells you where it's going to put them and whether you want to leave the pages open when you're finished with them. So after I click OK, then over on my other screen it made add 2 and add 3 and then you can mark these up however you want. It's a nice shortcut when you have a bunch of pages to make and don't want to have to type a bunch of different function names over and over and over again. So, okay, let's talk about guide pages quickly running a little short of time. I'll try and make it fast here. So we got a new guide page. Let's talk about, you know, let's just call it. These are my, my functions. That was too big, so my functions. And it saved the page where it should go. Let's for reference, go ahead and jump over here. This one's a little more straightforward since it's just a title, an abstract and a list of functions, but still useful to have it open. Now you'll see on guide pages we have a toolbar here too. The toolbar is. There's. It's nothing here that you couldn't get from the palette, but maybe you don't want to move across the screen every time you want to use something from here. Eventually there will also be a toolbar in the function page one, but there are just so many useful things in the function pages list. I haven't quite yet decided on the list of things that I want to put in the toolbar. And if you have to click three or four times to get through everything in the toolbar, then what's the point? Right? So, but in this case, these are the most important things that you would need for, for a guide page. And so that was, that was pretty much a slam dunk for what should go in the toolbar here. But anyway, we have my functions and we can type in, you know, this is my list of functions. And here, as Bob alluded to this a little earlier, we have what are called one line function listings. These are for typically your more important. Your more important functions that need their own, that would want their own description. So we're gonna put say hello here. We're gonna say say hello to someone. But I don't think add two and add three are so important. So what I'm gonna do is, what I'm gonna put in is what's called an inline function listing. So it's functions Inline and you'll see that it's functions inline listing, it's the same, does the same thing. So we click that and then we get this cell here. So we can say I want add two to go here and then I click another one and let's say add three. And you'll see that it's just that to get rid of noise, you'll see that each one of these is. Doesn't get their own description. So you know, it's, it's similar to here where you've got more important functions and then you've got the list of kind of tangentially or less important functions listed just side by side. And that's how that goes. Just like in ref pages, we have cross linking. If you have related tech notes, if you have related guides and related links, the metadata section is a lot smaller here we don't have to worry about syntax coloring, we don't have to worry about things like that. So we have less, we have fewer options here. But we still need to worry about search terms and we still have to worry about uris if you want your page to be findable via search. So let's quickly move on to tech notes. And it put on screen. So more about my functions. Ok, so the first thing you might notice here is the color mismatch or the other ones were nice and uniform. I don't have a good excuse for that. It needs to be fixed. I will say tutorials used to be this brownish color and then tech notes got redesigned into being this purple color. And so that's, that's probably the main reason. I don't have a better reason than that. But very similarly, we got, you know, this is the base template here. It's got a few different things, you know, it's got a few different styles of cell filled in here just to kind of show you at a glance what's. What's possible. But when we're talking about mapping this one to one, it's important to note that this, these set of jump links here that are in this list, this is automatically generated during the build based off of these section cells. See importing and exporting data. If we scroll down, we'll see importing and exporting files. And so this, this is created by the build. There's nowhere to spell this out in the authoring notebook. You simply put your sections in the notebook as you're writing it and these will, these will generate there. So I'm just going to talk about inserting a few things into tech note templates and then. And then we'll move on in. In this toolbar, we've got a series of dropdowns. So if you want to insert, you got section subsection, text, example group, and example example caption, which maps directly to these five buttons here. So again, nothing in the definition or nothing in the toolbar that's not available in the palette. It's just a different way of accessing it that might be easier for some people. If you don't want to use it, you're not missing a thing, but all you do is just pop it open, click a section, and then you can type the name of it. Easy. Dead simple. The other thing to talk about here are definition boxes. This is an example of a definition box in the offering side. This is an example of a definition box in the built side. So what a definition box is, is a lot of times how these tech notes are laid out is you list your. The functions that you're going to talk about in a definition box and generally what they do, and then you go on to talk about them. Maybe there's more than one definition box per table or, I'm sorry, per section. And you can see that this particular section has, you know, has multiple definition boxes with slightly different content in them. There's no limit to how many definition boxes you have, but. Oops, accidentally clicked when. I didn't mean to in that. Why does it keep doing that? Okay, there we go. Under definition boxes, you have the choice of two column, three column. Now add, row, and merge, of course are not. They're not inserting anything, but if you have more than one. If you had more than one definition box selected, you could of course, merge them. Thank you, everybody.