Wolfram Repository Updates: Paclets, Functions, Examples, Prompts, Tools and More

Hello, welcome. My name is Bradley Ashby and I'm going to do a quick talk here on updates with the Wolfram Repositories and Resource System. Let's see, quick place to start if folks aren't too familiar with the resource system itself. The Resource System is the framework that we have created to maintain our collection of repositories you might be familiar with, for example the function or the packlet repository. We've also got a number of others, some new ones. We've seen a number of talks on individual repositories in the tech conference so far this year, so I've listed the names of the talks and the presenters. So if you want more detail on those repositories, you're going to get a lot more good information from those dedicated talks. Today I'm kind of going to do a little bit of an overview of the resource system, some developments or new repositories since last year, and some things we can look forward to in the future. So new repositories, new features since last year. If you were at the tech conference last year, you probably saw a talk or six on packlets and the packlet repository that has been released. You can find the Resource System repository homepage here. Resources.wolframcloud.com and that will also give you links to all the other repositories we have. The packlet repository is a is a good place to collect things that are bigger than functions. If you've used the function repository that's real good for one off functions that stand alone. Packlets are collections that are bigger than that interrelated functions, perhaps even applications or things that could benefit from style sheets or pallets to be included. Bigger collections. We saw the packlet repository as a necessary component for having a vibrant community that's intercommunicating and having a lot of participation and cooperation and development so that it's easier for one person to build on top of someone else's good ideas. Currently we're at Today I think 133 packlets in the repository already and that pace of new submissions, new packlets, new versions of packlets has been increasing in pace, so it's definitely worth checking it out. Getting involved Content in the pack repository is not reviewed before it's published. So so even if your work is maybe a little rougher than you want, you could release it as a pre version 1. Get it out there, get some comments, get some feedback so that you can really start iterating and contributing. Just getting it out there is a Big motivator. I have my own packlet in here that getting feedback through the packlet repository was a real good motivator to to brush it up and add some features. The prompt repository has been released earlier this year. Very focused on LLM prompts. Again there's been a talk about this already this conference, so I really suggest checking out the recording of Eli's presentation here. This is very focused on working with various LLMs that we are currently building connections to and it's a good way and a good place to start with using prompts or finding ideas for writing them. Bob Sandheinrich did a talk on LLM tool and mentioned the repository for that. Again, worth checking that out for more detail. I have this listed as almost released rather than released. It's definitely in an under construction state but is growing and getting attention. See One of the other new features that we've added to the resource system in general since last year is the this presence of star ratings for resources. So in the packlet repository we had a little button here where you can give something, you know, a like or a thumbs up or as we say here, rate it as good. And that was pretty popular and had been asked about for other repositories as well. So for most but not all repositories in the resource system you are going to start seeing seeing these appear and there we go, little stars under the name under the contributor or off to the side where you can say I found this useful, I'm going to give it a little bit of recognition and that'll help people find good content and encourage people to write more good content in the future. So we're pretty happy to bring that to all of the repositories. Almost all of the repositories. Let's see. One of the other significant developments since last tech conference has been the rollout of a more comprehensive publisher portal. So if you have a Publisher ID and are participating in the resource system from our homepage here you can find your publisher IDs and this will show most people probably aren't going to have one. Obviously I've been on and off a number of them internally through the development process. So if you have multiple it'll list it here. It'll also give you a rundown of what a publisher ID is and so the same page can be used to request a publisher ID if you still need one to start submitting content. But the portal itself, let's see, we have added an easy way to view your pending submissions if you've Submitted something and it's still being reviewed, maybe hasn't come back yet, or maybe it was sent back and you lost the email. You can find everything listed here. I don't have any personally, so I pulled up the one for Wolfram Publisher ID in general. Obviously we have quite a few. For example, I know Eli submitted this one for the prompt repository just the other day in relation to the dicconference. So you can go in here, see what you have submitted that hasn't been published yet and there's a quick status on whether it's awaiting review if no one has looked at it yet, or awaiting updates if someone has looked at it and sent it back with comments and suggestions. So if something is awaiting updates, that means it's waiting on you. Review means it's waiting on us. So a good place to reference that. There's also published resources if you want to take a look at things that your publisher ID has already published. That's all listed in one place. So you don't have to keep your own your own notebook of links just in order to find your own stuff. If you're ever going to do an update to a resource, it's good to start here. Right now it'll link you to the page. In the future we're going to start adding more utility to this where you can get your definition notebook and even check for status of how many times have your resources been used. This is really the first expression of the publisher portal and we have a number of things we want to add to this over time. Similar to this page, this publisher portal page that is full for the publisher, we've also added a public facing publisher page. We've had questions and requests for this as well to find other content by the same person. So if you put out good work and people really enjoy it down here, underneath the examples, there's a link to the publisher and it'll bring you to this page which is quite, quite similar in that it lists all of the content that that publisher ID has posted. Except this is this is for people who aren't the publisher to use. It won't have any of that data or that internal information, but it's a good way if you want to share one link to say here's all the work I've done in the resource system. You can take a look at it here. Or if you want to find out other good content from someone you found to have good trustworthy submissions, Let's see some other notable achievements. Just to toot our own horn, since the last tech conference, as I mentioned, Packlet repository is up over 133 packlets right now and that pace is increasing. So it's a good place to look for growing content, keep an eye on that and keep a lookout for good and interesting packlets, even if you don't find something you want to use immediately. If you're interested in packlet development, the packlet repository has a number of packlets that link to their source. So if you're wanting to get into developing your own content, it's a good place to go to find, find hints, find examples of how they are laid out. I still visit the pack lit repository and some of those, some of those source content links. When I'm trying to do something new with my own pack that I haven't done before, I look for existing content that does, does the same thing and see a good way to achieve that goal and kind of copy those patterns. The function repository which people are probably most familiar with in the resource system, let's see, is now up over 2,800 functions since our release, which is just fantastic. We're still seeing good content come in. I will put out a call that now is a really good time for submissions. We have a number of weekly review meetings for submitted content and we have gotten to the point where we haven't taken up every single one of those hours every week. So if you have a submission for a function, now is a good time to get that in. It'll probably get some attention within the week. And if you have any, if you have any functions again, like I do, where you sent it in, and there has been some suggestions for improvements sent back to you. I've got a couple of those where I said, okay, yes, that's a good idea, but I don't have time to do that right now. Now's a good time to pick those up, brush them off and send them back in for review after making any adjustments, because they'll be getting good attention pretty quickly. Another achievement that I'm pretty happy about is the tech conference this year. We've had at least eight talks that I counted just by looking through the summaries, not from the Resource System team that were based on content that you can find in our repositories, in the packlet function or neural net repositories. And that to me is a really good sign that the content that we are encouraging, that we are enabling with our system is, is really useful to people. Useful enough for them to not only make it, not only submit it but then to do a talk around that content and that, that to me is, has always been the goal of what we're, what we're doing here. So I'm pretty proud of that achievement personally. So let's see. I think the, the next thing up here is probably the newest information in my, in my talk. So we've talked about stuff since last tech conference and the new news is the imminent release or announcement of our example repository. So you're all kind of getting a little bit of a sneak peek here at the tech conference of our next repository. Kind of obvious on the face, the Wolfram language example repository is going to be a growing collection of examples using Wolfram language code. I bet you guessed that the examples you'll find in here are longer than what you'll find in the Wolfram language documentation, which is usually one, maybe two lines long. So the idea here is that it's more of, it has more of a through line. You've got multiple steps for achieving a specific goal and so it'll help walk through solving something somewhat specific. However, I've highlighted a few examples of examples that I think really illustrate some of the value that the example repository will be adding. So here Generate the isomers of decane is very chemistry related. Perhaps you could tell. It not only shows off some of the, you know, functions that we've added for atoms and molecules, but it goes a little further and shows you establishing patterns, making functions for replacement. And then how can you can apply that to multiple molecules, how you can access the Wolfram knowledge base and the entity stores and even down to the point where you're making an image for some output and now you can process those entities into something that you would generate as a report or a demonstration. So it really colors multiple areas of functionality that we can provide and give some insight into how you can interact with those areas and put them together. We also have Explore symmetries of crystal systems as an example. This is another good one that covers entity framework interactions which I know I need to reference pretty frequently. I don't use them often, so anytime I come across entities I'm looking for good examples to help me remember some of the nuances or the easy ways to achieve what I'm looking for. So this is a good example, starts with that and then goes into some more complicated input output here. But this is also a case where we're starting to touch on displaying information, putting it into table table and it can give you some good ideas of, you know, I can use common name for Example to pull out headers to create a table. And this is a good example of processing the data so that you can use it in different ways. And again, this example has multiple parts to it, not just the entities, not just tables, but a number of steps here that can really help you put some things together, even if you're not specifically working with crystal systems. And along the same line, we have generated timeline for Rembrandt's paintings, again working with entities. And then here we've got dates and timeline plot and kind of the same thing, how to pull out some data, get some valuable information out of entity Framework and ways that you might want to massage that data so that you can use it in different presentation or visualization formats. So each one of these really illustrates that more of a full story. And I think in our documentation it's pretty good at showing working with one or two of the functions. But this really, these examples are going to help tie multiple systems together. So not the demonstrations. The examples I've showed you are perhaps if you've followed what we've been doing for a while, you may have noticed these examples are very similar to the marketing, as we call them, the marketing examples. In our new in version examples, so new in version 1312, we often have pages that highlight the new functionality and the new capabilities. So that was the origin, one of the first use cases we knew the example repository would be good for. And so the content that we put in first and that you'll find right now very much is focusing on this first bullet point of what is the example repository good for. It's good for learning about the language and its capabilities, how you can put different things together. Also, like our demonstrations project, which you might be familiar with, examples can be used as good references if you have a good example of using data or demonstrating how a system works, you can link to content in the repository, in a publication, in other websites, in classrooms. And then like many things, the example repository can be a good source of code to find and repurpose in your own project, which as I said, will be how I'll probably be using this a lot. Anytime I need to work with entities, I'll come in here and search for entity and it will remind me of some of the, some of the tips and tricks that really help. So the content we might be looking for in the example repository for you all to submit will probably be more than one line. One line of code is pretty short and may even could be a function in the function repository. So we're looking for something that's a little bit more of a story, a little bit more of a through line of combining a few things 2 to 15, including some explanation text between most of the steps here that illustrates a single process that's still something you can summarize with a one line title Other good content is after the conference I'm probably going to be contacting a lot of the presenters because just watching some presentations I've seen an awful lot of good content and in other talks. One thing I think this could be very useful for is discussions on community or other forums I've seen a number of times someone asks a question and maybe a couple of months or years later someone asks a similar question. Something that comes up fairly often. And rather than having one long post with a bunch of steps, you could put your answer to one of those questions as an example in the example repository and refer to that as a good way to answer a common question that our community has identified as something that comes up fairly often and that will be a nice canonical way of the same walkthrough downloadable Example notebook that will be really useful. And then once we do polish up and release the example repository, there will also be a set of guidelines available from the website and from the Definition Notebook that you can use to kind of help guide your content. Speaking of Definition notebook, I know you'll be able to find that in the file new menu repository item in the next version, version 14. I'm using 13.3 so it's not showing up at the moment. See, you can look for the example repository to come out within weeks. So things to look forward to from the Resource System Example repository Mere weeks away from an announcement, it's already public. You can navigate there and check out the content, but we're probably going to do at least one release that will tidy up some of the presentation of the website and its usability before we do an official announcement blog for it. Another couple of projects that are in quotes soon will be the Demonstrations project. We are going to be migrating the Demonstrations project fully into the Resource System. If you are an author with demonstrations and have submitted content there, we will be in contact with you if there's anything you need to do with the conversion. For the most part the thing that's going to happen is the submission and review process in the back end of Demonstrations is going to use the Publisher ID system and the other systems that we have built and refined for the other repositories. So we're going to be adding some consistency to that experience. There will probably have to be some changes, but we're going to try and keep that minimal and make it as smooth a transition and if you need to do anything in particular, we will contact you and let you know and make that transition as easy as possible. We're also working on a color palette repository that will expand the capabilities of color data and color data function. This one's pretty exciting and easy. It was one that we identified as an easy repository when we started it and then we very quickly found that the UX and workflow for the website was much more complicated. So I want to take a second and give a lot of credit to our UX team and to our web developer for what they're going to be accomplishing with this site. It's going to be. It's going to be more involved and hopefully still easy to navigate compared to the other repositories. It's going to be exciting. Less than soon is about all we can promise on these other ones. They haven't been scheduled. We have had discussions on a style sheet repository to make it easier to create, use, share style sheets. So if you use a style sheet and a notebook and email it to someone, if you've done that before, you may often get an email back that says hey, I can't see this because I don't have the style sheet. And then you have to work through that. This will make that much easier to have consistency in sharing content and then not unrelated a book like repository for actually publishing books with written in notebooks in a consistent manner. These have both been started and discussed, but as we saw earlier this year, LLM showed up and we hadn't even considered a prompt or LLM tool repository and then they took up a lot of our development time and and priority. So there's always. It's always difficult to promise something that hasn't been started because there's a lot going on right now, but those are on our minds. I am basically at five minutes till that is rushing through all of the content I've got for today. So if you've got any questions, now is a good time. I see we do have one already from Dave. Since different LLMs may perform differently for the same prompt, would it make sense for the promotory to specify one or more LLMs that have been tested for each prompt? Yes, that is a great idea. I know that has been discussed. I'm not sure if that has been implemented. Eli would be a good person to answer that question. I don't know if he's in the talk today, and he may have covered that in his talk, but I know the difference between LLMs is one of those properties that we've had in mind and we often have. We don't have anything at this point, but many of the resources have compatibility information listed at the bottom and so the compatibility or confirmed ability to work with the different LLM I think is going to be a really good feature. I'll make sure that's noted down if it's not already in the works. And yes, anton, LLM excuses LLMs come in like a wrecking ball and have changed a lot of plans this year, so hopefully that's something that we can get back to. I very I'm personally very excited about the possibility for a book repository. I've seen a lot of good content written in notebooks and making that even easier to share and distribute is to me seems like a pretty obvious next step. All right, let's see. Rickenigan answered a bit of your question there, Dave. There is a link. Let me put it on the right chat here on the right screen. Prompt assessment for testing on different LLMs. What is my favorite resource and why is it Birdsay? Well, I have published a few of my own, Rick, so you know there's some competition. I do have my own Packlet for Notebook Workspaces which does auto saving and helps me manage the far too many notebooks that I have open on any given day. It makes it easier to switch between them, so I'm kind of biased towards that. But Birdsay is obviously up on the list as our long running mascot. No one can deny the appeal of Birdsey. All right, I'm not seeing anything else in the Q and A, so now's a good time to drop something in there. I'll give it a couple of minutes, but otherwise that's my content for today. Thanks for joining me and I look forward to giving even more exciting updates next year. Thank you.