State of the Wolfram Cloud

Thanks everybody for joining us. My name is John Pacey. I'm the director of Cloud development here at Wolfram Research and today I'm going to give an update on the state of the Wolfram Cloud. So thanks everybody for joining. And before I start, I want to make sure to at least mention our office hour that we're going to have tomorrow on the wolfram Cloud at 10:30am Central tomorrow for about an hour. Be myself and a lot of my team leads, Matt, Joel and Philip. So please attend that and ask some questions and reach out to us when you can. So I want to make sure I at least mention that before I start here. Okay, so I'll be switching a little bit between my slideshow here and demonstrating some things in my web browser. So bear with me a little bit on that and let me know if things are not transitioning quickly enough or am I getting ahead of ahead of my screen. So thank you on that. So I'll just do a brief review or introduction Wolfram Cloud capabilities for even familiar people or people who aren't as familiar with that. I'll give a little bit of an explanation of some key features or in this case key differences of the Wolfram Cloud environment to try and clarify some things that I think are a bit confusing. Sometimes some recent highlights of things we've been working on or we've accomplished in the past year or so, give you an update on our current and future roadmap and priorities and have a little bit of time for questions. So the Wolfram Cloud, what is it? Well, it's a way to work with notebooks and write Wolfram language code in a web browser without any downloads. It's a way to share interactive notebooks and also other arbitrary content via our Cloud object system and HTTP requests On the web you can deploy applications, particularly applications written in Wolfram language. You can also deploy other kind of hybrid or applications that rely on Wolfram language or Wolfram language driven or more typical web and JavaScript script and HTML based as well. And I think it provides a powerful hybrid sort of desktop plus cloud workflow. So the Wolfram Cloud is not a replacement for desktop. It's an addition to our suite of Things and like I said, brings this powerful hybrid desktop and cloud workflow that we've used to build lots of various things which we'll sort of talk about. So working with cloud notebooks. So let me switch out of my presentation view here and switch over to my web browser and think things are going through. All right, so here I have my Cloud notebook. So I've already signed in with my account. Actually our little landing page here looks pretty similar to what the desktop experience is in terms of creating new documents. So we have a new basic or a standard notebook here and we have other notebook types experience, including Wolfram alpha mode notebooks. And our new experimental chat notebooks are also available to experiment with in cloud. So I create myself a new notebook and what you have here is a kernel running purely in the web browser, or at least the kernel is actually running over on the server, but your web browser is communicating with it. So I do it very, very simple evaluation. I can do other examples for plots. My typing is not great. And we support various things here. So we have a plot which can be resized and moved around. We also have existing manipulates supported as well. So I'll just use my same plot there and I will use my very standard example of shifting phase. And okay, so again, these are not new. These have been around for a while and we still support them and do various different things. That's a really key part of it. Another key part of what the cloud brings is the ability to deploy or publish and share content. What I will go ahead and do is deploy this manipulate correctly. What we've done there in the background, we've done most of the hard work. This automatically packages this up in a way that is basically deployed. Or you can now view this corresponding WOFO language expression via the web URL that you can hand out to people and it works. And there you go. So with these primitives we built up in the language, you can create and deploy and share content. And so in particular here I'm emphasizing sharing Wolfram language things, but you can share other HTML or sort of arbitrary text and things like that. As I mentioned before. Let me go back to my other examples is we had support for different notebook types in the cloud. As I sort of showed you on the, on the interface there, we are still supporting and improving our rendering of WolframAlpha notebooks. So here for example is a WolframAlpha notebook where we have more natural language queries that are getting inputs and we can see the results. So in this case, this one is simple, initial one, whoops, I messed up my selection there. And we can get the full WolframAlpha results. Gonna go ahead and evaluate that and give them to me and while it's thinking a little bit. So again we are continuing to support these and the newer notebook types such as as chat notebooks as well. One key question that gets brought up is, well, what's supported in cloud or not? And it's a great question. Sometimes we have good answers on that, sometimes we don't. So other than trying things out and asking us, there's things that we have built into the language itself, Wolfram language data which will go ahead and query specific properties. In this case we have cloud support status which should tell you supported or not supported. And we also have additional categories called cloud support notes which give some additional details. So here we know that plot is supported, in other words, that things will render when I try plot and cloud. But for example, run process is not supported. It says restricted in public cloud. In this case it's restricted for security reasons. And in this case the cloud support notes tells you, well, at the very least, if you want to an ABC and an enterprise private cloud, you could enable this if you disabled the sandbox. These also show up in our web documentation as you'll see little tags on the documentation page here saying restricted in public cloud. We have other closer integration of notebooks with the web via the concept of embedding notebooks and external notebooks. So again, this has been around for quite a while for us and we use this on different sites such as community. Sorry, excuse me. And so you can publish your notebook either in our public cloud or your private cloud as well, and then use our JavaScript notebook embedder to embed that in other websites. There are other examples of doing this in terms of just pointing in this case the public cloud at a publicly viewable document. So for example, there's a particular parameter here where I can view that, see if I can pop this up. And so what I'm doing here is going to public cloud and then giving it a URL of a publicly accessible notebook file at this location and it's rendering it on demand in the Wolfram Cloud. That's just again highlighting features of our notebook embedder that we've had for a little while. The other concept is I'll get a little bit more into is working with deployed content here. I will switch from doing things within cloud that I did before in my example here. I was creating things in my interactive notebook session here that's tied to me in the browser and then deploying it. And I could still view things there purely in the web. What I will do here is cloud connect from desktop. So I'm running this in my slideshow notebook here. And so I will cloud connect as myself. And just to sort of prove to you that this is doing what it does, our Process ID is this is my local kernel running this slideshow and then I will do cloud evaluate, which basically just says evaluate this Wolfram language code on the cloud that I'm connected to. So yeah, so it's a different process id and each time that I evaluate this, it's going to be a different process id and I'll sort of get into this next. But there's a little bit of a difference between the different types of kernels that we use in cloud. So for this type of cloud evaluate or deployed content, the kernel is very short lived. So in other words, this kernel is only going to live for the life of this evaluation versus if you go back to my interactive session one, that kernel was tied to me for a bit for a lot longer. And that has some consequences in terms of thinking about about how you deploy things. And again I'll get to that in my next point. Here's some examples of those types of things or what we consider deployed content. We've added things in the Wolfram language such as the API function that allow you to do convenience things to make it easy to go ahead and deploy things. What I'm going to do here is deploy this very simple API function which is factor integer of a parameter. The cloud deploy machinery just makes it really easy for me to go ahead and deploy it to this URL. Then when I visit that URL, this is basically tying that Wolfram language code to this HTTP request. What it's telling me is your API could not be evaluated because I didn't get an input parameter of X. If I add an input parameter of X equals I don't know 50 now it actually executes the function that I gave it with the parameter and returns my result. Again, these things are new. Just doing a quick review of that. There are other of these more typical web type things that we have specific things in the language for including form function. Just a more exhaustive or different examples of API functions. There lots of examples in the web documentation. I don't want to get that. Sorry Danny, I followed up increasing my zoom. That would help. Is that better? Okay, sorry about that. So again, here is a key concept that I want to get at I sort of hinted at before. Generally these things are about the Wolfram public cloud. So in general we're running this on Linux virtual machines that do not have access to gpu, which means if you try to do rasterization or other kind of GPU intensive abilities, you'll get there in that. Okay, so you want me to increase my zoom on my cloud notebook I can do that. There are per user limits on overall storage and evaluation, memory and time constraints, and we limit or disallow certain functions for security reasons. I briefly went over one before. There's a combination of different sandboxing techniques to isolate execution of user code. Note that all of those restrictions can be reconfigured or changed in custom or enterprise solution. That's just how we have things configured for our public cloud. This concept of kernel types which I talked about, there's the session. This is the long lived kernel that's assigned to your single user when you're editing the notebook. That's what you would typically expect in a desktop session where you have the same kernel configuration that you're using. It's a long lived kernel and as long as that kernel stays alive, you don't do anything like quit it. It has access to that state. That is as you expect when you're doing the editing. My long lived kernel versus the public kernel that was in an API function, a form function or a deployed manipulate. That's a short lived kernel again assigned to the single user and typically a single evaluation. What that means is understanding that just tells you about what you're going to do for your API functions. If your function relies on some kernel state, if you didn't properly define that with the right module scoping or variable scoping, it's not going to behave as you would expect. Or if you want it to make sure that you load in other content, you have to make sure that you again add that to your API function code to say, hey, go download my other publicly viewable thing to load this each time. It's basically a very short lived kernel that gets reused every time and then thrown away. One other slightly more advanced thing to talk about here is the front end kernel communication overhead in the cloud is going to be different than what you expect in desktop. So the browser is communicating with the kernel over the network call. So some things we can do purely on the client side. A lot of the other things are most evaluations is still happening over on our servers. So it's important to understand and reduce front end to kernel communication. So on your desktop session normally that latency is about zero because the front end and the kernel are right, they're talking to each other on the same machine. But for us you have to go over the network hop. So one sort of more advanced feature to do that is, if you're looking at your application on desktop, is to look at the link snooper kernel configuration and sort of identify how much communication you have for there. So it's a little bit tricky, but at least understanding those types of things in the kernel environment and cloud is kind of pretty key. All right, I think I can switch to this. As I said before, I think we have a very powerful hybrid ability to develop applications, and we have done this a lot internally and externally. One of the key things we've done in the past year is the ChatGPT plugin. LLMs and ChatGPT in particular has sort of taken the world by storm, and we've been one of the leading peoples in coordinating or working with various LLMs for bringing the power of Wolfram language to them. So our Wolfram plugin for ChatGPT was built, designed and prototyped and utilizes the cloud. So that was something that was pretty cool and exciting for us. It does that in a couple different ways. So ChatGPT itself is capable of writing and executing Wolfram language code with the Wolfram plugin, thanks to our cloud. So it enables execution of Wolfram language Wolfram language code through HTTP requests. Basically, the ChatGPT plugin will evaluate Wolfram language code on our public cloud server and take the results and work with that. So we've designed that to safely execute that arbitrary user code. It's allowed us to do rapid building, prototype and deployment of new versions, basically using our Cloud Object and cloud deploy system that we showed a little bit before in terms of iterating on Manifest and basically the underlying API functions that the LLM is calling those underlying APIs, analyze the generated code for common LLM mistakes, and in the case of errors, it responds to help the LLM or in this case ChatGPT improve its query. And there are similar APIs and other integrations with other LLMs. And that's. There's quite a few other talks at our tech conference this year regarding chat notebooks, LLMs and other things that we've had the pleasure working with those groups. So I definitely suggest you check out more of those. But that was a big thing that we utilize the cloud to not only build and prototype, but that's what it also uses under the hood for evaluating Wolfram language code. Another thing using capabilities of cloud is the Wolfram Language Playground. And this works as a way to provide a sandbox to evaluate an experiment without requiring sign in. So I don't know if people have seen this too much, but. So this has been around for a little bit. This is our Wolfram Language Playground page where it gives you the opportunity to try things on under the hood, this is a cloud notebook that is integrating with our notebook embedder and it's also integrating with our JavaScript notebook API to build this integrated web application. We have some pre populated examples where we have some nice buttons to help people run them. Then you can also still freely code in here. I can just also say graphics of red circle and go from there. And so this is all done without having to log in and is utilizing our public cloud. And again it's very easy to iterate and build upon and customize for different sort of use cases. And again we have this whole expanding suite of Wolfram applications and initiatives that are built on top of or with the cloud. Our expanding resources system. I'll just briefly cover this. We keep adding very different things to this. This is where our Wolfram demonstrations product has moved to cloud. We have all our neural net repository, the function repository, the prompt repository, the data repository and many other things. These are interesting in that for example, the Wolfram function repository is a combination of HTML pages that are curated either generated by Wolfram language and then backing Wolfram language APIs that manage content there. Again, I think there's some good talks this year to go see about the resources system and how they build things and how they're working on different things. Another big user of the cloud is Wolfram Community and they use our public cloud and the Wolfram notebook embedder to do that. And there are countless other internal applications ranging from our internal customized ERP solutions, our internal log analysis and task systems. And we have internal clouds and development instances and cross team sharing of materials and contents on our own Wolfram servers for, like I said, internal clouds for different teams and other things like that. So lots of things are being built on top of cloud now to our roadmap updates and priorities. One of the things that we've accomplished this year is an improved release cycle overall. Our historic average in the last few years or so had been a public cloud release cadence of at least two months, usually a bit longer. I think we've reduced that to approximately about a month and been pretty stable with that in the past few months. So that's been good. And again, as part of cleaning up our improving our release cycle, kind of keying in on more frequent and timely enterprise private cloud releases, which sort of go hand in hand with some of our public cloud. We had not done a great job in the past about keeping up to that, but we've certainly made strides to improve that and continue to continue to improve that for cloud Notebook support. Just a few highlights in terms of our current priorities. Obviously we are kind of always trying to maintain feature parity with the desktop front end. We're a few years behind in development on that. They've had a pretty big head start on us, but we continue to work with them and we'll always continue to try to strive to be featured comparable to the desktop as much as we can. And that's just a general ongoing thing. Some key things to highlight there is prioritizing manipulate performance improvements, particularly in the deployed object view. So hopefully look in the next few releases to see some significant improvements there. Our ongoing continuing project we've talked about previously, which is native graphics, which is a way to reduce that server client overhead I briefly talked about before, one big overhead that is for generating graphics. We've had a long going initiative to move more rendering of graphics purely onto the client side and I think we might be close to actually finally declaring victory to have plots or majority of plots there again in the next few releases. And then we've also started on or made first steps onto another long term project we've talked about for a long time, which is our general editor overhaul for improved performance and capabilities in our cloud editor. So again these are sort of bigger ongoing projects, but these are our clear priorities for that cloud objects and kernels. So this is generally talking about the cloud object system and how performant it is in terms of deploying and managing content and then how stable or performant or fast our kernels are there. So again we've just really focused in on performance improvements and stability basically kernel startup times and other things. That's sort of behind the scenes underneath. That's still a continued focus and in particular we want to optimize bulk cloud object operations and improve performance overhead there in terms of things there. So again a lot of under the hood improvements in addition to still bringing feature releases such as the upcoming version 14 to the cloud and other, you know, new features to cloud as well. I want to highlight enterprise support. As I mentioned, our last EPC release was in August. Our next EPC release 165.1 should actually be out to customers hopefully the next week or so and the following subsequent EPC releases already in the middle stages of Texas testing. That is part of our improved build pipeline and release process I mentioned earlier. And it's certainly the goal to have EPC releases be available a few weeks after public cloud releases. That's our goal and we're still continuing to reduce that gap and we've made Good strides for that. Another thing this year in terms of things that we've done for enterprise and other support is to improve our abilities for other sort of customized solutions. One parts of that is having auto scalable EPCs and some other related options for different cloud providers. We have some customized or particularly tuned instances for particular use cases such as auto scalable APIs in particular for LLM integration. We've made strides and improvements there for custom solutions and auto scaling of EPC or cloud like abilities there. Our long term plan to containerize the cloud architecture to move into a more modern container structure is still ongoing and it is prioritized and we made some good strides there. A lot of the improvements we've made on EPC are sort of related to that project and we don't quite have something ready yet for customers there. But look for that in the coming year. I'll possibly at the end of Q1 for some early betas of that, but I guess part of the good news out of that is a lot of that work has also driven smaller pieces of that. So what we're working on is something that is wide enough to fit all of our needs internally and externally, which means it needs quite a bit more testing. But there are again more targeted use cases where we have some existing solutions for containerized or other similar, similar things that have a bit smaller scope that we can certainly split off and help use. So this does say enterprise, but we are more than willing to work on, you know, small, medium and large product projects and custom solutions. You know, contact us about that. All right, I want to at least leave a little bit of time for questions. Sorry if I've run through this a little bit fast. I have a more information section here to see about cloud implementation notes and also about our release notes, but a quick summary. We're continuing to build mini Wolfram cloud based storage applications. It's important to understand some differences about the cloud environment, including the different kernel types. Overall, we're shooting for a more consistent and quicker release cycle, particularly for EPC releases. Continuing to prioritize performance and stability improvements, key highlights and notebooks area that we're working on is manipulate performance, our editor overhaul and the ongoing containerized architecture work in progress. I want to thank all of my team members. There's too many to thank there for that. They make a lot of this possible. We do have our office hours tomorrow and at that point I will shut up and take questions. All right, let's see. Just running down from the top. From Dave, let's see when cloud deploy is used to make a notebook available to the public, is there any way to make the URL easier to type than a URL with a huge number in it? The answer is yes, you can actually specify the URL that you want. It's another argument to cloud deploy. I would say take a look at the cloud deploy documentation page and if you pause I'll try to add something at the end of the chat for that. So Nicolay asks, would it be possible to use a local kernel for the cloud or web based front end to remove the latency between them? So, short answer. In principle, yes. In practice I'd have to think about that more. It wouldn't be super easy to do that, I guess, I guess, I guess my answer is I sort of do that with my local development cloud. So when I run my local development cloud, they're obviously on the same machine and there is less latency there, but that doesn't always eliminate the largest part of it overhead. Good question though. What are, you know, George asked, what are the limitations of the Wolfram Playground? That's a bit custom, so there's some options there. So basically it's utilizing certain restrictions in terms of you can restrict it to say maybe a certain number of evaluations for each user. If the user refreshes, it's going to go back to a completely blank notebook for their page and then again it's subject to evaluation and time constraints limits that are tied to that account. So basically there are resource limits and some things there. But it's a good question again, kind of going down here again, if I'm going too fast, just let me know. Don asks, have there been improvements in logging the location of a user connecting to the cloud, duration of connection location, etc. I would say you might have some experience there that we want to hear about and help you. So there is some internal logging that we do around Cloud Connect, which may be what you're talking about. And of course we do have logs about, you know, on our servers when something reaches out. I would have to look at your more specific example to kind of better answer that. So yes, we have logging. I know there have been some improvements, but we probably want to hear more from maybe your situation or specific. So again, feel free to reach out to my email or come to our talk tomorrow and maybe, maybe that's one of the topics we can do. Okay, looks like Kurt also responded to the cloud object naming. Did I miss questions? Other than being late on Danny's request to zoom in on my Cloud notebooks. All right, so we are at 11. Again, appreciate you guys all for showing up to my talk. I hope it was somewhat informative. Please reach out to us in different areas and we are always welcome to your feedback once needs and concerns for cloud or related things. Again, thanks to all my team. Thanks to the team helping put this conference on as well and go enjoy lots of other talks. Thank you. Looks like Joel did answer one question that I missed about EPC release separately, and I think he answered it pretty well there, but I'll say it again. So Michael had asked, you know, what, what is an epc? Why is that released separately if there are different features? So the. I guess the first answer is it's essentially the same code base. It's bundled or packaged a little differently, which makes it a little bit trickier on our end to do some things. What it does is basically expose configuration parameters that let you change things a little bit differently than what the public cloud has. For example, you can be more permissive in terms of some functionality that you can allow, whereas in public cloud we disallow. There's quite a few functionalities at the language level that we disallow around, for example, socket connections or run process or other things that we just disallow. We didn't want to try to have to deal with security ramifications for that, but in your epc, you can basically configure those to still be on. And then there are some other things sort of bundled with EPC around authentication mechanisms or interfaces as well. So there's a few other things around that. But it's essentially the same underlying code base, but packaged a little bit differently and gives you a little bit more control on some configuration settings. And so primarily the cost for us is testing the different configurations and making sure that we've tested it enough in a way that we can give it out to you guys. That's primarily the reason why it takes a little bit more time. And yeah, I think we hopefully haven't overstayed our welcome too much, but I will continue to hang around a little bit until they kick us out, I guess. Okay. Yes. Kurt's relying me to. Somebody had asked about compliance with various things. I would say reach out, reach out to us. It's sort of dependent on your. Your case and how we set it up. Usually in this case, we were talking about, you know, either a private cloud, either that you host, or a cloud that we work out with you that's running separately on a virtual private network inside Amazon, or a different cloud provider, but that's also a bit more dependent on how you're using it and what you're doing with it. And, and we have to get more experts on my end to answer that better. Yeah, I mean, you know, I mean, somebody else trying to answer there a lot of. It's exactly on how the data is used and other things. But, yeah, I can't, I can't answer that with more, more details and sitting down and with, with more knowledgeable people on our end. I'm sorry, looking at my other screen. Awesome. Okay, I think I'm going to go to some other talks. Again, thanks, everybody. And see you around, hopefully.