Use AI Agents to Analyze Power Grid Optimization

Hello everyone, and thank you for joining today's session. My name is Rhys, and I'll be your moderator for today. We are going to get started in a couple of minutes. We are just waiting so everyone has a chance to to join. In the meantime though, make sure you registered for this session. If you haven't done so already, you can scan the QR code that's on screen now. You can also find the link that I've put in the chat as well. And you can also head over to datacamp.com/webinars where you'll find this session as well as all of our future sessions as well. If you would like to join in today, you can sign up and create an account with Zerve, and then you can also get 100 additional credits on top of the 50 you would usually get Uh, but you gotta follow the link that we've put in the session resources, which is pinned in the chat, uh, or in the video description. So you can look in either of those places. Um, we've also got the prompt that we're gonna be using in this session. So yeah, if you do wanna join in today, sign up, create an account with Zerve, and then, uh, yeah, make a note of the session prompt that's in the video description and in this session resources doc, which is pinned in the chat. So either option, whatever suits. You. If you have any questions or comments throughout the session, let us know in the chat. We're going to be answering your questions towards the end of the session, so make sure you stick around for that. And for anyone else that's just joined, welcome. Uh, my name's Rhys. I'll be your moderator for today. We are going to get started very shortly at the top of the hour. We're just waiting so everyone has a chance to join. Uh, if you haven't done so already, please do register for this session. You can do so using the link in the chat or using the QR code on screen. You can also head over to datacamp.com/webinars where you'll find this session as well as all of our future sessions as well. If you would like to join in with the session today, uh, you can create an account with Xerve and then get 100 additional credits on top of the 50 you would usually get. Uh, and you can do that by following the link that we've put in the video description and in the session resources document that is pinned in the chat. Uh, and then once you've created an account with Zerv, which is actually very, very quick. You can make a note of the session prompt that we've got in the video description and in the resources doc. And yeah, that's pretty much what we're going to be working through today. So very easy setup. If you haven't done, done so already, it's very, very easy to do and you can get it done beforehand as well. We're also going to be covering a few slides at the start of the session. So those will go out with the on-demand version of this with the recording. So if you'd like to get the slides for today's session, make sure you register for today's session. They'll be going out with the recording, which will land in your inbox tomorrow. Brilliant. We are pretty close to getting started, but I will just run through the housekeeping notes for everyone one more time. So yeah, if you are new here, this will be new to you, but if you've already heard this, apologies, we'll be going live very, very shortly. So yeah, for anyone new, thank you for joining the session. We are just about to go live. If you haven't done so already, please register for the session using the link that's in the chat. You can scan the QR code that's on screen and you can also head over to datacamp.com/webinars where you'll find this session as well as all of our future sessions as well. If you'd like to join in live with the session today, please do follow the link that's in the video description or in the session resources doc that's pinned in the chat. You can create an account with Zerv and get 100 additional credits if you use the link that's in there. We've also got a session prompt in there as well, so make a note of that if you'd like to join in with today's session. And we've got some other stuff in there for you as well. But yeah, the two main things, if you'd like to join in with us live, create an account with Zerve using the link that's in the video description and make a note of the session prompt as well. Brilliant. I think that's everything from me. So now I'll hand you over to your host for today's session, Richie. Richie, please take it away.

Hi there, Data Scamps and Data Champs. This is Richie. Welcome back to the Agents in the Enterprise series. I'm particularly excited for today's session since it combines two of my favorite things, data science and AI agents. So I'm sure you're aware that the hottest of hot topics this year is using agents to solve almost any task you can think of. It turns out that agents for data scientists are starting to get really good, so I'm keen to see how well they perform today. We're gonna be working through a case study on, uh, Power Grid Load Balancing using the Agentic Data Science platform, Zerve AI. And I've got two fantastic guests for you. First up is Greg Michelson, who is Chief Product Officer and co-founder of Zerve AI. Welcome, Greg. Great to have you here.

Richie, I just think we're having some issues. I'm not sure Greg can hear you at the moment.

Um, okay. That's not a great sign. You can hear me though, right?

Yes, I can hear you just fine. Everyone, if you can hear Richie Cotton and myself in the chat, please do let us know. All right.

We'll give Greg a moment to figure out his speaker settings.

In the meantime, I can't hear anything. I've got no audio. Refresh, please. Okay.

Okay. We'll give him a moment. So Greg is building Xerve AI and he champions the perspective of Xerve's users, helping ensure the platform reflects the realities of day-to-day working with data. Uh, Greg's background is, uh, that he was an early leader at DataRobot, so the machine learning automation platform. Actually, Greg, can you hear me?

I can, loud and clear.

Wonderful. I was just introducing you.

I don't know what happened there. That's wild.

All right, excellent. Yeah, so the short bit: Greg's a machine learning and AI expert. Uh, so, uh, joining him, uh, today is, uh, Zach O'Hagan, who leads the go-to-market and growth team at Xerve. Welcome, Zach.

Hello everyone, how we doing? Good to be here.

Great to have you as well. And I'm presuming you can hear me.

I can hear you loud and clear. Yeah, yeah.

Wonderful.

The tech issues are the hard part. The actual data science and AI stuff, easy peasy by comparison.

Yeah, we've been on for half an hour just talking and prepping and getting ready and everything. Audio is fine. The moment somebody pushes go, it goes away.

You know what, I figured let's just do this. Go on, take it away. Who's going first? Greg, you're listening? All right. I'll hide backstage.

All right. Awesome. Well, Zach, how are you this morning?

I'm very good, Greg. How are we doing?

Pleased to hear it. Can't complain. Let me get my slides up. All right.

We can hear you.

You can?

Yes, we can.

All right, fantastic. Share slideshow. Okay, awesome. So we've got, I think, something really fun today. It's a code-along. So I think I didn't hear the intro, but we did point people towards the signup code and the prompt and all the materials for the session, Zach.

Yep, we did indeed. So it's in the YouTube description. So you've got a link to sign up for Zerve that will give you 100 free credits on top of the normal free credits that you get with Serve, and then there's a session resource link too below that in the YouTube description. Awesome.

Okay, so we're going to do some cool stuff, and I hope that all the folks on the call today will, will follow along, will sign up and follow along and have some real good sort of meaty type questions to kind of motivate the conversation. I'm going to try to keep this to, you know, 30, 35, 40 minutes or so so that we can have plenty of time for Q&A at the end. But what we're going to do today is look at an example of a kind of a research or a data science workflow that we build live in real time using ZURB, using an agented coding environment, because the ground has really shifted under our feet. I remember when I was working at Travelers Insurance, we had this problem that we were trying to figure out why a particular type of car, car insurance claim was more expensive. And that led to us building a whole data pipeline that would pull data and analyze it and identify, you know, why a particular type of claim was more expensive today than it was last year at the same time. And the process of building it was so painful because we had to write every single line of code and check it and verify that it was doing what it was supposed to do. And the unit of work was writing code. And so you had data scientists and machine learning engineers and you had quants or whatever you might want to call them that were sitting on machines typing code into an IDE or a notebook or something like that to try and make stuff work. And Zach, you've been out talking to all kinds of folks that are doing this type of work. Is that what you're seeing now, or are things just wildly different?

Things are wildly different, particularly in the last 12 months. I think the interfaces are changing. I think folks that never used to code before get their hands dirty now, and whether that's a good or bad thing, we'll see. I think it's ultimately a net positive, but the, the pace of change and development is just insane.

Yeah, it's crazy. You know, when we, when we first flicked on our agent in Zerve, we went from, from one day where 100% of the code that was being written in Zerve was written by the user, and it was, it was a tool for experts and data scientists and coders and engineers to develop code and host it and build pipelines and so on. And the day we flipped on the AI agent, we went from 95% We went to 95% of the code written in Xerve being written by the agent. Yeah. It's, it's outrageous how good the large language models are at writing code. And so now instead of the big hurdle being, you know, who's, who's writing this code and, and how are we testing it and, you know, checking syntax and making all that stuff sure. It's now, it's different. The bottleneck is different now. It's like, what should we be doing? What questions should we be asking? You know, how can we trust this? When do we need to intervene? What kind of tests and guardrails do we want to put up? And so the, the workflow is just super different, even though the questions of the actual process are the same, right? We're still trying to solve the same sorts of problems. What we're, what we're looking at now is a fundamentally different approach to working. Whereas in the past you might sit in a Jupyter Notebook or in VS Code or in, uh, you know, Databricks notebook or, or Snowflake or whatever and, and pound out code in order to produce your visualizations or to host your dashboards or whatever. Now it's a question of iterating with the agent. So it's like, tell the agent something to do, you know, write a good plan, make sure that the plan works, watch the agent write the code, run it, review the results, and so on. And so it's wildly different.

Hey, Greg, where do you see the trust coming into that? So you mentioned view the results. So questions I get asked all the time are, you know, how do I trust this? Now, I think the more sophisticated the LLMs get, there's more and more trust there. But how do you think about that in the data science context of, you know, how do I trust these results? Or at least how do I, you know, sense check them?

Yeah, that, that's the right question to be asking. Because, I mean, all of us have used ChatGPT or Claude or somebody in the background to ask us questions about like day-to-day life, right? Like I ask questions about, you know, I'm cooking steaks on my Traeger and I need to know how long to cook them for. And it tells you an answer, but how do you know that it's right? Right? You know, especially when these things first came out, the questions were, or the answers were questionable sometimes. And hallucinations do actually happen. The big difference between what's happening in Zerve and what might happen if you asked ChatGPT or an environment like that to help you with your data science is that Zerve's environment is persistent. And so I can actually see the code so I can read what it's doing. I can look at the charts and I can evaluate what they're showing me. I see the, I see the process that was used to generate the results. And so in, in data science, the, the output is really the way that you evaluate what's going on there. So it's not a question of, hey, you know, how do I optimize my power grid, which is the example that we're going to be looking at. And it's more like, okay, I know I need to optimize my power grid, but now I've got like, you know, visualization showing me load on each line and I've got cost measures for the different types of power and I've got, you know, different regions that are being connected to other regions by distribution lines and stuff like that. And all that can be visualized so that you can actually see what's going on in the project that you're building. So I think that's where trust comes from. It comes from being able to look at what was done, look at those visualizations, see the pipelines and the trail that was taken by the agent, and then, you know, evaluate it, edit it, tweak it, change it, and then really honestly feel good about what was done, but without actually having to write all the code. Do you agree with that, Zach?

Yeah, I think so. I think that makes sense. I'd love to see it in action because I'm sure there's some details that, you know, seeing is believing. But yeah, maybe when we get to the hands-on bit, we can look at that.

Yeah, so we're going to get right there. So the power grid is kind of an example, a bit of a toy example, not a toy example, but kind of a toy example. We're going to simulate a power grid and we're going to have different types of power generation that costs us different amounts. And then we're going to have different transmission systems for getting the power to the consumption areas and then different demands that vary over time. By those different regions. And we're going to use that optimization problem as a way to illustrate how you can build one of these workflows, but also, you know, evaluate it and trust it in the end. So for the purposes of this example, we've got, like I said, generation, we've got transmission, and then we've got load. That's the demand that's coming from the, the users of the power. And we're going to generate this data. So we'll be using synthetic data for the purposes of this. And then we're going to try and build an application that will allow us to not only optimize the generation of power for, for the purposes of minimizing cost, but also a way to evaluate weak points in the grid to say, okay, if this generator goes down, if, you know, this line is cut, you know, if this, you know, that sort of thing, whatever might happen, we can still evaluate how our grid would work. So we can do some what do you call it, we can like identify points of failure and stuff like that when it comes to what we're trying to do here. Pardon me. So it should be a good example. Have you ever worked with this kind of data before, Zach?

No, but the first question that was coming to mind was, I guess, like utilities are a pretty heavily regulated industry. So, you know, synthetic data is cool for this purpose, but I guess, what would you answer if someone was to say— a utility company was to say to you, well, I can't just throw my data on your cloud. How does that all work?

Oh, uh, most of our customers are actually self-hosted. So we knew from the beginning that, uh, the way Zerv would be needed to— the way company organizations would need to use Zerv is, uh, locally, like in their— in their— not locally on their laptops, but in in their own VPC so that the data could live, you know, behind their firewalls, the compute could happen in their environment and so on. So that's not really a concern.

Yeah, yeah, makes sense.

So here's the prompt that we're gonna use. This is in your session materials. We're gonna build a synthetic power grid with a certain number of structures in it, and we're gonna simulate some demand that varies over time throughout the course of the day. And we're going to try and build a system that'll optimize the generation of power and minimize cost without having outages. And so we're going to ask it for a certain set of visualizations like what does the grid look like? How does demand over time change? What does cost look like for each different type of generation? And then we're going to ask for an interactive dashboard that'll allow us to tweak some of those levers and see how it impacts our overall cost. And whether we can simulate some outages and stuff like that. So it should be really interesting. So if you want, go ahead and copy-paste that prompt and get ready to paste it in. There's a link in those materials that'll allow you to sign up, and you get 100 bonus credits for signing up. So definitely follow along. Definitely, you know, work with the app along with us so that you can ask them Really good questions and so on. The point here is that we're not going to be writing code by hand. We're going to be managing the agent doing the work here. So this should be a really fun experience. If anybody's worked with, with Claude Code or with Copilot or with Cursor or any of the agentic coding environments, this one's going to feel a bit different because it's wired into the infrastructure. Our job is going to be to watch it do the awesome stuff that it does. Let me switch over here into the app and let me copy that prompt and we'll get going. Okay.

Greg, that's probably the first question I get, to be honest. You mentioned Cursor and Claude Code and a bunch of those guys. What are the key differences between Xerve and those other tools that you mentioned? Because it's almost every single call I have, it's, you know, why wouldn't I just use an independent LLM, or why wouldn't I just use a cursor? So maybe double-clicking on what you mentioned about tied into the infrastructure, I think, could be helpful for folks.

Yeah, I like it. Okay, so I've just put my prompt in there, and let me give a bit of a tour that hopefully will answer your question a bit, Zach. Sure. What you've got here in the middle of the screen is something that should look very familiar, and that is it's going to look a bit like a notebook. But what we're actually working with is different here because this is all cloud-based and you've got an entire environment here baked in. You have a file system, you've got environment management, You've got like assets for data connections and credential management, stuff like that. And the agent over here, which you can see has created a plan for accomplishing what we've asked it to do, the agent has access to see all of that stuff. And so if I had, say, like a Snowflake database or something, the agent is going to have knowledge already about what's in that Snowflake database. So it will have already, you know, scraped the table names and the column names and the, you know, all those details. And you're going to see some things here. So look, what you can see here is the agent can not only write code, but it can also execute code. And you can see it tried to use a package that wasn't installed. And so there's a couple of paths that might take. It might try to install that package into its environment, or it might try to rewrite that code in order to use some other sort of package. So what the agent does here is not just write code like you might see in a Copilot, or you might experience if you were like running locally using VS Code or something like that. Here it's wired into the entire system so that when you, uh, when you have code written, not only that, but you can also have execute that code and have the compute automatically spin up and manage the storage and all that sort of thing. So you can see here, it's gone in and it's fixed that error. And this first block here is it says grid synthesis. So I can look at the description of what's happening in this block. And so this is where the agent is creating the, the dataset that identifies the demand and the number of nodes and stuff in my particular power grid. The agent can see your data, any files that you have, the output from the file execution. It can see the code that's been written. It can inspect the values of the variables that are created. That can inform the process that it that it takes for writing the code rather than you having to say, okay, there's a file here, here's the file name, these are the contents of the file, you know, these are the column names in my dataset, this is the column that's in particular interesting. And so instead of having to teach the agent all the details of what's going on in your environment, instead of having to manage all your infrastructure and make sure that the compute that you need is available and that, you know, you've got the right servers spun up and the deployment is built properly and stuff like that, this agent is wired into that. And so the agent can do all the things that you would normally have to manage yourself. Does that make sense, Zach?

That does make sense. And the next natural question that I always get asked is, is this for folks who haven't seen it before, is this Zerve's LLM? What, what models are powering this agent and is there flexibility there?

Yeah, it's a bring your own key type situation. So right now we're using Claude under the hood for the app now. So Claude is driving the agent here. But most of our enterprise customers end up bringing their own keys because they already have like enterprise deals with, you know, whoever might be OpenAI or Anthropic or whatever model providers, or some even use like open source like Llama or you know, some of the, some of the open source ones that they're hosting internally. So there are lots of different ways to power the agent.

Yeah, got it. Makes sense.

So a couple of things that we did ask the agent to do. And so what we're going to be looking for, I think, as the agent goes about its way working on this particular project, we're going to want to be looking for visualizations that are showing what's happening in the power grid. If I can zip down to the bottom here, I can see a bit about what the agent is approaching. We'll want to see like, are there— what does the map of the grid look like? What's close to what? Probably by closeness in this particular example, we're looking at you know, how, how reliable or how heavy the connection is between two particular nodes. So it looks like a visualization has been created here. Here's my grid. And so I can see the darker red color looks like it's congested lines. So these are lines that have a heavy, heavy load at peak time, whereas green lines are less congested. The stars represent power generation and the node— the connections between the nodes look to be transmission lines that take power from the generators to the consumers of the power here. And so we've got a bit of a map. This is the data that was generated. And so, you know, you might look at that as an expert in the space and say, you know, this isn't really, you know, this XYZ might not be realistic. You could pause the agent at that point and say, hey, you know, let's revise the, you know, what, what the, what, how the, how the network was generated. You might go in and, and tweak things a bit and edit it. And of course, the code is editable here. So like you get all the code here and it's not an ephemeral type environment. So in a tool like you know, Perplexity or ChatGPT or some of these, it's an ephemeral type thing. Like your stuff only exists in your chat window. And then, you know, there's no artifacts along the way. That's very different here. So if I understand how this graph works, you can see the agent is just working linearly here. But what each of these nodes does here is you can think about the connections between the nodes as the passing of data between one node to the next. Okay. You see up here also, this is where our application is. So we're going to come look at that here in a little bit.

And Greg, I see that. How do we see that data? How can we inspect that? Going back to that kind of trust element of, you know, how do I know that these results at the cell level are, you know, reliable or what I would expect as an expert in this field? Okay.

Yeah. Yeah. So, so every— so you can think about each block or each cell in your notebook, it's very different than Jupyter. So if folks are familiar with Jupyter, Jupyter has a global variable space. And so anytime you run any cell in any order, it's impacting the values of those variables in memory. And so it's real easy to get into a bad state where, you know, you've run something out of order or you've run something too many times and you've got to restart your kernel and then rerun all your code and clear your memory and all that kind of stuff. But here what happens is when a node runs, when a cell runs or a block runs or whatever you want to call it, I click run and it spins up serverless compute to execute that code. And then the variable space for that particular block is serialized, cached, and stored on disk. And so if I open up my canvas view again, here's the block that I'm on. This variable space right here that I'm looking at that I can preview and debug and so on is inherited by the following block. And so it's, we when this block executes, it always starts with where the previous preceding blocks ended. And so then that block runs and I can see not just the output, but I can also see the variables that are created there and so on. So each block preserves all of the artifacts of the analysis so that you can not only easily debug it, but also use it. And it's asking if we want to deploy our, our app. So I'll go ahead and click approve there so that it can get to deploying. So you can not only view those for debugging purposes, but like if I train a model, say if I train a random forest or something like that, and I want to deploy that thing, it's already serialized. So I don't have to worry about like Dockerizing it or pickling it or, you know, hosting it somewhere. It already exists. It's already serialized. It's already stored. It's available to be used by our deployments or by, you know, some external deployment or whatever it might be. Okay, so it's 8:25 now, 8:25 AM my time, different times for different time zones, of course, and the agent just finished. So what it does when it finishes here, it's giving us a summary of what it did. So here's the data generated, here's the demand. I can see all the charts and graphs that it created. So here's like demand profiles by region within the topology, different types of generators. Cost different amounts of money. So solar is the least expensive, whereas a gas, this type of gas, is the most expensive. I've got different sorts of— pardon me— what is this here? So this is generation amounts, different types of power plants and how much they're generating. And then congestion. Okay, wow. So we have a super congested map here. So that, that's actually interesting. So maybe the demand that the generation created may be excessive, maybe too much for the generation that the— so the data creation may be weird. So we might have to go in and look at that. And then we've got some price heat maps across hours of the day by node. So something looks a little funny there. So we might go in, let's look at the dashboard.

And so maybe before we look at this, a couple of questions in the chat here that I might call out for looking at. Oh yeah, sure. So first one, does this power grid optimization use integer programming or linear programming solver?

That's a good question. We can go and inspect the code here. So let's look and see what's going on here. Let's look at— here's our optimization. Using linear programming.

Okay.

So that's the, the approach that it's taken. I can, if I click here, I can look and see a bit more details. And I can also ask the agent. So I could say like, describe your optimization technique. Describe your optimization technique. And it'll come back with an answer for how it did what it did here in a second. So I can converse with the agent about what's going on in the project. And the agent can see all those details and understand what's going on here. We'll look at that here when the agent answers.

Nice. While that one's cooking, in a real-world scenario, are these visualizations geo-referenced?

They could be. It just depends on what your data looks like. For example, whenever I work with geo data, I always switch my environment. To geospatial. So we have a bunch of pre-baked environments built into this, the system here that utilize, you know, things like Geopandas and stuff like that. Of course, you, you can add any custom requirements that you want in here, but it all just depends on what the data looks like as far as, you know, what, what can I do from a geotagging perspective and all that sort of thing. So here's a description of our optimization technique. It's talking about linear programming and what the decision variables look like and the objective function and all that stuff. And so that's some more details in terms of how it did this problem from an optimization perspective.

There's one other question here, but I think it's kind of a UI question. How does one add the prompts on the far right and the data used and the code on the far left? I'm looking at a console. So do you know what? Rich K6094, I can respond to that in the chat because I think you might just be looking at the home page. So crack on, Greg. Let's have a look at the dashboard that you're going to show us.

Yeah, let's see if it works. So, so I will pop this up in a new window and we'll see if that sticks. All right. Can you see the dashboard?

I can indeed. Yep. Power Grid OPF dashboard.

Awesome. Congestion, that's a concerning plot. This makes me feel like maybe we need to go revisit the data generation step because you would want to see more, some variety in what's going on there. In fact, let's go ahead and do that and just have a look and see what's going— wrong, stop. There it is. Okay. Let's jump back here and say the congestion graphic looks a bit odd. I think maybe the data generation step has created an oversubscribed power grid. Do we need to add more generation or reduce demand? So in a normal problem, you can't just go in and say, okay, there's too much demand, right? But here we simulated data and it looks like maybe the, the constants or the assumptions that we made going into the process of doing this was— were maybe not what's the word, maybe miscalibrated so that the, you know, things are a little out of whack. So the agent's going to go in and have a look and revise it. But now I should notice here that these are not— this is not a locally hosted application. This is just a preview deployment. You can actually set it up with a real URL. So I might call this, you know, Power Grid Then you can see the URL down here. It's going to be pretty faint, but it's power-grid.hub.azure.cloud. I'd have to click Deploy and it would take 3 or 4 minutes to deploy the thing and have it hosted. But that is a live URL. Of course, you can manage access and security and all that stuff. But that is powerful because what you've got here is a way to build these workflows, to build something that is shareable. And so like, for example, if I were to take this and I'll share with Zach and invite him in here, and I can change his privileges just like in Google Docs. I can make him an editor or a viewer or whatever it might be. And, uh, you know, he can come in and see everything, uh, and change things and code and all that sort of stuff because I have him as an editor. Maybe I want him as just a a viewer so that he could have a look at it and so on. So I have the ability to adjust that stuff, and he's not having to like, you know, download files and, uh, you know, set up his environments, create virtual environments to make the code able to run. We're all working off the same page here, same version of like, say, NumPy or, or, you know, whatever, Geopandas or whatever it might be. So it solves all those problems related to you know, like setup and dependency management and things like that.

So a couple more questions here for you, Greg, if you don't mind.

Sure.

No, of course. Yeah, there's one good one here, I think. So in agentic systems like this where optimization, data quality, orchestration, and human steering all interact, what typically becomes the limiting factor in production? The model, the data, then in a second message, design or human decision layer?

Wow, so many variables there, right? So a lot of times the data is a big hurdle, but one of the things that Zerv does is something called data discovery. And so like if you were to connect, like I said, a Snowflake or a Databricks or, you know, a BigQuery or, you know, whatever, a MySQL database, whatever it might be to to Zerve, you do that using what we call assets. And so you create a data connection. And so like when I add that data connection into Zerve, then what happens is I get a lot of like, where's my Snowflake? So like if I add in Snowflake here, if I were to click add, it would add it to this project. And Zerve has already investigated what's in that database. And so you're adding a whole lot of intelligence that's prebuilt. And so it knows, you know, how many missing values are there in a particular column? How many, you know, what are all the tables in my database? What have I previously learned about this data that'll inform the way that this project develops? And so a lot of the data obstacles and hurdles get overcome by just having an agent that knows what's in the dataset. Um, the, the coding is no longer a hurdle. It used to be a problem, uh, but nowadays you don't really, you know, you, you, the, like yesterday, for example, I'm working on a little electronics project, uh, just in my, in my free time to, uh, and I won't go into the details what it is, but I needed a housing for, for a motor, uh, that would go into this grid that I'm building and I don't know anything about 3D printing. My son 3D prints and I was like, oh, maybe he can 3D print this part for me. So I jumped into Zerve and I said, hey, make me a 3D printing file that has the following specs. And it did it. You know, so the expertise gap has really sort of gone away. I don't know a thing about STL files. It turns out they're STL files that get created. But within the space of 10 or 20 minutes, I had one and I could send it to him. And I said, hey, can you 3D print this? And he said, sure thing. That gap has kind of gone away. Um, the monitoring is, is becoming much more important. So when you're the one who writes your code and you're the one who is responsible for every line and all the debugging and stuff like that, um, then monitoring is less important because you monitor it while you build it, right? But now the agent is building it, and so it's more of a verification step that has appeared on the, on the sort of like in the environment of doing data science or doing research as you work that didn't really exist before. So I don't know that, that, you know, we've eliminated a lot of bottlenecks, but also there are some other bottlenecks that are sort of created here. So it's definitely a changing environment.

And on that point, Greg, because you mentioned verification there and there's kind of another question here that touches on the I guess the reliability and kind of the deterministic nature or lack thereof of the code that's generated. So someone had asked, I noticed my optimization code was very different, but functionally the same tiers. Will we get different code each run? So is that down to the data or synthetic data generation step? Is it the non-deterministic nature of the LLMs? So what do you think's going on there?

Yeah, the, the, there are a number of ways to skin any cat. And so the approach we took here was really describing the problem. We didn't ask for a particular methodology. I'm not an expert in like linear optimization or linear programming or something like that. You know, I only have a sort of a student's level knowledge of, of how all that stuff works and how you do it in real, real time. But we could have. Right. So to the extent that the user is an expert in a particular field, it might have said, use this particular package and do this particular type of optimization for this, you know, for this problem. So if you leave it open-ended, then the agent will pick and it may pick a different optimization method for at different times. Right. So that, that may happen. It's just sort of the nature of the beast. I will jump in here and say that it's, it's gone in and it's created a plan to kind of fix things. And so you notice that the code is being rewritten and rerun here. And so we've got some updates that will be coming down the line here, I think, for the way that that data was generated here. So it did find some stuff in terms of how that, how that's laid out there.

Yep, we've got another one in here, Greg. So some SCADA systems already have this kind of visualization. What added value does Xerox give compared to a traditional SCADA system? I had to look up what a SCADA system was, just being totally honest. Supervisory Control and Data Acquisition Systems. Are they systems you're familiar with?

Yeah, there's no question that if you were actually operating a power grid, that you would have a lot of this telemetry already built into systems. And I'm by no means suggesting that everyone that manages utilities should go out and rebuild their entire management infrastructure in order to, you know, use Zerv, right? This is an example of— this is— you think of this as a toy example. So what you might not be able to do, though, in some of the older school systems is the analysis related to the data. So you may have, say, telemetry on what the loads for each transmission line in your grid look like and how much power is being generated by each, each, each generation system, you know, by the solar in your, in your, your grid. By the nuclear in your grid, by coal in your grid, you know, whatever it might be. But your system might not be able to say, hey, you know, if you built a transmission line from here to here, then, you know, you could use more solar and you could use less gas, you know, whatever. Right. Whatever the optimization might be, that kind of stuff might not be built into the systems that, that are built just because they were built so long ago. Right. I mean, You know, everyone that we talk to goes, hey, our data is like really bad and our systems are like really antiquated and old. And, you know, we're working on new integrations and we've got, you know, we've got new things on the horizon. But, you know, right now it's kind of a mess. Like literally everyone that we talk to says that. And so this gives you a way to take the data that you're receiving from the systems that you have and do additional analysis and research on top of it. right? It may be that you say, you know, wow, this is really great. I can build like a dashboard that is really informative and really helps us to, you know, evaluate the way this grid is working and so on. It may be that it's more of an ad hoc type project that you'd want to do, right? Like, what are the most— what are the failure points for my system? And, you know, can I simulate an outage by, you know, if the temperatures get to 110 degrees and everybody in the city is running their air conditioner and you know, demand goes way up, but then where are my points of failure and stuff like that. So it just depends on the specific set of software that an organization is using and, you know, what it can do and what it can't. So that's sort of the nature of the beast with data science, I think.

Sure. One thing you said there, Greg, kind of stuck in my head, and it was kind of questioning the analysis. And, you know, dashboards will take you so far. Okay, yeah, you're beating me to this. So Um, does it tell me about this report? Um, does this give, uh, you know, an extra layer of detail? Like, how do you— how should users be thinking about these reports?

Yeah, so if— can you see the report coming up?

It's just— yeah, it's just generating.

Generating. Okay, awesome. Yeah, so one of the things that our users found, um, is that the, the stakeholders that they had might not feel comfortable working in a programming environment. Right? And so they wanted to provide a way to sort of interrogate the findings of an analysis that was safe. Right? So one of the things about large language models is that they're very eager to please. And so if I ask a large language model a question, you know, and I word it just right, it'll tell me anything I want. Like, if I say, hey, don't you think that X, Y, and Z is true? Then the large language model is going to look for a way to say, why, yes, Greg, you're so smart. That is exactly correct. So those, those type of hallucinations are insidious because they kind of like tell us what we want to hear. And so what we've built here is a way to interact with our project in a way that gives us some control and some safety in terms of what the, what the project could tell us. And so this report is cooking up here, but what it does basically is it uses the Canvas as source material for explaining what's gone on in the project here. And so I didn't give any specific prompt. I just said, give me a general summary of what's going on. And so it's, it's here creating this report. And so it's given me an executive summary and the, you know, the feasibility, you know, we're meeting the demands and we've got significant transmission generation and stuff like that. It's showing our grid infrastructure. It'll show many of the charts and many of the findings that happened in the original project. But as soon as it finishes loading here, what I'm going to be able to do is converse with it. And so I'll be able to have a chat with the agent and the agent will be able to answer the questions that I ask it, but with guardrails, because it can only use information in the project that it has access to, right? So it's not going to be able to go out and, you know, try to search the web or try to hallucinate or try to, you know, come up with stuff in order to make me happy. And so that conversation that I can have with the report is a bit safer than it would be if I just were asking ChatGPT or if I were doing, you know, some sort of analysis that was a bit more broader. So we've limited the ability of the agent to, to answer questions in an over-the-top kind of way. And we've limited it to the amount of data that's actually in that project. And so the report is done here. So I could go ahead and read that report. I might say, you know, let's see, this is too detailed. Can you make it more for our C-suite? And so, you know, you can give it direction as to what to make the report to say, or you can send it to your stakeholders and then they can talk with the agent and have the report update to match what they wanted to say as well. So here we go. It's rewriting the different sections and so on. And, you know, as soon as it's done there, we can ask more questions and so on. So this one, it's done here. Oh, the agent. I'm going to switch back to the project and see. So remember, if we remember, we asked it to reevaluate the congestion because it looked a little strange in terms of like what it was looking like. And it found that the line capacities were unrealistically high. And so on. So it redid a lot of the analysis here in order to, you know, update some of those changes that it found. So, so lots of stuff happening. It's just finished the report here. And so what is it? It's 8:45 now. We've been at this for, you know, 40, 40 minutes or so. And we've done some pretty impressive stuff. So we've got a report here that I could send out to stakeholders that they can talk with. I might ask it a question. Let's see, what's a question that it wouldn't be able to answer? Um, let's see. Okay, what are the environmental impacts of, uh, utilizing the optimized power distribution method? Do I need to consult with the EPA? On this plan.

Now we're getting some awesome feedback in the chat as well. Folks saying, I just built a dashboard from the prompt and this is amazing. So if more people want to share what they've been doing, we'd love to hear about it. But that's— and last, if anyone else has some cool stuff to share, drop it in the chat.

Yeah, I'm about ready to pause and just switch back to some Q&A here, but So yeah, let's go ahead and do that. So let's look at, uh, where is my— how do I get back to my thing here? There we go. So I'm going to stop screen sharing here, uh, for a second. And yeah, let's look at the, uh, the Q&A here. So drop lots of Qs in and we'll, we'll go through those.

Let's give folks a minute or two. Um, I think we got through Yeah, that was great.

And thank you, Zach, for, uh, going through the questions. It makes my life a lot easier.

No problem.

Um, yeah, so, uh, for anyone in the audience, if you still got questions, uh, for Greg, please do ask them now. In the meantime, actually, I've got a couple of questions for you. So I'm curious about how collaboration works, because quite often analysis is more than just one person working on these things. So what happens if there's several of you trying to solve problem.

Yeah, it's a very Google Docs-like experience. So like, if you ever used Google Colab or some of these other collaborative tools, it's a bit of a nightmare. In fact, I always laugh when I hear someone talking about Google Colab because the collaboration in Google Colab is so horrific. Like, it is a nightmare, you know, because you've got like cell locking, and if two people are doing the same thing at the same time, it's a big problem, and, and so on. We built this with collaboration in mind, so it does feel like Google Docs. And so if Zach and I were both in here at the same time, we could both run code simultaneously. We don't have that global variable space problem that notebooks have because each cell is basically an independent kernel. And so I can kick off the compute within each cell without, you know, impacting the— without creating errors, basically. And so we've created it so that you can collaborate smoothly. I was working on an analysis project with the US government during COVID and we had 4 or 5 folks that were working on an analysis of infection rates at the county level in the US. And we had a notebook, we were doing it in a notebook, a Jupyter Notebook. And at the end of like my shift, 'cause we were kind of following the sun with this analysis, I would package everything up and Slack the files to the next guy, and that was a nightmare. Like, using Slack as a collaboration tool for, for coding environments is crazy. I mean, you can't, you can't really manage notebooks in GitHub. Uh, you know, there's like so many, so many problems with collaboration in the space. So that was one of the problems we specifically set out to solve, uh, in Serve.

Uh, okay, that's good to know. And Actually, on that note, is there GitHub integration then, or integration with these other sort of version control platforms?

Oh, we lost you for a sec, Craig. If not, I can— no, we can't hear you.

Sorry, I'm muted. I'm muted. Okay. Yeah, it auto-muted me there for some reason. Yeah, no, we're 100% integrated with GitHub, so, or Bitbucket or whatever source control tools that you want to use. So In the actual integration, each cell becomes its own file. It's easy to look and see what's going on without creating— like a notebook, if you're managing a notebook in GitHub, every time you open that thing, you create 1,000 merge conflicts, like issues, because all the metadata is updating. We've solved that problem in terms of how you do that work.

Uh, yeah, certainly, uh, like the standard notebook format is not great for putting, putting version control. So that's good. Um, I guess more generally, like, uh, how does integration with other pieces of software work? So for example, uh, if you wanna do scheduling stuff in here, like integrating with Airflow, all that kind of stuff, or you mentioned integration with databases already, but more generally, like, yeah, how does Zerve play with other pieces of software?

So you can use Airflow with Zerve, uh, stuff like that. So there is an API you can, you can, you know, trigger block execution. You can, you can do all that stuff using the, the API. Zerve has a scheduler built in. We built some apps and stuff like that. You can also create APIs in Zerve very easily. So just as easily as I created that interactive dashboard, I can say, hey, create an, uh, you know, a FastAPI that, uh, you know, takes this as input and gives this as output. And the agent can do that stuff too. So you can create kind of like microservices along the way too. If you want to, if you want to go down that road. So lots of, lots of ways to get that done. Okay.

All right. And we just got a question in from Mark. So Mark says, when I used the prompt, I was asked 7 questions on all sorts of aspects, didn't know it was the same for Greg. Do you want to talk us through like, how the behavior differs and why some people might get questions or not?

Yeah, I mean, there's definitely a some unpredictability in terms of like, what's the agent gonna look at this time, right? You know, 'cause we noticed when like, when we kicked it off the first time, it tried to use a particular package that wasn't super common that was related to graph analysis that wasn't installed. When I, you know, I worked this 2 or 3 times myself before as like prep to make sure, you know, I could, you know, just as prep. And it didn't try to use that package in any of the times that I'd done it. So, you know, the— there is some variability in the way that large language models approach the, the problem. And so, yeah, it is what it is, right? I guess it's just like, you know, every time's an adventure. I don't think, though, that the right approach to solving that problem is over-engineering your prompt. Like the prompt that we put in, it was like, I don't know, 7, 8 sentences. That's about as long a prompt as I will ever use. I would really much rather put in a short prompt with the key details and then iterate with the agent than trying to overly constrain it with a very highly detailed, you know, long prompt. I don't think that long prompting, complex prompting is the right approach here. And that is probably the most common mistake that I see people make when they're interacting with agents, good agents. Poor agents, yeah, you have to over-engineer a bit. But with an agent like Xerve, it's better to be quick and short and then iterate.

Greg, worth calling out maybe the different modes that the Xerve agent has as well. If people want to be like, you know, we'll say a little bit more cautious and don't want the agent to jump into stuff straight away, maybe worth talking about those different modes?

Yeah, that's a good idea. Let me see here. Can you see the screen again?

No, not yet. Yeah, there we go.

Okay, awesome. So there's a couple of modes here for the agent. No one will accept— no one will be able to see Evolve mode. That hasn't actually been released yet, but it's super, super exciting. This is where you give the agent a budget and say, here's my problem, keep iterating until you solve it and get better and better. So the Evolve mode you won't see. Ask mode is like, don't write any code, just answer my question. Plan mode is make a plan and then ask me to approve it before you move forward. And then Build mode is just go, just do it. I don't need it. I don't want to give approval. I'm going to ask you a question and I want you to build. Those are the 3 options at the global level. The global section over here, this is where the agent has access to the whole project. If I want to constrain the agent a bit to just a particular cell, I might go here and say, I don't understand this code, add a bunch of comments. Says to the agent, look, I don't want you to be changing other cells. I just want you to go into this particular cell and I want you to make edits here. So this is a way that you can constrain what the agent is looking at. And you can further constrain what the agent is looking at by selecting just a bit of, of text here. And then if the agent's thinking, but otherwise there'd be a button here that says add to context. And so if I want to constrain the agent even more to say just a few lines of code, then I can do that as well. So I've got really fine-grained control over what the agent can edit. And so I'm not like going to worry that if I prompt the agent, it's going to blow up my project because I'm going to use the right level of control to control what the agent's actually doing.

Thanks.

So let me stop sharing there.

Okay. All right.

Nice. So with that, we're pretty much at time. I think the audience would have questions. So lots of great questions from the audience though. So yeah, thank you again, Greg. Before everyone dashes off, I want to say we got 3 more sessions this week. So please do come back tomorrow for a session on putting agents into production. So we've got someone from AI21 Labs coming along for that. On Thursday, we've got a session on how AI is changing search, particularly enterprise search. So I think lots of opportunities for basically improving like a lot of a lot of business websites there. And then on Friday, we've got a no-code session on creating GPTs. So please come back for those. So with that, yes, I'd say, yeah, thank you once again, Greg. Great to have you here. And that was a very, very cool project. I love the way, you know, we solved energy distribution in 45 minutes.

Yeah, I hate that we didn't get to all the, all the questions. If folks want to have follow-ups after, they can go on our website and join our Slack channel, our community Slack, and I'll be able to follow up with any questions that folks have that we didn't get to answer.

I've dropped a link to the community Slack in the YouTube comments as well, guys, so feel free to join.

Excellent. And thank you, Zach, for helping out with the questions. I love, like, you have the banter between you. It's like, I guess, the new Laurel and Hardy.

Yeah. Which one's which? Abbott and Costello. I guess you're Costello. Yeah.

Excellent.

All right. Thank you to everyone in the audience who asked a question. Thank you to everyone who showed up today. See you all again soon.