How to Securely Deploy Agents in Sandboxes (ALM Best Practices) | Salesforce

Thank you so much for joining today's session. My name is Amanda and I am a product marketer here at Salesforce working primarily on sandboxes. And I'm lucky enough to be joined today by Aaron, one of our internal engineers here who helps us ensure that we are building agents securely in sandboxes. But today we're gonna be talking about sandboxes in the context of Agentforce. So for those of you who know sandboxes, We like to think of it as a separate environment, a separate sandbox, if you will, where you can throw the sand around, you can move fast without breaking things. Because the cool thing about sandboxes is since it's a separate environment, you're not worried about disrupting your business, you're not worried about corrupting your data. And second about sandboxes is you can actually help validate your agent performance. We found that using a full copy sandbox gave us a lot of success because this is a 100% replica of your production data. So you can run realistic user acceptance testing, and you can also make sure that you're testing those new agent features, giving developers a safe space to, to refer rapid prototyping and iteration. Raise your hand if you guys are familiar with Asiaforce VIBES, if you heard it in the keynote today. Okay, a few hand raises. So Mark talked about AgentForce Vibes, which is really helping us boost developer productivity. So the easiest example I can give is if you want to build a button that says click me, you can literally type a prompt that says please create the code. I want to make a button that's blue and says click me and boom, you can see the code. The last feature that we're going to talk about is AgentForce Testing Center, which is actually embedded in your sandbox. So I like to think of this as similar to a flight simulator. You wanna make sure that you're running through all the different scenarios before you literally take flight. So for Agent Forest Testing Center, we talked about that help agent earlier. And so an example here is that if a customer is writing in and saying, "Help, I forgot my password," they may say it a different way. They may say, "Eek, what's my password? I'm locked out." So you wanna make sure that those inputs are generating the same output. And so you can see in the screenshot here, we're talking about input, we're talking about those utterances. Because we can't talk about all of the tools that we here at Salesforce use to test our own agents. We mask our data, we use Agentforce Vibes, we use Testing Center, we also use some of our Shield products like Event Monitoring and Security Center, of course, Backup and Recover as a CYA measure. To double-click into our environment management strategy. If you look at the top row here, when you're doing something similar like Salesforce releases or business process changes, we find we've been successful just using the scratch org. Developer Pro works fine. But when we were building our agents, we found it was really helpful to have a separate full copy sandbox. Just to reduce the risk. And to put that all together and to give you a visual, I am going to introduce Aaron. Aaron, please take it away.

Thank you, everyone. Thank you so much for the opportunity to talk about sandboxes and the testing center and CodeBuilder with you. So I'm gonna just jump right into the demo to kind of build up on what Amanda was talking about. So the screen that you're seeing here, I'm logged into a sandbox. And I'm logged into Agent Studio, the agent builder. This is a full-fledged development environment using low-code. So I'm a developer, but a business person can use this without using code. And here I can create agents using from scratch. It comes with templates for agents. It comes with templates for your actions. It comes with templates for your topics. Or I can use AI to build my agent for me. We can use AI to build AI. That's amazing. All through your browser, all through the sandbox. So the sandbox is the Amanda was referring to. All of those come with this IDE built in. I can deploy agent from here, I can undeploy, I can create it. Everything you can think of building an agent right through your sandbox. As Amanda was mentioning, you're isolated to your environment. And if it's a dedicated sandbox, you're not interrupting your customers, your production, or if your other teams are doing testing, you're not impacting them. You have a dedicated sandbox to test and build your agents. So here we have the sales agent that's widely used in Salesforce. You heard Mark refer to the sales agent. So I'm making a change to the sales agent. And before I ship this to production, I have the agent open in my studio and I'm testing, make sure it works. So I've given it a prompt here. I said, can you draft me an email? And the results I'm seeing here is what I expect. But more importantly, it's showing me the inner workings of how this agent goes ahead and responds to this prompt. So I can see that it goes, this prompt ID is used, sorry, this topic ID is used to reply to this prompt. Here's the reasoning that it tells me that it's using. And it also is telling me which action it used. So all the inner workings of the agent that is used to answer this prompt is right here. And because this is a studio, I can go here and make improvements right here. I can expand this email, I can shrink it, whatever I want to do. Okay, great. So I tested my agent. It's ready to go. But I did minimal testing and it was manual testing, right? That's not— that doesn't work in the real world. We need automated testing and we need, we need a platform for that testing. So how do you create that platform? Well, the good news is that Salesforce gives you that platform out of the box with these sandboxes. So again, these AI-enabled or Einstein-enabled sandboxes, they all come with an agent code builder as well as a testing center. Everything I need to build my agent securely, test it thoroughly, and ship it to my customers. So this is the testing center. Now there's two main ways that I can create a test here. One is, again, using AI to test AI, and I don't want to use a CSV. I'm going to show you the AI first. So I can go here, click new test, give it a test name. I need to specify which I have 4 agents on this sandbox. I'm telling it this agent's under test, give it a brief description. Now, if you watch Mark's keynote, he talked about Data 360. Data is context and context is data. So you can connect your testing center to your data to get granular context-enabled test cases. But for sake of this demo, we're gonna skip that for now. And here I'm gonna tell AI to get, I can specify up to 100 cases here, but for this case I'm gonna say 20. And I'm gonna say which topics for me to test. I can pick only, I can choose one topic or I can pick multiple topics. In this case I'm gonna pick all my topics. And additional testing, this is more for keeping track of, make sure your agent doesn't drift, make sure your agent doesn't introduce toxicity. I can select additional for completion, completeness, coherence, and I can persist this data on my sandbox. And this is ready, generate test case. So AI goes ahead and generates the test case for me. All the test cases here is generated within the test center. It goes ahead, figures out what my actions are, what my topics are, what data is stored in my sandbox, and based on that, it creates all the tests for me. That's it. I just created 20 tests using AI. Now the other way to create tests is CSV. So we're going to create quickly a test using from scratch. There's a few keywords that Testing Center needs. Adherence is my prompt. Let's see with that. Adherence is my prompt. So I want to make sure that I give it the same prompt I use in a manual process I'm using here. And I can make this test from scratch because the manual Step told me that my expected topic is this one. So I just copy that from the manual screen and I paste it here. And also it told me what my expected actions should be. So that's it. I can create a test from scratch, just these 3 keywords. Now I'm doing negative testing as well. I want my agent to stay within its realm of authority. I don't want it to go answer unrelated questions. Questions because you know AI can get out of control. It can try to be smart and answer questions. So you can add negative tests to make sure that they treat those as prominent injection and it's empty action. So that way you make sure your agent stays within its boundaries. And that's it. I just created a test from scratch. I can go ahead and upload this test. I give it a name, same agent under test. Some scenario, some description, add my context variable, and I can upload the test right here. I can upload that CSV and that's it. It's done and the test is executed here. So for the sake of this demo, I'm gonna try to speed things up. Takes a few minutes for the test to run, about 2, 3 minutes. The test is run here. You can see that expected action matches actual action, expected topic matches actual topic. My negative test has passed. This gives me a good scenario. Now for this demo, I'm only showing you two cases, but just to give you a sneak peek, this is our actual test case for a sales agent. We have 180 tests here. Wow. Anytime we release a change to the sales agent, all of these 180 tests get executed. So it gives us full assurance before we release that agent to production that is fully tested. So when we release it to production, we know full confidence all of these tests pass. If one of these tests fail, that production pipeline is broken. We're not going to production. The other thing is— how am I doing on time? OK, good. The other cool thing about Test Center is that I'm a developer. I'm a DevOps engineer. That's my title. So automation, right? What I showed you right now is point and click. Well, that's not how you build agents. We want to build agents through a CI/CD pipeline. So what I can do with VS Code, I can retrieve that test as an XML file, the test that I just created, whether through AI or whether through CSV, I can retrieve it as an XML file, add it in my Git repo. So now I'm testing as code. So anytime I do a commit to that Git repo, my CI/CD pipeline will make sure that those tests run. And if the test fails, any of those tests fails, the pipeline will be broken. You'll be notified, Slack channel, whatever that case may be. So that's another good thing about the testing center. All right, so that's for testing center. Amanda talked to you about Vibes. Salesforce— I'm sorry, Mark talked to you about Vibes. So let me just give you a quick sneak peek into it. We're short on time, but we'll give you a quick sneak peek. So as developers, you know, We've been coding agents with Apex classes, flows. You can even use Python, Node.js, your own endpoints, but that's all writing code, right? Well, AgentForce Vibes allows you to use natural language. So what I'm gonna show you here now, I have this prompt, okay? I just have this prompt. It's a create an AgentForce agent capable of autonomously managing the complete lifecycle of a customer support issue. And I give it some environment, it's like your name should be this, your audience is this, this is your core capability. These are technologies I want to use, I want you to use. And this is the business value I want to get. So I'm gonna give this to my agent. So I have VS Code, I have Salesforce, Salesforce VICE plugin installed. It's here. I have an empty project. There's nothing in this default folder. And I'm going to paste that here live. And Amanda told me that I'm being very brave. So let's see what happens here. I'm going to paste this, all right? Fingers crossed. All right. AI is that every time you give it the same prompt, chances are you'll get a different result. So here, this, I'm getting a little bit different result than I did it in my warmup, but it understands what I'm asking it to do. So you can read some of that text for yourself, right? It's designing the architecture. It knows that it has to use Salesforce flows, Apex classes, and it knows how to organize that data structure. I am already connected to environment. So these are, I'm connected to my org here, this environment. And I wanted to create the deployment files. I'm not going to— I'm going to deploy it myself. So I say create only the— it goes ahead and creates it for me. So these are my Apex classes. So these are my actions. And these get automatically generated on my project here as we speak. So it's creating a case closure class. It determines the business logic, what it needs to do before the case And that can be closed. Now it's creating the flows. But once it's done, I can, from right here, I can deploy it straight to my Arc. So it creates the agent and I can deploy it to that sandbox that I was logged in. And once I deploy it there, I can go back and tweak it through that agent builder that I was showing you. So it builds everything from scratch, deploys it for you, and you can go back to the agent builder and tweak it further.

Thank you so much.

Thank you.