Googles 6 NEW AI AGENTS Takes The INUDSTRY BY STORM! (Google Gemini AGENTS)

So 2024 is the year of agents and Google just hit the scene with an amazing update because they just announced these six agents that they are going to be continually working on for the rest of the year and into the future. You can see customer agents, employee agents, creative agents, data agents, code agents, and security agents. And this video is going to show you some of the demos that Google actually recently talked about that are quite impressive. And it seems that we're gearing up for a year of agents as we previously talked about. So with that being said, let me know which of these agents you're going to be most excited for in the future.

Now let's look at the types of agents customers are building on Google Cloud using generative AI. First customer agents. You know, similar to great sales and service people, customer agents are able to listen carefully, understand your needs, recommend the right products and services. They work seamlessly across all your channels, the web, your mobile app, your point of sale and your call center. And they can be integrated into products experiences with voice and video. Mercedes Benz is working with us on customer agents to help people in their amazing cars. Let's hear from their CEO Ola Kalinas.

At Mercedes Benz, we want to offer our customers an exceptional digital experience. That's why we're equipping our cars with high end computers. Each car should only get better over time, just like a good wine. And with the power of Google Cloud and AI, we will make the user experience even more personalized. Our partnership across Google helps us build more intuitive and customized experiences. Last year we announced our partnership with Google Maps and Today more than 3 million customers are using Google places in their Mercedes cars. And we are applying Google Cloud AI across a number of other use cases ranging from a smart sales assistant, improving customer service in our call centers and optimizing our marketing. The sales assistant, for example, helps customers to seamlessly interact with Mercedes when booking a test drive or navigating through Mercedes offerings to find their next favorite vehicle. And now we're exploring further opportunities to work with Google Cloud AI, such as next level navigation features. In addition, we're partnering on one of the most exciting technology topics in our industry. Automated driving. This beautiful car right here is equipped with a Level 3 system for conditionally automated driving. We were the first manufacturer to get it certified in Germany, California and Nevada for our next generation internal development and test platform. We will use Google Cloud as the backbone, helping us to become even more efficient and flexible in our product development and Google Cloud's expert knowledge in processing massive amounts of data and Scaling AI workloads will ensure that our cars get even more intelligent and AI driven. Partnering with the very best in their respective fields is an important part of our software strategy and Google is the perfect example of that. With Google Cloud, Mercedes Benz is building new ways to deliver the most intelligent vehicles to our customers and to create personalized, intuitive experiences. We're really excited about working together. Thank you for having me.

Thanks Ola and team. In addition to these great innovations, we're very excited to collaborate with Mercedes Benz and work towards bringing Gemini models to their in vehicle conversational system. Another great example is Samsung who's deploying Gemini Pro and imagine on Vertex AI to the Galaxy S24 smartphones. Users can now take advantage of amazing features, text summarization, organization and magical image editing. Gemini Pro also provides Samsung with critical Google Cloud features including security, safety, privacy and data compliance. We're inspired by the agents our customers are creating using our Generative AI platform and our models. Intercontinental Hotels Group will launch a travel planning capability to help each of you the guests plan their next vacation. ADT is building an agent to help customers select and set up home security systems. Verizon gives agents better recommendations. Magaloo, one of Brazil's largest retailers has put Generative AI right at the heart of its customer service. IMG built a chatbot to enhance self service and improve answer quality and Target uses AI on the Target app and website to personalize offers for Target, Circle and Starbucks At Drive Up, Minnesota's Department of Public Safety helps non English speakers get licenses and other services with real time translation. Best Buy is building an assistant that will help troubleshoot product issues, reschedule or combine, order deliveries or manage software. Discover Financial Services is using search and synthesis across detailed policies and procedures during customer service calls and Orange's French language agent is grounded in support knowledge, transforming their help and contact site and their customer experience. Oppo and OnePlus leaders in smart devices are incorporating our Gemini models and Google Cloud AI into their phone to deliver innovative customer experiences including news, audio recording, summaries, AI toolbox and much much more. You know the opportunity for customer agents is tremendous. To help each of you build customer agents faster, we're introducing Vertex AI Agent Builder. You can now create customer agents that are amazingly powerful in just three key steps. First, you can use Gemini Pro to create free flowing human like conversations with text, voice, images and video as inputs and personalize them with custom voice models. Second, you can use natural language instructions to control the conversation flow and guide it on specific topics. You don't want it to discuss, such as current events. In the same way that you train your human agents, you can also control when it hands over to a human agent with transcription and summarization of its conversation history to make these transitions extremely smooth. Third, you can improve response quality with vector based and keyword based search to connect your internal information and the entire web. You can also use extensions to complete tasks for customers like updating contact information, booking a flight, ordering food, and many more. And you can integrate enterprise data from operational databases like AlloyDB, predictive analytics with BigQuery and SaaS applications like ServiceNow. Now let's take a look at an example of a customer agent in action. Please welcome developer advocate Amanda Lewis.

Thank you, Thomas. So last night I was watching a video of this band and I love the keyboard Player shirt. So I was thinking I'd really like to be wearing that shirt tomorrow night. But can I find it in my size and in time to be rocking it at the concert here in Vegas? Let's head over to my favorite store, Symbol Fashion. They just launched a customer agent and it leverages Gemini and Vector Search to deliver a seamless shopping experience. What can we help you find? Well, I'd like that shirt, but I guess I have a few other specifications as well. So find me a checkered shirt like the keyboard player is wearing. I'd like to see prices, where to buy it, and how soon can I be wearing it. I'm going to include the video. Now the customer agent is using Gemini's multimodal reasoning to analyze the text and video to identify exactly what I'm looking for. Then Gemini turns it into a searchable format. Okay, how cool is this? It found the checkered shirt I'm looking for, right? And some other great options in no time. And that's because these results harness Google's trusted search technologies, which ensures customers like me get the right results in record time. The suggested products are grounded in Simple Fashion's inventory and historical performance data to make sure customers leave happy. And with that purchase in hand. Okay, the first one is perfect. It looks like I can have it delivered in four days or pick it up nearby today. Like I said, I want to be wearing it tomorrow night. So I'm going to go with the local store to be safe. Of course, it never fails. They only have three left in my size. I don't want to miss out on wearing this shirt. So I'm going to go ahead, give the store a call and ask them to set it aside for me. But first, let me tell you what's happening behind the scenes? Symbol Fashion's customer agent is using Google Cloud's full suite of AI capabilities to offer customized support interactions, facilitate transactions like purchases and returns, and ensure that I'm receiving the most up to date information in real time. I'm so close to having this shirt for the concert. Let's give the store a call.

Hi there, this is the Symbol Fashion customer agent at South Las Vegas Boulevard. Am I speaking with Amanda?

Yes, this is Amanda.

Great. Thanks for reaching out. Amanda, I see you had a session on another device. I've sent you an SMS message with a link to our live chat companion. If you would like to switch to chat, please click the link. How can I help you today?

I'd like to purchase the shirt in my cart with the cart I have on file.

Absolutely. I see you're also a Symbol Fashion Rewards member. Looks like you have a 20% off voucher available to use. Would you like to apply it to this purchase?

Yes, please. That would be great.

The shirt you're purchasing goes well with these items. Also available for pickup in your preferred size. Would any of these be interesting to you?

Absolutely. Please add the white shirt and the boots to my cart.

Great. Your total is $203.73. Okay to proceed with the card on file? Yes, your purchase is confirmed. Do you need anything else today?

No, I'm all set. Thank you. Incredible.

Thank you for shopping with Symbol Fashion. You'll get a text when the items are ready for you.

In less than five minutes, I was able to find and order the shirt I wanted and an outfit to match. I'm ready for the concert. Back to you, Thomas.

Many of our customers are building employee agents using Gemini for Google Workspace and Vertex AI to help their employees be more productive and work better together. For hundreds of millions of people around the world, Uber is an essential mobile app that makes it easy to get a ride or a home delivery. There are many complex processes that must run smoothly behind the scenes for Uber to keep providing its magical user experiences.

Generative AI opens up opportunities across all parts of our business. But the opportunity I'm particularly excited about is developer and employee productivity. Our partnership with Google Cloud is helping us create a stronger and more empowered workforce. Gemini for Google Workspace helps us save time on repetitive tasks, frees up developers for higher value work, reduces our agency spending and enhances employee retention. We're also using Google Cloud's AI platform to build our own AI agents to help our support teams better help our customers for frontline support. We've launched new tools that intelligently summarize user communications and surface context from previous interactions. Anything that helps our teams go faster helps our platform go faster so our customers can get where they want to go or what they want to get delivered all at uberspeed With Google Cloud, Uber's creating new ways to empower workforce and better serve our customers. Thanks for having me.

Gemini for Google Workspace is our AI powered agent built right into Gmail Docs sheets and more. Gemini doesn't just help our customers be more productive, it's a member of their team. Need a quick answer on something? Just ask Gemini. Need a world class note taker in an important meeting? Bring Gemini along. Need insights from a dozen analyst reports. Gemini is your research partner. Customers really appreciate the quality of output that Gemini for Workspace provides. More than 70% of Help Me write users in Google Docs and Gmail accept Gemini's suggestions and more than 75% of people who have created images and slides insert them into their presentations. We're seeing the impact of this every day with our customers. Pepperdine University has students and faculty who speak many languages. Gemini in Google Meet enables real time translated captioning and notes. PennyMac is accelerating recruiting, hiring and new employee onboarding. And Woolworths, the leading retailer in Australia, is seeking to unlock greater productivity for over 10,000 administrative employees by providing them with Gemini for Google Workspace now. Today we have quite a few announcements with Gemini for Workspace. First, many customers have asked for a way to get AI powered Google Meet and chat company wide at a lower price, particularly to move away from Zoom. In a recent benchmarking study that tested the top video conferencing applications, Google Meet outperformed Microsoft Teams Zoom and WebEx in overall video and audio performance. So we're announcing the AI meetings and messaging add on with take notes for me, chat summarization and real time translation and it will only cost $10 per user per month. Second, we're making AI powered data protection available with our brand new AI Security Add on Now. Workspace admins can automatically classify and protect sensitive files and data using privacy preserving AI models trained for their organization. Third, Gemini and Chat can now catch you up on long conversation threads, summarize decisions and be a real time creative partner. Let's have Product Lead for Workspace collaboration apps Christina Baer give you a quick look at at Gemini for Workspace.

Thanks Aparna. There are lots of ways Gemini for Workspace is helping our customers save time. Let me show you the magic of one example in action. I'VE been asked to evaluate two proposals for a new payroll system. I open up my drive and I see that there are two statements of work here or sows that outline the vendor proposals. Google Drive is AI ready without any additional pre work and everything here is protected with our industry leading security. All I need to do to get started is drag and drop the two files over into the Gemini side panel which is currently in preview. In just a few seconds, Gemini for Workspace will give me an outline of these proposals so that I can have a good idea of what they're about. And I could go even further and ask compare the price.

Let's see.

Price of the two offers. Each of these documents is over 70 pages. It would have taken me hours to review these docs, but instead Gemini is going to help me find a clean answer to save me a ton of time. And as you can see, I have the answer right here in the Gemini side panel. Let's go ahead and dive into one of these proposals. So I'm going to pick this one. One of the things that you'll notice about Gemini in docs is that it will proactively show me a summary of this file so that I can get a sense of what's going on without needing to take the time to read the whole thing. But before I proceed with this vendor, I need to ensure that no compliance issues exist. And I'm going to be honest, I have not memorized every rule in our compliance rulebook because it is over 100 pages. I I would have to need to scour the 80 pages of this proposal and compare it manually with 100 pages of the rule book. So instead in the side panel I ask does this offer comply with the following? I'm going to ention our compliance rulebook, hit enter and see what Gemini has to say. Okay, so interesting. Gemini has found an issue because the supplier does not list their security certifications. Because Gemini is grounded in my company's data with source citations to specific files, I can trust this response and start to troubleshoot before selecting a vendor. Gemini for Workspace was terrific in this example, saving me a lot of time without compromising on accuracy. And this technology can be applied in so many, so many other use cases. A sales team analyzing RFPs, a recruiter developing interview questions, and so much more.

Back to you Parna. Thank you, Christina. Many of our customers are building employee agents to automate workflows that are tedious and repetitive. For example, Bristol Myers Squibbling is transforming its document processing for clinical trials, something that took scientists weeks now takes them just minutes to get a first draft. Dasa, the largest medical diagnostic company in Brazil, is helping physicians detect relevant findings in test results. Sutherland, a digital transformation company, is boosting its customer service by suggesting better responses and automating insights and employee agents can work across people and teams to complete a task. One of the nation's leading healthcare providers, HCA Healthcare, is using our generative AI technology to help caregivers spend more time with patients and less on paperwork. They're testing Katie, so Katie is a nurse handoff digital assistant, helping to ensure continuity of care when one caregiver shift ends and another begins. It delivers summaries in everyday language to ensure that caregivers can share information easily and provides them up to the minute patient data to expedite the discharge process. It also helps prepare patients for recovery at home by simplifying post discharge tasks like post op physical therapy and more. The Victoria's Secret and company is looking to help in Store associates offer tailored recommendations to customers. The Home Depot's app called Sidekick, helps store associates manage inventory and keep shelves stocked. Etsy uses Vertex AI to improve search, provide more personalized recommendations to buyers, optimize their ad models, and increase the accuracy of delivery dates. Vertex AI, especially when combined with Workspace, lets you create powerful employee agents. First you create a custom model in the ways that we've shown before. From there you connect them to all your company and web data. This can also be done with translation so that your company information is available regardless of language. Similarly, we support multimodal inputs including videos, call audio, images in addition to text. Now you will want to ground that in enterprise truth using databases like AlloyDB, BigQuery, and data from enterprise apps like SAP. And announcing today, HubSpot. Let's take a look at an example of an employee agent in action. Please welcome developer advocate Gabe Weiss.

Thanks Aparna. Hi folks. So I know you all want to hear about awesome AI stuff that's coming, but I need to talk to you for a minute about my annual benefits enrollment. See, I forgot I have to finish signing up by today and as you can see I might be a little bit busy. So if you don't mind, let's go ahead and look at this open enrollment email together. Okay. Yep. I've got a deadline. I knew that. Thank you. I've got FSA stuff. I've got an online portal for my company, HR Benef. Okay, there's a lot here. Oh, they included video. Let's see if this makes my life easier. And the video Is. Ah, okay. So it's almost an hour long. Yeah, I'm not going to have time to review all of this stuff. Let's see how this employee agent that we've developed using Google Workspace, Gemini models and Vertex AI might be able to help me. As you can see, it's integrated directly into my Google chat so I don't have to context switch while I'm figuring all this stuff out. First things first, let's. Let's have it summarize the email and the video that it sent me. Summarize the body and attached video from my recent email with subject open enrollment closing. So behind the scenes, the agent is referencing that email body and its attachments as context in the prompt using retrieval augmented generation. That way its response is limited to the content that matters to me.

Me.

The Gemini model's multimodal capabilities allows the agent to understand and reason across text, audio and video from a single prompt. I mean, this is a way quicker read. Okay, good. And I can immediately see that the medical plants have been completely revamped this year. Let's go ahead and jump into the benefits portal to see more. Now, I've already done my dental and my vision, but I procrastinate. I mean six saved the most important plan for last my medical plan. Let's see how this option stacks against my existing coverage. Compare these coverage options to the PDF doc I have on the Platinum plan. The Gemini model's long context window paired with vertex extensions enables the agent to cross reference large amounts of data from a variety of sources, including unstructured data like PDFs. Leveraging Gemini's advanced reasoning capabilities, the agent is able to understand the complex details of my current plan and compare it with the new options for 2025. And since the enterprise grounding features links me to the exact data that Gemini used to draw its conclusions, which you can see linked here, I can confidently trust its recommendation that the gold plan is best for me. And done. So now let's get a summary of my coverage. Let's say my house is multilingual, so I'd like to have it in Japanese. Also, please generate a summary of 2025 benefits in a Google Doc in both English and Japanese. Although my source material is in English, the Gemini model Support for over 40 languages enables it to understand and respond in Japanese. And here we go. By engaging with employees in their native language, the agent can help everyone feel comfortable navigating their healthcare needs for the next year. So, one last thing. Now that I've officially completed enrollment, my daughter's gonna need braces this year. Let's use the agent to help me find the right orthodontist. My daughter will need braces this year. Help me find the best in network orthodontist near my home With Vertex extensions, the Gemini model can integrate with any external or internal API. This allows the agent to join my dental coverage details directly with real time Google Maps and Places data to help determine the best ranked in network providers near me. Taking a quick look at my options okay, so it looks like Symbol Orthodontics is the only one that offers Saturday appointments which works best for my family schedule. But what do I really know about them? How do I know that the agent actually picked the best orthodontists? Because the Gemini model natively supports grounding With Google Search, I can see what the agent found summarizing, saving me hours of scouring the Internet for fresh information. The best part is I can get the annotations to see where the information came from. Which means if I want to look for myself, I can. I have great places to start and a quick scan of the summary shows me that they're great with kids, which is great. I particularly love this customer review. I'll go with them. Let's schedule a call. I don't know how it's going to find a schedule though. I mean, I'm here. The agent knows that I'm at Google Cloud NEXT because it's integrated with Google Calendar and it can find a time to schedule the call. Yes please. Awesome. We got a call. Do I need anything else? Nope.

Good.

No thanks. Perfect. Nice and easy. Just imagine what else an employee agent can help you with.

Thanks.

Back to you Aparna.

Next up, Creative Agents Janjeev AI is transforming how marketing teams are creating content, personalizing it, and making real time campaign adjustments. Creative agents can supercharge your design and production teams working across images, slides and other modalities. We're seeing really interesting creative agents from customers. For example, Carrefour, one of the world's leading food retailers, is pioneering new ways to use generative AI for marketing. Now, using Vertex AI, they were able to deploy Carre4Marketing Studio in just five weeks, an innovative solution to streamline the process of creating dynamic campaigns across various social networks, empowering the marketing teams to efficiently promote products and engage with their target audience. In just a few clicks, they can build ultra personalized campaigns to deliver customers advertising that they care about. Other customers, such as Canva, are using Vertex AI to power its magic design for video, helping users skip tedious editing steps. Procter and Gamble is accelerating the development of photorealistic images and creative assets, giving teams more time back to focus on higher level plans. Wpp, the creative transformation company, built a performance brain using Gemini 1.5 Pro on Vertex AI to power its media activation tool called WPP. Open Media Studio Advertisers like the Google Pixel team can now see predictions of which ads are going to work the best before a single impression has been served, avoiding millions of dollars of media waste over the life of campaigns. Now, while these examples are about professional creative teams, we believe that everyone can be a great creator and a great storyteller, but the formats and tools for storytelling at work haven't really changed that much. How many times have you heard, should we start with a dock or a deck? Well, we can do a lot better. I'm absolutely thrilled to announce our newest workspace app, Google Vids, Sitting alongside Google Docs Sheets Slides. Google Vids is an AI powered video creation app for work with Gemini. In bids, you have a video writing, production and editing assistant all in one. Let me show you how simple it is to get started with bids. Now, after a full week with all of you here at Next, I'm going to want to share a recap video to share share all the excitement with my organization. When I open up Vids, Gemini helps me get started. I simply type in a prompt using an existing document for context. Now, based on that prompt, Gemini suggests a narrative outline for the story that I could easily customize and edit. I choose an expressive style and Vids works its magic. Let's see what I get. So wow. Just like that, I get the first draft. With beautifully designed, fully animated scenes, complete with relevant stock media and music, and even a generated script, I'll be able to add fun videos and photos that I captured this week from my media directly by accessing Google Drive and Google Photos. And even without any video expertise, I can build out scenes like a pro just as easily as I make slides. But it doesn't have to stop there. With Vids, you can personalize your story your way by recording yourself or by using one of our preset voices by Gemini, and by enriching scenes with high quality media from our stock content library or from your own to really make your story shine. And because Google Vids is a workspace app, it comes with all of the collaboration capabilities that your teams already know and love. Vids doesn't just help you create videos, it helps you become a better storyteller with a rich new medium at work. This isn't just a vision. Vids is already in the hands of Alpha customers who have been giving us really great feedback as we work towards exchange, expanding to Workspace Labs in June. I'm looking forward to sharing more about Vids with all of you in my Spotlight session tomorrow. Beyond Vids, many customers are building agents for their marketing teams, audio and video production teams, and all of the creative people out there who could use a hand. We provide a tremendously powerful platform and stack to build creative agents first, it takes the best image generation model Imagen 2.0, our most advanced text to image technology, helps businesses create images that match their specific brand requirements. This is now generally available in Vertex AI. Imagen's strong language comprehension and photorealism capabilities contributed to Google securing the leadership position in the recent Forrester Computer Vision wave. Second, as a part of Imagen, we're now introducing text to live image in preview. Today, marketing and creative teams can generate animated images from a text prompt, including product images, ads, GIFs and storyboards. Here's an example of a daylily opening in the morning dew and a pot of something delicious simmering on the stove or an aerial view of a mountain range. Third, we are proud to announce the general availability of digital watermarking for AI generated images produced by Imagen. It's powered by Google DeepMind's Synth ID. Finally, we're announcing new editing modes for Imagine 2.0, which will make it really easy to remove unwanted elements in an image, add new elements, and expand the borders of the image to create a wider view for all of you. When all of these tools come together, it's a powerful combination.

Our Symbol Outfitters product team is working on designing a new tent that we're eager to bring to market. The tent itself is still in development, but we want to get started on a marketing strategy right away. Let's see how the creative agent we developed with gemini models, imagine 2.0's API and Google Workspace can help. First things first, let's get a sense for the look and feel of our brand. What are the key styles and themes.

Of of our brand?

The creative agent can analyze previous campaigns to understand our unique brand style and apply it to new ideas. In this case, the creative agent has analyzed over 3,000 brand images, descriptions, videos and documents of other products that we have in our catalog contained within Google Drive to create this summary and as you can see, it summarized our brand identity as bold, adventurous and aspirational. This is exactly the look and feel that we're going for. The Creative agent was able to use Gemini Pro's 1 million token context window and its ability to reason across text, images and video to generate this summary. Next, I'm going to prompt our creative agent using Imagine to dynamically generate amazing new images of our tents in iconic and real outdoor settings. This will help us brainstorm campaign locations. All of this, of course, will match our unique brand style. Now, because I'm a terrible typist, I'm going to use Chrome's built in clipboard history, so we've asked it to create these images. We'll give it just a second. Check those out. Machu Picchu. Not bad, Half Dome. These are real landmarks, accurately presented. Now let's work on a narrative to really help us land the message. I want to make special note that in this prompt I'm going to ask the creative agent to provide captions in both English and Spanish and that I'm using a multilingual prompt to do so. Awesome. The creative agent leveraged Gemini Pro's multimodal capabilities to brainstorm creative captions that match each photo. These look great too. And thanks to our multilingual capability, we're able to share the same story with our customers.