Google IO: Agents is The Future - Demos

But let me show you the kinds of use cases we are working hard to solve. Let's start with shopping. It's pretty fun to shop for shoes and a lot less fun to return them when they don't fit. Imagine if Gemini could do all the steps for you. Searching your inbox for the receipt, locating the order number from your email, filling out a return form, and even scheduling a pickup. That's much easier, right?

So let's go to her inbox and take a look. Lots of unread emails. Let's click on the first one. It's got a PDF that's a detachment from a hotel as a receipt and I see a suggestion in the side panel. Help me organize and track my receipts. Let's click on this prompt. The side panel now will show me more details about what that really means. And as you can see, there's two steps here. Step 1 Create a drive folder and put this receipt and 37 others it's found into that folder. Makes sense. Step 2 Extract the relevant information from those receipts in that folder into a new spreadsheet. Now this sounds useful. Why not? I also have the option to edit these actions or just hit ok.

So.

So let's hit ok.

Gemini will now complete the two steps described above. And this is where it gets even better. Gemini offers you the option to automate this so that this particular workflow is run on all future emails.

Another example that's a bit more complex. Say you just moved to Chicago. You can imagine Gemini and Chrome working together to help you do a number of things to get ready. Organizing, reasoning, synthesizing on your behalf. For example, you will want to explore the city and find services nearby, from dry cleaners to dog walkers. You'll have to update your new address across dozens of websites. Gemnack can work across these tasks and will prompt you for more information when needed. So you're always in control. That part is really important. As we prototype these experiences, we are thinking hard about how to do it in a way that's private, secure and works for everyone. These are simple use cases, but they give you a good sense of the types of problems you want to solve by building intelligent systems that think ahead, reason and plan all on your behalf.

Now you can ask search to create a three day meal plan for a group that's easy to prepare. And here you get a plan with a wide range of recipes from across the web. This one for overnight oats looks particularly interesting and you can easily head over to the website to learn how to prepare them. If you Want to get more veggies in? You can simply ask Search to swap in a vegetarian dish. And just like that, search customizes your meal plan. And you can export your meal plan or get the ingredients as a list just by tapping here. Looking ahead, you can imagine asking Google to add everything to your preferred shopping cart. Then we're really cooking.

It all starts with a prompt. Okay, so here we go. We're going to Miami. My son loves art, my husband loves seafood, and our flight and hotel details are already in my Gmail inbox. Now, there's a lot going on in that prompt. Everyone has their own things that they want to do to make sense of these variables. Gemini starts by gathering all kinds of information from search and helpful extensions like maps and Gmail. It uses that data to create a dynamic graph of possible travel options, taking into account all of my priorities and constraints. The end result is a personalized vacation plan presented in Gemini's new dynamic ui. Now, based on my flight information, Gemini knows that I need a two and a half day itinerary. And you can see how Gemini uses spatial data to make decisions. Our flight lands in the late afternoon. So Gemini skips a big activity that day and finds a highly rated seafood restaurant close to our hotel. Now on Sunday, we have a jam packed day. I like these recommendations, but my family likes to sleep in. So I tap to change the start time. And just like that, Gemini adjusted my itinerary for the rest of the trip. It moved our walking tour to the next day and added lunch options near the street art museum to make the most of our Sunday afternoon. This looks great. It would have taken me hours of work hours checking multiple sources, figuring out schedules and Gemini did this in a fraction of the time. This new trip planning experience will be rolling out to Gemini Advanced this summer.

Space and AI Premium customers. Let me start by showing you three new capabilities coming to Gmail Mobile. This is my Gmail account. Okay, there's an email up top from my husband. Help me sort out the roof repair thing please. Now we've been trying to find a contractor to fix our roof and with work travel I have clearly dropped the ball. It looks like there's an email thread on this with lots of emails that I haven't read and luckily for me, I can simply tap the summarize option up top and skip reading this long back and forth. Now Gemini pulls up this helpful mobile card as an overlay and this is where I can read a nice summary of all the salient information that I need to know. So I See here that we have a quote from Jeff at Greenroofing and he's ready to start. Now, I know we had other bids and I don't remember the details. Previously I would have had to do a number of searches in Gmail and then remember and compare information across different emails. Now I can simply type out my question right here in the mobile card and say something like, compare my roof repair bids by price and availability. This new Q and A feature makes it so easy to get quick answers on anything in my inbox. For example, when are my shoes arriving? Or what time do doors open for the Knicks game without having to first search Gmail, then open the email and then look for the specific information and attachments and so on. Anyway, back to my roof. It looks like Gemini has found details that I got from two other contractors in completely different email threads. And I have this really nicely organized summary and I can do a quick comparison. So it seems like just quote was right in the middle. He can start immediately. So green roofing it in.

As you can see, the teammate has its very own account and we can go ahead and give it a name. We'll do something fun like Chip. Chip's been given a specific job role at the set of descriptions on how to be helpful for the team. You can see that here. And some of the jobs are to monitor and track projects. We've listed a few out to organize information and provide context. And a few more things now that we've configured our virtual teammate, let's go ahead and see Chip in action. To do that, I'll switch us over here to Google Chat first. When planning for an event like I O, we have a ton of chat rooms for various purposes. Luckily for me, Chip is in all of them. To quickly catch up, I might ask a question like anyone know if our I O storyboards are approved? Because we've instructed Chip to track this project, Chip searches across all the conversations and nose respond with an answer. There it is. Simple, but very helpful. Now, as the team adds Chip to more group chats, more files, more email threads, Chip builds a collective memory of our work together. Let's look at an example to show you. I'll switch over to a different room. How about Project Sapphire over here? And here we are discussing a product release coming up, and as usual, many pieces are still in flight. So I can go ahead and ask, are we on track for launch? Chip gets to work not only searching through everything it has access to, but also synthesizing what's found and coming back with an up to date response. There it is. A clear timeline, a nice summary and notice. Even in this first message here, Chip flags a potential issue the team should be aware of. Because we're in a group space, everyone could follow along. Anyone can jump in at any time. As you see, someone just did. Asking Chip to help create a dock to help address the issue. A task like this could take me hours, dozens of hours. Chip can get it all done in just a few minutes.