Building Deterministic Reports with AI | Amazon X AI Working Sessions

For those who don't know me, I'm Andrew Hamada, co-founder and CEO of Reason Automation. And joining me is Jacob Mattson from Motherduck. He is going to give his own introduction here in a second. But while we're waiting, Jacob, where are you calling in from today?

I am just north of Seattle. I'm at home today in Everett, so not, not too far from you.

Oh, very cool. All right. So you've been enjoying the ridiculously nice and also a little bit hot weather we've had recently.

Yeah, correct. Correct. My kids are just like, you know, they don't— they don't know what to do with this. Yeah. Yeah.

They're like, wait, it's not 4th of July yet. Why is it hot and sunny and nice?

Yeah, yeah, exactly. Exactly.

Jacob, do you mind giving us a quick intro?

Yeah, absolutely. Hey, everybody. So I'm Jacob. I work in DevRel at Mother Duck. Motherduck is a company building a cloud data warehouse. And we're doing lots of really cool things that are like AI-first in the way that we're thinking about it. And so I'm going to walk you through some of the way we're thinking about this today, but not really from a product perspective, just from like a, hey, like, let's talk about the problems we're seeing. And yeah, that's, that's kind of what I'm, what I want to talk about. Yeah, I, one thing I will say that is a little bit unique about me is I come into the tech space from an accounting background, actually. So I worked in accounting for 10 years. I always wanted to be a CFO. And then I finally got the room in the room where I was interviewing to be a CFO. I was like, what am I doing? I don't even like this. I don't even like this. And so that kind of has led me just to be chasing curiosity and led me to, to a lot of things that are doing Lots of fun things enabling really cool stuff with data. So.

That's awesome, man. I love that you have an accounting background. Weirdly, that seems to be a not uncommon ingress point into e-commerce for some reason. So not the first person I've heard of like that. One other housekeeping note I forgot. Apologies, everybody. You can chat using the webinar chat if you want to ask questions or talk to each other, but you also have the ability to submit anonymous questions using the Q&A tool here in Zoom. So if you want to ask something anonymously, use that. And we'll try to answer when we can.

That is awesome.

All right.

We ready to go? Should I throw some slides up?

Let's do it.

All right. So, allow. What? Okay. Hopefully this works.

Yeah. Okay.

You see my screen. Perfect. So, I just threw some slides together. And really I think this for me You know, this comes out of the experience of working with ChatGPT or Claude and then asking real business questions. And like I kind of mentioned earlier, you know, I am an accountant, so I care about the numbers being right. And I kept getting the numbers slightly differently back every time. And so what I want to explore is like, what type of things do we need to think about in order to make that actually work? In the AI era with the data we have. You know, the feeling, the feeling that I really struggled with as we kind of got into a little bit more of this is like, the first answer looks incredible when you're asking your, your AI. And then once you get like, you know, 3 or 4 questions into it, it's really hard to determine what's right and what's wrong. And so that's what the talk's about. This title is going to make sense by the end. I'm not going to define it yet. But let's Start somewhere more concrete, right? So here's a question. What was our net profit last year, right? That's the whole question. It sounds pretty trivial. It's about the most basic thing a business can ask about itself. So where do you actually go to answer that question right now for your business? And I don't mean like the gross number on your dashboard. I mean like the real one. I mean the one that really lets you, you you know, that you're thinking about after your chargebacks, after your FBA, after ad spend, the things that really you need to know about to make decisions, right? Where do you go? Um, the problem, right, is that you don't have the cut that you want in your dashboard and you put that in a spreadsheet. Um, and export some reports, start stitching them together, and then in a couple hours you have a number, right? The problem is it's not really fully trustable, and then the next time you have to build it, you have to build it again. And, you know, we've all been in the meeting where it's like, yeah, yeah, I think this number's right, and someone else is like, no, no, no, I think it's like this over here. And then you guys, then it's like, all right, let's take that offline, right?

Let's go, let's go off somewhere else and figure out how to do this.

Uh, you know, I think, uh, one thing that, um, I didn't actually share this. I went, so I worked, um, I worked at this company in Everett called Funko. Um, we actually sold a lot of products on Amazon back in the day, probably still do.

I don't know.

I don't work there anymore, but I stood up our first e-commerce store. Uh, we sold, we set something up on, on Shopify in 2015. Um, and so that was a lot of fun. Um, but I got to experience this challenge firsthand of like, what, Okay, what's the actual, like, how profitable are we really, right, on these units? Um, and it's really easy to be like, okay, well, we know, um, you know, we have a landed cost of X and a, you know, sales price of Y, but what about all the other pieces in between?

Um, anyways, so I think, uh, you know, mentioning that you were responsible for understanding the profitability of Funko in e-commerce, I think everybody on this call is now paying 10 times more attention.

Yeah, I'll just drop little nuggets like that. Yeah, we had a lot of fun building that stuff out. It was crazy. It's crazy being at a toy company that was on the path to going public and did go public here locally. It was a lot of fun.

We're going to have to have a separate follow-up conversation.

Yeah, exactly, exactly. We can absolutely do that. Anyway, so we kind of end up in this situation where we have like these— we have our tools. Right. And we end up in a couple, in, in some bad trade-offs situations, right? Trust or flexibility. And then we still have to do a bunch of work anyway. You know, on the left, you know, we had our dashboards, right? Tableau is what we used actually back in the day. And it did, it was great. It was always the same number every morning. We could see it. It was emailed to us, all that kind of stuff, right? But I couldn't really ask anything. All the interesting cuts of the data that I really wanted to look at were not really available. And so of course what I'm doing is now I'm exporting to Excel and then building the spreadsheet, right? I can build everything, total freedom, but it's all on me, right? If I mess something up, it's hard for other people to introspect it. I always like to joke that like understanding someone else's spreadsheet is kind of like akin to alchemy, right? It's like, okay, like, how did you build the thing? Like, no idea. Like, good luck. So, you know, we end up with something where we can't ask questions and then something that is not updatable and there's a bunch of work to keep it updated and is fraught with risk, right? And so, you know, what we really want is an assistant, not a tool, right? One of my favorite tweets is from a researcher at OpenAI. He said, you know, hey, text is the universal interface.

Right.

This is what we really want. When you have a question about your business, you don't want to sit there wishing your pivot table had more features. You don't really want a better tool. You just want to ask in natural language, right? Hey, what was our net profit last year and which products are dragging us down? And I want to know that answer to be right and for me to be able to trust it, right? I want an assistant. I want a tool or not, not a tool, something I delegate to, not something I operate, right? And in fact, that's probably how a lot of teams end up getting hired around the space, right? Is that the need that people express is, hey, like, you know, I just want to be able to ask this and get the right answer so I can make the right decisions.

I've had that job.

Yeah, yeah, me too.

Me too.

Absolutely.

Yes, exactly.

Exactly. And I think like what's really cool now is that line between the assistant and the tool is blurring, right? The chatbox is where we've really seen a lot of this innovation surface recently, right? But it's not really the thing we want, right? It's just the interface because text is the universal interface, right? So, so what we really want is the intelligence behind the box, right? Something that understands the question and can go get an answer. Now the problem, of course, is, and this is true for humans in addition to AI, right, is you can ask the same question twice and you can get two different answers. Intelligence is non-deterministic, right? If I ask the same question twice, I get two different answers. That is normal. But we cannot put that into our QBR. We cannot put that into our P&L. If I ask what our net, what our profit was, I need the same number on Monday and on Tuesday. And so that's the real problem, right? It's not, can I get AI to talk to my data? It can actually. It's can I trust the intelligent thing enough to actually hand it the questions? And so let me show you what I mean. I'm going to actually switch into a little chat app that I built, which, by the way, I don't know how many builders we have here. This is all open source. So what I'm about to show you, if I can figure out how to share—

weep.

Let's see, DataChat Mini is what it's called. Here we go.

All right.

So This is just a chat that I was in. We can just— I'll just show you briefly instead of like just hammering it. Although we will do that a little bit. Maybe we can do that at the end. So I said, hey, what is our net profit last year? And it came back with an answer. And this looks pretty good, right? It gives me a little table. It gives me a little buildup. Tells me how it calculated it. And so I said, okay, that's pretty cool. I like those numbers. I'm going to actually just start a new thread and ask the same question. Okay, and so let's go back up here. What was our net profit in 2024? Walk me through how you calculated it. Do do do do do. Okay, so we get a different number. This is awesome, right? So we're now in a situation of not only are these numbers way different, we don't know why they're different. And then, you know, if we can do something really dumb with this, right? Which is like, which of our SKUs are losing money? Well, I'm already on a bad foundation. I know I'm wrong, right? I know my two numbers don't agree with each other. So what happens? You know, am I making the right decisions on this? Who knows? By the way, for this demo, I made a custom kind of like AWS or not AWS, Amazon kind of seller shapes dataset. So some of this stuff may look familiar to you all. So that we could kind of ask questions. So, now we know we're not kind of on a stable ground here, right? So, like, what do we do? Right? Like, what do we do? This is a really interesting question. And I'm going to switch back to my slides for a minute here. We'll do some real chatting in a minute instead of, you know, me having canned ones, but I wanted to get us here. All right, perfect.

Cool.

I want to be really clear, the model is not dumb, right? In fact, I just used a really smart model. It writes better SQL than I do, certainly, and it's much faster. The problem is not that it's intelligent, it's that it's missing context. It doesn't know your business. Right? The first one, as an example, Right. A reimbursement is a credit. Uh, you subtract it. Uh, how did it get handled in our previous chat? I'm actually, let me go look. I'm just curious. Did it handle the reimbursements right? Well, there's not even a line for it. So who knows where it ended up going, but I assume it actually probably added it instead of subtracting it. Who knows? Um, let's see. Uh, what else? Um, a shortage claim in the business is a cost, right? Not a refund. If you read the column name shortage claim, it might sound like it's money coming back to you. The model doesn't know what that means in the context of your business, right? And then of course, cancel orders aren't revenue, right? Hey, you need to filter, you need to filter on orders that are set to canceled so that we don't actually count those as revenue. And then of course, you know, none of these are in our schema or anywhere discoverable. So this is knowledge that we just know because we have been in the business, right? We're doing stuff. And it's stuff that you kind of discover, frankly, some of it is like discovered and hard-won, right? By you presenting a number and someone being like, hey, that's probably wrong. Did you know about this special thing that we put in there? Anyways, so So what I'm going to do now is we're going to start a new thread and we're going to do a little bit of exploration and see if we can start getting the same answer back out of our system, right? So I'm going to switch over here. Do do do do do, back to our thing. Bam, we're in. We'll start a new thread. And so you can see here's the data that I have, by the way. We'll do a quick overview of what's in here. You can see that here's Jacob's E-Com Company, great name. And we've got some tables with columns in them. And they have order items and campaigns and all these, you know, fun things in here. Okay, so the first thing I'm going to do is actually I happen to have, because I built this database, I happen to have some context, I have some real queries that I know are good about this database. Okay. So the first thing I'm going to do here is just say, let's save context based on these 5 good queries. I'm just going to copy paste those in. And so what it's going to do is it's going to parse through these and it's going to actually— we're going to see context start getting associated to our database objects on the right. And we'll see how long this takes to go. Can be, you know, sometimes takes a little bit of time. But like what I want to show you again, this is all open source. This is local. You can run this locally, you can run it in Vercel. It's very easy to run it. You know, this is the type of stuff that we're thinking about. What are the primitives that we need in order to make this stuff all work? Let's see what it's thinking. Let's see. I'm distilling the core logic from the provided good queries. Oh, thank you. That's so nice of you. Great, great, great. So it's doing some work here to take these queries that actually have, you know, all of our joins, all of the things that we really care about from a data perspective. Again, like from a business side, we don't really care about these things.

But like, they're really important in order to get the right, the right information.

Okay, so it saved 9 context fragments in here based on what we just put in what we just loaded in on the side. And so let's look at these. Alright, so telling us how to join the data. That makes sense. What else do we have? Latest SKU, okay, you need to find— you need to sort them on row number to find the latest SKU ID. That makes sense. Right, here's some definitions of ad spend. Net margin, aha. Net margin definition, gross product minus product cost minus Amazon fees minus settlement adjustments. Product cost should also exclude canceled orders. Okay, great, that makes sense. So it's giving us all this information that it's mining from these queries. So now we can ask a question again and let's see what we get. So the answer should be $487K if I did this correctly. I did this right. We'll find out what it says.

My favorite moment in every live stream.

Yeah, yeah.

Do I look amazing or do I look something else? Okay. Okay.

No, this is okay, though. I mean, it says 590, but this came from my script that I had Claude build. So we're going to do a little bit more digging on it and see if this is right. So this is what, you know, this is where we can start really, really seeing if this works. Let's see, back of the napkin. I have, what did I say, $487K as net profit. Can you check your numbers? Let's see what it can do here. One of the things that's really cool about like this Notion here is we can actually do a little bit of self-learning. Right, which is like, as we discover, hey, oh, you know what, I needed to set this other flag, you know, here it's over here. Um, I'm actually not sure my 487 is right. Like I said, I generated this dataset, and so we will, we will see, we will see how much, how much research it kind of can do here and figure it out. So it's, it's exploring, it's like just, it's just poking around trying to figure out how, if it can figure out why, why this is the way that it is. So this is pretty cool. This is what I would do if I was a human, right? I would just be like, all right, let me like fool around here. Let me like change a little bit of these, these parameters. Let's see if this works. Okay, all right. Oh, look at this.

Aha.

It discovered, we learned something. And what's really cool is it did this thing, it wrote new context, right? It said, oh, hang on, I see, I need to exclude returns. That makes sense. So it updated, it updated our net margin definition and then it said, hey, I need to exclude returns. So it learned a little bit from here. So now we're at a 487. Amazing, right? All right, so cool. So we're there. I like that. And again, we can ask these questions.

Okay, really cool.

So now we can start saying, all right, which SKUs are losing money right now? We can actually— now that we've baselined on a number that we know is right, we can start poking around. We can start poking around on this, right? Several SKUs are operating at a loss. Thank you. That's so nice of you to tell me.

Okay.

Felt furniture pads.

Okay.

Yep, yep.

Stop losing fake money, Jacob.

Yeah, exactly. Stop losing fake money. What are you doing? Fixed costs are high ad spend. Um, okay. So now we have a way shorter list and that's different and tighter from before. Um, let's see out of our total products. And then let's see what it says. Um, many are losing money. Um, let's see what it does here. By the way, this is making real SQL queries. This is all, you know, the data is fake, but the harness here is real and something that I, you know, I'm happy to share with everybody. You know, this isn't, this isn't secret magic. This is stuff I want to share with everyone. All right. This is pretty bad. That's pretty funny. All right. This is the fun part here.

Okay.

So the thing that I ended up loading here that really made this work in any cohesive way was two things, right? It's really, it's context, right? And how do we get that context? Well, in this case, you know, I'll just show you. This is the data generated. So we have a data generator that we have at Motherduck. I said, hey, like, make something look like an Amazon seller dataset. I want to be able to answer, you know, questions about net margin. And so I said, okay, great, I'm going to build that for you. Here's some sample queries that you can you can use to hydrate your context. And even actually, it actually corrupts the data slightly, or not corrupts it, but injects an insight into it too. It's very fun. Anyway, so we can take these sample queries. So the first thing is, A, we have this notion of like sample queries, right? So where do those come from in your real business, right? Well, your analysts already know them, or you can figure them out yourself, right? In MotherDuck, we have a feature, you know, like any database called query history. You can find those good queries and throw them into, throw them into an LLM, whether it's Claude. This, in this case, I'm actually using Gemini, but Claude, Gemini, ChatGPT, these all work, right? Just, hey, like, here's a good, here's a good known query. What things about this generalize that we can use again in the future? And we can save that as context. And those are those. There's a bunch of interfaces for that context loop. And the second thing is we know specific fact, right? I said, hey, it's $487K is our net profit. You need to go figure out how that number is right. Right. So this is actually, this is like from, uh, I 100% had this job when I worked in accounting, which was like the CFO was like, hey, this is the number. Like you got to go figure out how all of these pieces fit together. And we can now make, you know, the LLMs do that. And they're very fast and it's very affordable to do that type of work. And it's actually really, really impactful to show you know, how do we get from A to B? Which is really cool. So, we did that. And we were able to enhance our conduct even further, right?

Jacob, we had one question come in specifically about this example. Is like what you're looking at here, you mentioned it's a thing you built, right? Yeah. And then you mentioned that you're going to share some of like how that— what went into that and whatnot. Okay, cool. We had somebody asking about how to do that. And just to confirm, like, they're asking, does that mean like that, you know, anytime you're running stuff here that you're, you know, connected to one of the LLM APIs and burning tokens and stuff like that?

Yeah, absolutely. So yes, I am. This is connected to OpenRouter. OpenRouter is a kind of gives you a generic API that you can use to connect to many different models. In this case, I'm connected to Gemini 3 Flash. I probably spent about 10 cents so far on this, something like that. Not a lot. It's a very good robust model for running data analysis in my experience. Yeah, yeah. So we are spending tokens and we are connected to a database, right? So these queries right here that you're seeing in here, these are actually running against MotherDuck right now, right? So they're just Motherduck has an MCP that queries the MCP and then can get the data back. So, yes, we have a database and we have LLMs. And then this is actually running on Vercel, but it's running on a free account. So, you know, all good. But you can run this locally. There's a million harnesses. I like mine, obviously. But, you know, I want to make this easy for folks to use.

One more question that came in before we keep going, Jacob. Excuse me. Somebody asked anonymously, their company has tried to create similar natural language data query agents and all have had the same issue of needing a whole bunch of business context, which, you know, makes sense. But from your perspective, what have been the actual steps to get that context into production?

Yeah. Okay, great question. Okay, so there's a couple of things. So right now I'm using local context tools to save this. So if you all went to this URL, which you could, and wanted to chat with this, you would not see my context. You would only see context that you injected into it. So on the MotherDuck side, we are actively working on building a feature to do this. What I have seen our customers do is use Claude projects, which lets you kind of inject context. Our thought is that that is a little bit too broad for like most data tasks. And we've also— so what we've seen actually other customers do is literally just like build their own harness that looks a lot like what I have here. Or, you know, I've said, hey, here's mine. And they said, okay, let me inject my own way to do search on context. The other thing I would say is like generally the amount of context you need is not that big. And so, if you can avoid search, that's one of the key insights that I've found. So, what you want to do is structure it in a way that's hierarchical so that the LLM can navigate the graph and find the different parts on it. To always get the right context. The problem with search is that like, you know, at, let's say like an SMB company, like search needs like 2 or 3 engineers probably tuning it to really build something great for your business. And you just can't afford those types of people. They all work at the labs anyway. So like that's all getting injected into Claude and to, and into other models. At least my experience so far has been that like search and then I guess like Related to that is like recommendation. I'm sure that that's probably top of mind for a lot of you too, is like you're thinking about like, what product should I recommend with this product? You know, you're either buying that from somebody else or you are spending a ton of time building your own, your own way for that to work.

Yeah. What I'd say is in at Reason, we've had similar issues and we've mostly overcome them. And the specific things that we've done are, you know, within the context of an MCP, building a lot of the tools and harnesses to Jacob's exact point, the specific tools and harnesses harnesses I think are going to vary by the shape of the context that you're trying to bring in. But for example, the, the data dictionary that I showed in my last session, which is, you know, an extremely predictable form of context and it's easy to search, we brought that in through, you know, an Airtable harness through their API in our MCP, and we gave it instructions in the tool to understand when and how to query it and then a bunch of other shortcutting in order to minimize the amount of searching to Jacob's point the tool needs to do. And then we separately do about different types of harnesses for different types of documentation and context. And again, trying as often as possible to be prescriptive about when and how it should be brought in rather than bringing it into every conversation.

Yeah, that's exactly right. That's exactly right. Yeah. We could talk about— I think there's a whole— that could be its own session. Learnings there. I did write some stuff on the Mother Duck blog about this that you can find about the semantic layer in particular. Happy to share those links, Andrew, after this.

Yeah, I would love it. Personally, I want to read more about it. And then I know our attendees, as soon as you say the word semantic layer, I'm sure a lot of people's eyes just rolled back in their sockets a little bit.

So this is kind of, you know, again, this is a slide just kind of summarizing what we were talking about. Like, how did we pull that? That SQL context into something that's more human readable. But ultimately, this is the thing. These are the 3 parts that you need. And we haven't talked about the last part yet, which is an eval, right? So we talked about context layer, right? Which is, okay, you need some sort of predictable store, whether it's Cloud Projects, whether it's MotherDuck, whether it's something else to get the data. You need an agent that's running against it, using tokens. And then, of course, you need an eval. And when I say, of course, like why? If you think about how accounting works, right? And I'm going to say this because to me, everything is accounting. And so, we have this really cool notion in accounting of like dual entry, right? When something— everything always balances to zero, right? And so, if you zoom out a little bit, what that actually looks like, that's like invariant testing in software, right? It's something that's always true about our, the data, right? And so evals are a way for us to take that same kind of notion and say, well, how do we make sure this is always true? And how do we do this for a system that is not deterministic, right? And Ledger is pretty easy to make deterministic, right? It always has specific rules. We always can say like, yes, this is always balanced. If it's out of balance, the transaction is rejected. Things like that. For an eval, it's a non-deterministic system, right? An answer to a question like, what is net profit, right? Or even more specific, what's net profit by these products in this quarter for this region, right? Ultimately, there could be a whole debate actually around this notion of using cubes and pre-calculating all those things. But I would just say, if those worked, we would all use them. And so there's very practical limitations of kind of cubing your data, unfortunately. Which is really too bad because it would make our lives way easier and then we wouldn't have to deal with any of this. But anyways, eval helps us build that. And so what is an eval? An eval is a way to take your harness or whatever you're using to ask questions and repeatedly, you know, make sure that there's not drift between the answers you care about or the questions you care about and the answers associated to them. And again, I have lots of— I've done lots of writing on them. I have lots of open source stuff that you can steal. They're not cheap to run. You really have to figure out, you know, you're really making the commitment accuracy once you start running evals. But it's not hard to do once you kind of have someone show you how to do it.

Excuse me. All right.

So, you know, just one thing that I want to say here is that like there's a lot of companies that are thinking about this in the space, right?

Here's 3.

Hex is probably the leader in the space. They are an analytics company doing something that is like a— it kind of evolved out of Jupyter Notebooks. I really hate notebooks as a spreadsheet enjoyer. Like notebooks don't— they don't hit for me. But they're doing really cool stuff. They've got the really mature context layer piece. You do— now one thing that's different about their offering is they sell the tokens to you. You buy tokens from them. Motherduck says, here's an MCP. You can just use it with your data. You know, you choose your model. Hex says, you know, we will sell, we have the best model, we're running our own evals, we will let you buy that one. There's a few others. Golden Analytics is one that is just launched, also built on DuckDB like MotherDuck is. And Probably.dev, another one, which is kind of thinking about this space. So there's a bunch of companies thinking about this problem in the space. I think we all kind of see that the wave is coming. So that is where, where we're going, right? Again, you know, determinism is the feature we really care about. We want to lock the question, so lock the context so we can free the question. We want to get that right answer every time. And so this is kind of like the core of what I want to think about. But I have one more thing for everybody. And everything I showed you today was a conversation, right? You ask, it answers, and that's great for finding things, poking around until you hit the answer that matters. But some answers you don't want to find once, you want to get them, you know, every day, right? Um, this last part is, um, about keeping the answers, and this is something that is, uh, We've been thinking about a lot about M. otherduck. And I can show you. So this is something I can show you a little bit differently. Let's just do— let me share my screen here. Here we go.

Okay.

Um, all right, I'm just going to say this. Wait, did I already do it? Let's see. Build me a dive on this. So, um, this is something we call dives, and what dives are is a way for you to do all this really great work where you're honing on a specific set of features that you care about in your data and then turn that into a visual. And so Those of you who are familiar with Claude or ChatGPT, I'm going to stop this while we don't need to share it while it's building. You have probably experienced building these really nice HTML artifacts. Right? They look really awesome. You can save them as PDF and email them around or whatever. The problem is if you want to try to build that later, how do you build it later? Now you say Claude, turn this into a skill. So the next time I ask this question, you use the skill. Okay, great. And then the next time you get a slightly different variation on that thing, right? We're back in the same problem of like, now I'm back in non-determinism land, even though I did have a skill and I won't. But, and, and then the second thing is how do I refresh it? Like, how do I get the same, like, I want the new data, the new number updated on that thing. And so what we've built with Dives is the way for you to kind of connect all of those pieces using the Motherduck MCP. And I want to kind of say like, you know, this is not, I don't want to lean too hard into a product pitch, but this is really What we're seeing is you need some sort of primitive to go from, you know, you have data and you want an answer, and that is context. And then you need to go from that answer to shared knowledge, and that is Dives, right? And you can call it whatever you want. Traditionally, that was like BI, right? Call it whatever you want. We already know what the primitives are. This is just one implementation of it, right? And what I really love about it is What I really love about it is it's like a very natural place for this to fit in. Like, this is, you know, when we talk about like text being the, the universal interface, like we want to be able to ask a question and then like once we see the answer, we're like, oh, that is the thing that I want to be able to see every Monday. Right. Well, I, I, it's funny. I, I, I worked with these guys. Um, actually I know these guys that they're, this is a BI tool called, uh, Evidence. They do really cool stuff. And I was like, well, how'd you guys like get the idea for this thing? And they're like, well, we were working in PE. And, um, we would do, you know, they would buy a company and we would go in there and we would like build a deck for management or to like explain the company. And they're like, you know, most of the time it was like good. The management was like, this is great. You know, love, love the work. Good job, guys. Um, and sometimes it was really good. It was like great. And they'd be like, I need this report every Monday. And they were like, we literally have no, like, how do we build? We can't, we can't give you that every Monday. It is impossible. And so, you know, I think what I'm seeing and what I'm thinking about here with like Dives and being able to use an MCP to build visualizations and to then save them and share them, which of course, like Claude is like, you know, thinking extremely deeply about this in my other window. But, you know, that is I think the natural way that these pieces fit together and how that, you know, we're really thinking about it, right? So like How do you really go from, you know, non-deterministic but intelligent to, uh, something deterministic and reliable and, you know, you need all of those pieces together and, you know, at Motherduck, we're trying to package it together in a way that makes a lot of sense. Of course, you can use lots of other ways to do it. Um, but I think like this way we're going is like inevitable. This is the way it's going forward. And so the real, the real question I think is like, well, okay. So like, what parts do you need? And like who's going to assemble them for you and are you something yourself or like what tools are you using?

Yeah, that's exactly it. I mean, and to your point around like, you know, that MCP-ification of Dives or, you know, repeatable visuals being like the path to future in BI, we have a very, very similar hypothesis here at Reason where, you know, even though we have leaned very heavily into analytics ecosystems through Power BI and other tools of that nature, and I don't see that ever going away. No, I agree. That the ability to be able to create that exact experience you described of, you know, showing your stakeholders and then when the stakeholder finds something that they're like, I need this, this is the thing, capital T, then you can actually go and turn that into a deterministic artifact that can be refreshed, etc., etc.

That's right.

That's right. And that ultimately your BI suite ends up becoming, you know, an assembly of those artifacts together. Um, and then it becomes a different assembly for your next team member and blah, blah, blah, blah, blah, blah, blah.

Yeah, yeah, exactly right. Um, all right, well, it just came back, so let me just, I'll just show this quickly. Um, so, uh, it built, it built a dive. Uh, this is, I was asking questions about, you know, NBA data. I said, you know, is Jalen Brunson better in the playoffs? Um, well, we, we saw it with our own eyes, and if we needed numbers, we now have numbers, right? Um, and so it built this. Um, what's really cool is it inlines it inside of the, the Claude chat, so you can see it here. And then also it gives me, um, a link that if I click on it— where'd it go? Uh, here's my link. And this is actually— if I open this link, this takes me into Motherduck, right? Um, and then I can share that link with anyone. So it doesn't keep that in that trapped inside your chat. Now that anyone who's in my org can also see that same type of information. And of course, as new data comes in, it refreshes too. So that's, that's kind of the story here. You know, find it in a chat, keep it in a dive, you know, build, build the platform that you can use to trust for answers. And I think we— that's what we all are trying to build, you know?

Heck yeah. Amazing. Thank you so much. That was educational even for me having hosted now 6 of these things. So I really, really appreciate you taking the time and especially at the last minute. For those who didn't know, Jacob was not originally part of the scope of this thing, but I got to meet him and he started talking about determinism and I realized, oh yeah, you are our people and people need to hear about the kind of problems that you're thinking about and ways that you think about solving them. With that in mind, we'll hold some time just for any last questions, but otherwise, if you don't have any, thank you very much everybody for attending. We've got 2 more days of these to go, 3 more sessions starting tomorrow.

Tomorrow.

Jacob, thank you very, very much for your time. I'm not seeing any new questions stream in. So thanks again. And we will— oh, never mind. There it is. There's one that came in right as I said that. This is why Gong Sales Coaching tells me you got to wait 7 seconds after you ask a question for people to respond. In this case, Donna asks, how much access have you given Claude in order to share within your organization? It must have some access, right?

Yeah, okay, great question.

So our—

so, so MotherDuck has an MCP, and that MCP inherits the role of your user. Okay, so whatever data your user has access to, you give that to Claude. Now we have an enterprise agreement with Claude, so they say they don't store it and train on our data. I believe— I tend to believe them. So, so that is the way that we've handled this. You know, we also, we also use like the cloud APIs for some internal stuff too. You know, mostly around like log mining stuff on the product side. You know, we're building it, we're building a highly technical product. The logs are complex, understanding them is hard. You know, when you have an incident, you want to be able to say like, you want, you want all, all hands on deck and having an AI assistant back in there is, is amazing just to be like, as I'm using my own intuition to, to Answer questions, that does it too. So, so quite a bit is the answer to that question. You know, I don't know if that answers— like, I'm kind of curious, like, what is the, what is the question behind the question there? Like, is there, is there a concern about sharing it? Is there a concern about something else? Happy to kind of—

yeah, while we wait for Donna to fill in, I do think that an ongoing theme with some of the attendees has been that they are trying to overcome some of those initial barriers with organizational adoption where they are getting really excited about a lot of stuff that we're showing here and they're like, how do I get somebody to say yes to ChatGPT, period, for the enterprise organization? Yeah.

Oh, okay. So, so, okay, this is actually okay. This is great. This is really— I see the follow-up now here in the chat.

So awesome.

Now, now I get to talk my book a little bit. So thank you for letting me do that. So, so this is really interesting. So MotherDuck's security model is different than other databases. So we have something called hypertenancy. And what this means is each user is isolated from each other and data is kind of shared with user, like with users in read-only mode. So when you use MotherDuck, the MotherDuck MCP with, um, with AI, you can only destroy data that you own, not data that's owned by, uh, kind of like the, the ingestion accounts or, you know, the, so like typically what we see customers do is they'll have like, if they're ingesting data, say from Amazon, they'll have like a central kind of pipeline for that. And then users can attach that and we have zero copy cloning. So you just, I attach that data, I get a copy of it locally, basic, not locally, but in my own instance, we call them ducklings. So I get my own duckling and you, you know, they're very cheap to run. You can run them like on our Pulse instances, which are like 40 cents an hour. You can just like have your AI go crazy and write queries and like, you know, do whatever, use temp tables, you know, learn about the data. And like the scope of the impact is very small, kind of as a benefit of our architecture. Now we didn't design it that way on purpose. We were kind of like, we were, how do we make DuckDB work in the cloud? And there's a whole bunch of other stuff I could talk about that from a technical perspective. But one of the ways we ended up doing that is with this notion of hypertendency where each user gets their own database. And so like that type of scale actually makes it very safe to scale it from an analytics perspective. So yeah, I totally, but I think the story, like the story of like, oh, like, you know, DBs being wiped and stuff like that, like totally real, right? Especially in systems where like, I guess what I would say is like there's no, there's no room for error on the security side, right? And so, like, you need to think very carefully about what the models are, and our architecture fits in really nicely there, right? If you forget to do a grant, there's no such thing as forgetting to do a grant in MotherDuck, whereas in Postgres or Snowflake or whatever, you forget a grant, now someone sees everything or they see nothing, and away you go. So it's just a different model. We definitely, happy that things have evolved the way they have because it's a benefit to us.

Yeah, I hear you. Accidentally building the right technology for the future is, is a theme at Reason as well. We'd love to take all the credit for the foresight, but sometimes the world just works the way it's supposed to. Perfect. All right. Well, I think that's it for questions. Jacob, thank you again. For anybody who has more questions about hypertendency, which is a freaking cool concept, as well as determinism versus non-determinism, etc., Please do follow up with Jacob. His LinkedIn is on our site, but also, yeah, you can find him. He's not that hard to find being a developer advocate. And we'll look forward to seeing you for the rest of our sessions this week. Thank you again, everybody. Cool.

Thanks, everybody.

Take care.