Workshop: Working with Relational Databases in the Wolfram Language

Hi, my name is Carlo Barbieri. I work in the Engine Connectivity Engineering group. And this is a workshop on databases. So, yeah, relational databases, if you want to follow along and evaluate the stuff. Unlike the other one that had a server running on my machine, this one is stuff that you can evaluate on your machine. Because we're not going to connect to an external database, but we're going to use an SQLite example database that's present in the layout in example data. So if you want to follow along, it's not going to be as interactive as the nice workshop on external evaluate that Ricardo and Todd did yesterday, but it's going to be more lecture style and less exercise style. So first I'm going to talk about the philosophy of the way we approached connecting to databases in the Wolfram language. And then I'm gonna go into what classes of entities are inspection, querying, relations. I don't know if those things speak to you, but they will after I get in the details. So what is the philosophy of this? So object oriented programming languages have things called ORMs, which are an object relational. M is. No, what is it? Model. Okay, we don't really have objects in the Wolfram language, or rather they're not the most native way to think about this stuff. So what do we have? When we started working on this on Connected to SQL, we already had a precedent for an entity property framework. And what an entity property framework does is that basically there is this resolver function called entity value. And it takes the first argument. And the first argument are classes of entities or lists of entities or single entities, and they represent the rows of the data. So if you imagine a table, these are rows in the table. And the first argument is in general an object that is fully inert and symbolic and represents what you want to get out of a given table and what table you want to get it out of. And then the second argument, the properties are what is called in relational algebra, the projection. So what columns you want out of that table? So these two things form a kind of the underlying algebraic structure of rational algebra. And they're exposed in a way that is slightly different in an entity relational model. So the principles are. And this part of the principle is more Mathematica, like before you call entity value, anything is inert and symbolic. You can do metaprogramming with it. You can do all sorts of stuff. You can construct it like Lego blocks. And only when you call entity value, things happen. Then in the database world, we consider entity value dangerous because it's like, sorry, entity Value entity list dangerous, because reasoning about single entities is something you want to avoid because it leads to inefficiencies. And so in general what people do is like, oh, I want all the countries in G8, let me do entity list so that I get the actual countries. And then after they've done that, they ask for the properties of that list of entities. That thing is really not necessary because you can pass the symbolic representation of the countries in Europe and basically each single call to entity value. And in a sense entity list is a very special call to Entity value is a single query. So if you do the thing in two steps, calling entity list first and entity value after, I mean, this is something that really doesn't happen when you're dealing with the built in tantalities like countries, because they change very slowly. But if you're in a situation where you're talking to a database which has customer orders and they come 102nd, 100 new orders that then get resolved, atomicity is very important. And so doing this two step process means that maybe first time selecting which order orders are currently active and then getting the properties of them. You do that, it got out of sync. And some orders are no longer active because they've been dispatched in the meantime, in the millisecond that it took for going from one call to the other. So try to avoid entity list. Try to do everything in a single call. We'll get more in detail of how to achieve that, to construct things first symbolically and then boom, executed in a single go. Maybe if I remove my shoes. Okay, so what are the classes of entities that we have? So entity class is the one most of you will already be familiar with, which supports. Well, there are either named entity classes like G8 or entity classes support basic filtering and basic sorting. So in general they use predicates like, you know, you can say entity class, country, population goes to greater than a billion and then you get India and China out of it. Or you can say, you know, you can ask for things to be sorted by some parameter and you can get a mix of those things. And I never quite remember whether putting multiple condition ors them or ands them. But you can only do one of the two. I think it's. And so it gets more restrictive if you put more filters. If, on the other hand, you want better control on the logical operators, when you do filtering, you can use filtered entity class which takes an entity function as the second argument in the entity function, you can pretty much put whatever Boolean logic you want and we will compile that for you to SQL. And when I say pretty much anything, it's because we are actually using the symbolic capabilities of Mathematica and so you can, you know, there's extensive stuff like Boolean function with four arguments, number 15 and it's like based from truth value and that gets compiled into n's and ors by using conjunctive normal form. So we do really rely on a lot of symbolic transformation that are in mathematics and that wouldn't be otherwise supported in SQL. So instead extended entity class is something that used to add a kind of a virtual property to a table. So it's like an annotation. Suppose you have countries with population and area, surface area. Then you can compute the. You can compute the density and you do it once and you can then reuse it in subsequent queries that are wrapped around this first query. Sampled entity class. It's like something that you use. It's what in SQL is called limit and offset. It's basically the same syntax as take in the Wolfram language. So it's used to get the first few things out of a query. The difference why we didn't, the reason why we didn't call it something that resemble take is that SQL doesn't really give you any guarantee of reproducible order unless you explicitly sort. So keep that in mind that subsequent queries that look identical if the data has changed, there's no guarantee that the server will give you the same thing twice. So it's a sample though another random one, sorted entity classes order by. It's like pretty trivial. And then we have aggregations. Aggregations are things where you say it's a little bit like group by the Wolfram language, except group by is hierarchical and so aggregate. So this aggregation keeps things flat. So it stays comes in as an entity class and comes out as an entity class. And this is pretty much the principle of all of these functions. They're like Lego bricks. They are homeomorphisms in the space of tables. So most of them take one entity class and return an entity class. I mean, they don't return, they represent because they are inert constructs, they don't evaluate. And the last one, which is probably the most complicated is combined entity class. I mean, it's not the last. Why did I say the last? But it's the most complicated because it's the only one that takes two entity classes and does what is called join in SQL or join across in the Wolfram language. And it does basically various version of outer products of all the properties of one and the other. So it's a powerful tool, but it's to be used with a little bit of knowledge what you're doing. And then we have three more entity classes that correspond to the set operations in the Wolfram language and that have like, they have this thing called same set properties, which is like, just like union, intersect, intersection and complement have the same test. Here we actually define properties so that you can say what are the properties that I have to look at to consider these two things equal? And by default this will use the canonical name or the primary key. But in some cases it doesn't work because maybe you are union in two things, which have entity. Sorry, which have canonical names that are not comparable, because maybe one is strings and the other is integers. And so what does it even mean to compare them? But you can say, I want to use another property to do the union of these or to do the intersection. So these are extremely powerful. You can also, like. One thing that many people don't realize is like, union can take like the same. In the Wolfram language, union can take one argument and use to delete duplicates. It's a sorted version of duplicates, or can take multiple arguments. And it's basically you're doing one operation, you're doing a join and then a delete duplicates and then a sort. So it's like it's the same here. By using clever tricks, you can use this as delete duplicates by. Because you use the same set properties to decide what you're going to consider the equality function. So this is the zoo of functions that you have to compose your queries. And it's not exactly like using select or using Join or using delete duplicates, but there is some parallelism. And the thing that makes it really different is that you iteratively build on things that are fully symbolic and only execute at the end. So it's a little bit less of an interactive and experimental thing, which is a good thing, because sometimes you have Data sets that are 1 TB, and until you've drilled down and selected things, you really don't want to bring things into memory. To figure out whether they work, though, we give you tools like Sampled Entity Class so you can experiment a little bit. So you take the first 10 rows and see if everything is good. And only then when you convince that you've built the right thing, then you execute the full thing. Are there questions so far? No, no, no. This is. So the question is whether WolframAlpha uses this. And the answer is, well, this came after WolframAlpha. So basically if you're using this on a built like on the built in entity types, some of this will work server side, some others will work kernel side. So you will first have to bring all the data in and then do the computation locally, which is very slow in some cases. If you're not happy about it, the person to bug is Jeremy. On the other hand, if you're using your own SQL server somewhere or an SQLite file, it's going to all happen out of Core. And so in that case it's. So Alpha doesn't support all of this. This is much more useful if you have your own SQL database. Okay, okay, onward. Shocked again. I said this before, they're like Lego, they all have the same type, so all of them are composable. With all of the others they're inert and symbolic. So you can do metaprogramming. It's if that's what you're into. And the other advantage that will maybe become a little bit clearer later is that you can, by just looking at the symbolic structure, you can reason about what entity properties they have. So for example, if you look at extended entity class, Extended entity class, what it does is that it adds one or more properties to the entity class that comes in its first argument. And so basically once you know what the first argument has in terms of properties, then you can recursively tell what properties were added at each step of this big onion of queries. And so in general what happens is that you have performed an inspection, which is next topic we're going to cover. So you know, each database table has a given set of properties and to know what properties will come out of very complicated query, you just need to statically analyze the, the entity class structure. And you don't need to perform any query to just know the properties. So this is a very big advantage of in general, the Wolfram language kind of functional symbolic recursive kind of programming. Ouch. I think I'm going to break my computer at some point. Static electricity. Okay, so how does inspection work? So this is, I mean we're connecting now to an SQLite file, but we do support Postgres, MySQL and MariaDB, Microsoft Server, Oracle, and I hope I didn't forget any of them, but the major vendors and basically you get a nice box structure where you can drill down and get the stuff or you can actually inspect it programmatically. And so you have top level properties like properties so you can get all the tables. And once you've performed the inspection, this is like you don't need to re query the database to get the stuff, it's fully contained in the object. So once you get the tables, then you can get table level properties, which are columns, the foreign keys, the primary key. So this object we're still in database land, we're still in SQL language kind of stuff. We haven't translated it to the entity world. This is a very low level thing. So you can get the columns and you can get properties at the column level like this. So you can know what the type is. You can know if it has a default, if it's nullable, if it has any index and stuff like that. So this lets you peek into all the really most horrible details of the database. But you don't need to worry about that because once you fit this thing into an entity store, we dress up this thing with a lot of meta information. Just to give you an idea. I open this up so that you can look at. Yeah, so that you can look at all the. At this nice box structure and things you will notice is that there is a bold property that's product hood. Bold means, I think it's written down here. But bold means canonical name. So those are what was the primary key in the database gets promoted to the, to the canonical name. And things that are underlined are virtual properties that don't really exist per se on the database and are called relations, which is something that we're gonna talk about later. But in general, if you have the arrow that's going to the right means this is a foreign key to another table. And if the arrow is coming to the left it means that the other table has a foreign key pointing to me. And what this. So many of you are like, what foreign key? What is that? But if you're used to Wolfram language, foreign key pointing to another table gets represented as an entity of another type or at the same of the same type if the foreign key points to yourself. And an incoming foreign key on the other hand represents an entity class. So many objects. So I think that this is like we will see later why this is quite elegant and it allows for very powerful querying that would otherwise require you to reason about joins. Okay, so if we register the store, then we can start performing queries on it. So for example, entity value of offices. Office code is like you take the first argument is an entity class. The second argument is a single property. And so this has the shape of a vector and not of A matrix, because we have only one property. But if you put more than one property, you can get a more complicated structure. In this case we're getting. So we want to know where these offices are. And some of them are in the us, some of them are in other countries port but which are the ones that are in the us so now we're putting a filter in the entity class at the beginning and then just getting the city property out. Am I going too fast? Is this clear so far? This is very simple, like trying to give the stupidest examples I can think of. And there is stuff like, for example, this is interesting, like, so we're taking some of the offices are in a country that has states and so we can test whether the state is null or missing. But why? The reason why I put this example here is because I wanted to show you that entity function that we've seen as the function that's used as the filter acts as a property here. And so the cool thing about this is that you don't have to do anything. You just put an entity function in the list of properties and you get like a property that's computed on the fly. Again, this is all stuff that is done packaged in a single big query and then it comes back. So there is only one network request every time onward querying. So I told you this is like Lego bricks. I'm going to assure you that it is the case. So here we want to know how many customers per cgm per country we have. I guess I don't know if that in this database it makes sense to if there's Paris, Texas and Paris, us but just to be safe, I'm aggregating on both of these properties and I'm getting the counts. No, I'm actually not doing that because I just constructed inertia thing. This aggro. If I remove this semicolon, you look at it, it didn't evaluate, it didn't do anything, it's just inert. I now pass it to entity value and I get out account of all the customers that this company has per city and per country. But it's not very useful like this because it's completely unsorted. And so what I do is I take the same query, I wrap sorted entity class around it and I had descending because I want to have the biggest one first and that's not the default, just like in the Wolfram language. And now I can get two biggest places where we have customers is Madrid and New York City. So this is like super simple example of how these things fit into one another and you construct a query that you then execute only once. Questions? That's an excellent question. Yes, and it's the next slide. So yeah, there is this trick which is not very widely publicized, which is like you have this hidden variable. Take a picture. I mean you can download the notebook so you'll have it. You can put any function there. So if you want to log to file or you want to use print instead of echo, we don't care, do what you want. But yeah, this is the SQL that gets rendered and you can see basically if something you don't agree with and you want to do differently. And I mean right now this is using SQLite. If I were to perform the same query on a completely different backend, you will see translated SQL to that backend. And that's honestly like a lot of work that's gone into that, not just by us, because we are using SQLAlchemy, but we have to make a lot of modifications to SQL alchemy to make it work smoothly with the Wolfram language. And as a matter of fact might as well speak about this a little bit. Like behind the way that the entity function gets turned into SQL, there is a type aware compiler that basically recursively gets down into the the expression and figures out whether how to do things and whether the casts are necessary because over language is untyped. And so if you want to multiply two strings you can, but you definitely cannot do that in SQL. And so like for example, here there's another thing where like there is something. Okay, so this is an example that runs on SQLite. SQLite doesn't natively have dates, so we're storing dates as unique timestamps. And now we want to do something which is pretty easy to do in Wolfram language, which is date value and get the weekday name. And so we want to know what customers have performed orders on a Sunday, which is like we want to get rid of them because why would you work on Sundays? Like you're probably going to be an annoying customer. Or maybe it's a way to figure out whether the data in this database was synthetic or not. Because so anyway, look at this monster. It gets turned into a case when expression there is like, well, there's some weird stuff like 00 + which I guess it's used to get an automated cast to real, but then there's another cast to integer. Yeah and then in the end equals seven and then you get this whole thing there. So sometimes even something that is kind of simple looking gets turned into a very big query. And we have things that dispatch like for example, just to give like a super trivial example in the Wolfram language, people are used to the fact that if you do three divided by two, they're both integers. It gets represented as an exact rational. Obviously SQL doesn't do that. If you do three divided by two, you get one because it does integer division. We are assuming that people don't want to do integer division like that. If they wanted to do that, they would use the function quotient. So for integer division there is, for division the divisions symbol. There is a number of things that we're doing depending on what the types are. And you can divide even some. Some databases support intervals which are like for us would be a quantity of type time. So you can do various operations on those. So there is a different code path. So for when you're working with that stuff and then when you get to integers, we're saying wait, wait, wait, wait, wait. This is not a quotient, this is an accrual division. So we have first to turn them into reals, which is probably the closest thing we have to rationals. And so it gets dispatched in all different ways. So a huge amount of work went into this. And if you think you can do better, well, maybe you probably can for a single thing. But it gets a lot of stuff out of the way. Okay, so yes, no, I didn't say that. I said the Wolfram language is always the same. The SQL that comes out is for SQLite. If you try to turn to run this on PostgreSQL, it will not work. It's a different dialect of SQL. So different database vendors have different dialects of SQL, different feature support. So for example, SQLite doesn't really have types and other like Postgres, MySQL et cetera have many types that do slightly different things from one another. Like the dates are not the same everywhere or there's like there are things like there are things that, for example, we wanted to support Windows function window functions, but unfortunately not all backend support them. And it's kind of difficult to express. In the Wolf language we don't really have an equivalent. And so sometimes we got stuck with the fact that something was not supported everywhere. And so we. It was difficult to support for us. But in other cases they had slightly like. You have no idea how complicated it was to implement round because everyone did round differently, these database vendors. So to have something consistent and consistent to what you expect from the Wolfram language, we had to write code paths that were different for all the different vendors. The answer. So I repeat the question. The question is, if I update my database and. And there is a feature like the fix a bug and I was relying on that bug and. Or they had a feature that wasn't there, it's like breaking backwards compatibility, what happens here? So I think that to an extent we are robust to that because we are using SQLAlchemy, which is very widely adopted. And so in the future we will Update versions of SQLAlchemy and so support for newer versions will come from that. If there is an egregious bug, we hope to get it in testing. We have our wonderful QA person dev is on it. And if a major new version of something comes out and there's breaking changes, we hope to catch it with tests and then release an updated version of this. But of course you can't really completely automate that. So there is, I think it's also the responsibility, like if you update your database before you do that, please stage it a little bit and test that it's not breaking anything. Also because maybe you are running an older version of Mathematica, who knows? Yes, That's what the extended entity class does. You give it a name. But on the other hand, since you have no right access to the database that is going to be like. It's going to be only available within that query, right? So like, and this is like. Okay, let me, let me give you an example. So you see, we've done this thing where we have missing queue of state. Okay, so what I'm going to do now, I'm going to rewrite this query with extended entity class of offices. Then we take and we say is missing and we use this as a new property and we put in this entity function. Okay? So now this is the Office table that has grown a new. A new column that is missing. And then I can go around it and say filtered entity class of. I want to see if this works. I'm feeling is missing. So without addressing it with the entity function, it should work. I'm not entirely sure. And then you can do something like ntt see how many. No, yeah, didn't work. So maybe this has to be entity function of O. So if you look at how it. You know, the thing is that Leonid is too clever for you. So there was an inner query that was using is missing as a named property, but it just got optimized away because it was a contrived example. So we actually constructed it in the intermediate thing, but it got optimized away. If you actually do want to see it, you have to do something like this. This. Well, yeah, you see as is missing is the thing that's constructed the new named property. So yes, it can be done. And that's what extended NT class. And it's useful because sometimes it's like. It's basically like when you're doing Wolfram language programming, you sometimes say with and like put a variable somewhere that you're going to reuse in a bunch of places and you want to compute only once. This is the same thing, but inside out. Because you have to put it inside for it to be available outside. So you have to kind of reason on the opposite way. If is missing is present in the inside queries, then. I got a message in the chat. Oh, Bernie says thanks. He's watching us. Okay, so yeah, can I move forward? Okay, I'm going to turn off the logger here for a second so that it doesn't. Okay, so yeah, this is my favorite part. Christopher, you asked me a question about this the other day. So. Ok, so let's look at. So this is like. This is a cool example because it's taking completely unrelated things in Mathematica and make them work together. So this is a real. An actual graph. You can compute with this. Like it's the head is graph. You can get the edges and the nodes and everything. And it's the structure of the database in terms of foreign keys. And so basically you can see that for example order details, points to products and points to orders. You can see the payments, points to customers. You can see the employee has foreign key that points to itself and that's the reports too. So basically who's your manager? So you can do stuff like this. That is you can look at it like this and as I said, you can look at it this way as well and look at these arrows in the boxes of the entity store. But basically what happens is that you can do entity value of office code from the employees table. And this is just the naked column and this is the foreign key in SQL. But if you do offices, then you just are getting the foreign keys dressed up with the head entity and the correct type, which is offices. So it's constructing this object that is super useful. But if you go from offices to employees, so you want to know. So the question of this is like for each employee, what office do they work from? The opposite question is like for each office, who are the employees who work from that office? And you do this and you get an implicit entity class that says, give me all the employees with office code equals to one. But the really cool thing about this is that you can actually change this. Since each of these is an entity or an entity class, you do something like this and you can construct something that says. So you're getting the first and last name of an employee. So you start from a customer, you get out the employee that is the sale representative of that customer, and then you ask what is the manager of the employee? And what's the first and last name? And so you construct this big thing that otherwise would have required two joins because you're getting two related tables. And so I think it's kind of interesting to look at the difference in terms of SQL that's being generated. I think this is more intuitive for people who are used to functional programming. But it's this monster with a bunch of subqueries. And if you look at this one, which is the equivalent with combined entity class, so this has one drawing here and another join here. So it's just two different ways of reasoning about it. I tend to find the one with relation to be a little bit more intuitive. But you have both. You can do both ways. And another thing that is really cool is that you can actually follow also the reverse relations, which would normally lead to something that is forbidden in SQL because you get stuff that is, you know, you cannot return a vector as a value for a column. And what you get here is basically so you, okay, this is. What is this? You start from one order, the order number 10,100, and you take. So each order is related to a table which has the order details, which are like the lines in the order. And for each line you have the price for the item and the quantity that's ordered. And so you're like doing, multiplying these two vectors element by element, the price and the quantity, and then they're computing the total. So you're effectively performing an aggregation without doing the aggregation, because you're doing it with this subquery, basically. So it's like, again, if you wanted to do this normal SQL, you would have gone on the order detail table, you would have done a group by one group buy, and then a multiplication and then a total. This is like much simpler way to reason about it. And I think it's one of the key things where you don't want to use SQL here directly because it takes much longer to think about, at least unless you're doing SQL every day in your life and you Think like SQL. Are there questions about this part? Yes, it will not know it for you. So I'm going to repeat a bit of the question. The question is like, it is common sometimes to have details and, or pre aggregated tables because otherwise you get too computationally expensive to do all the aggregations on the fly. The problem is that a lot of that logic is like your business logic. And there is no metadata that we can read that tells us that this table is the actual materialized aggregation of this other table. I think that there are features of SQL like there is this thing called the materialist view, which keeps the meta information about how the view was created. We don't support it yet, but if you were to do things in a little bit different, there might be a chance that we can read that meta information. But that would also require being able to parse SQL and turn it back into this expression, which is yet another can of worms. Because one thing is generating SQL. Parsing SQL from five different vendors and turn it into, back into entities is a lot more complicated as a computational problem. Yes, Brad, that's an excellent question. So Brad asks, what are the advantages of using SQL instead of an association? And I would like to answer with a quote from Star Wars. This is a weapon from a much more civilized, civilized age. So SQL is like, you really can't compare the two because an association is bound to always be in memory. And so it's limited by the memory of your machine. Of course, if your data is small enough, the association is much more powerful because you have all the rich structures of the Wolfram language. So if you go look in the page for new int12, there is an example where I'm using a data set that is larger than a terabyte and we're performing queries in less than 10 seconds on a data set that is 1 terabyte large. Because SQL is optimized for that. It's. The other thing is like. So this is one, one case, like stuff that you wouldn't literally be able to fit in the memory. And the other case is atomicity and concurrency. You have a website where there's 100 people per second coming and making changes to a state. Think about, I don't know, think about Airbnb and people making reservation to places. That's like a giant database table with the reservation of people. And each time somebody connects to their website, when they pay and make the reservation, there's something that changes. And you have to make sure to update the state of the apartment so that nobody rents it. At the same time, this type of concurrency is something that is very hard to do with something like an association and that SQL has now almost 50 years of history of being able to do that. So. Yes, but it's. You're not. So I repeat the question. The question is if you have a parallel algorithm that shares the memory and searching for something, sure you can use an association, but not the reason people use SQL are often situations where the data is also changing while you do that. And so in general, I mean we're. Roman is working on that stuff. I don't know if he has talked about it this year, but there is stuff like locks so that if you're using a parallel algorithm you can make sure that the operation that you perform are atomic. But in general, again, SQL is designed. Not all SQLs are designed for like data science kind of workflows. Some dialects of SQL are better at that. But in general what SQL is very good at is like concurrent atomic access. You can do things like sharding. You can put like half a database on one server, half of the database in another server. You can have replication. It's very, very much of an industrial thing compared to an association. Association is fine, it's much faster if the data is small and you're the only person using it. You can definitely do things with an association, but don't run a website of an association, at least not one that has many people visiting at the same time. Does that answer your question? Yes. So it's an excellent question. So the question is what do you have for profiling? If I can summarize. So there are things that look surprisingly similar in SQL, but one is terribly efficient, one is super fast. So do we have tooling for that? A little in that in the relation when you perform the inspection, you can get meta information about whether or not there are indices. It will list those for you. On the other hand, we haven't yet. I guess if you go like convince Stephen that it's worth spending time on it. We haven't found ways of exposing things like Explain also because not all backends support them. So for those who don't know in SQL there's this thing called Explain that is going to kind of do a dry run of a query and tell you what parts can give performance problems. But exposing that in a way that is user friendly and symbolic across different vendors is kind of a big project and we haven't tackled it yet. But if there is enough adoption, maybe they'll give us. I mean it's Something I would love to work on. I think it's very interesting, but I don't know. And also maybe automatically using that kind of information to change the query in a way or in another. So far this stuff is still at somewhat an experimental level. I think there are features that are more important than that, such as writing to databases or using a different mental model for. So there is. I don't know, I'll talk about this, but if you tell someone else, I'll have to kill you. There is currently an internal effort in trying to see if it's possible, for example to compile things that are written for data set into SQL queries. It looks like in many cases it's possible and that is like very algebraically complicated project because dataset is inherently hierarchical whereas SQL keeps, all. Keeps this nice stable structure. And so I think there are more people who want that than people who need something like that or would be able to read it when it came out. So maybe it's not at the top of our priority list, but it's a project I would like to. Would really love to work on. I don't know if we'll get a chance to do it. Yeah. So I'm going to move to. What do you do if there's something that. Something that we don't support and that you need? So we have this thing that's called embedded SQL. Well, it's actually two things, not one. There is an embedded SQL expression, which is an expression, so it works. Maybe some of you remember embedded HTML. That's something that we built for cloud when you wanted to have verbasim HTML in your cloud stuff. This is kind of the same. Like there are some things, for example, this date functionality, SQLite is very weird because it doesn't support dates as objects, but it has probably the best stuff for computation with dates of all the other backends. So in this case we start from now, then we go to the beginning of the year, then we go to the month of June and then we take the first Sunday of June and we want to know what that is. And this is something you can do by writing something like that in SQLite. So we do this and we get. Now it's now six months, it's July. So the third of July was the first Sunday of July. And it's, you know, in this case it's like just inserting the SQL verbatim. And this is a kind of a stupid example because it's like it's just putting a string into a function. But the cool thing is that this supports binding from the rest of the query and so you can inject things in it as if it was a template. And so for example, in this case I'm using this glob pattern matching for strings to get to figure out what countries so glob the native glob in SQLite since they don't have Booleans returns one and zeros. But so I want to know what countries start with a capital U. And I can write the pattern this way and you get out like this nice little list. And as you can see, it's smart enough to know how to insert the right thing in the right place. So we went from something that is addressed as a property and inserted it into the template as the named column. So I think this is kind of nice. So this is for expression level things, so things that go into an entity function in general, but you can also have things that go at the query level. So something that we do not support out of the box is with recursive. And this is the way you write a factorial in SQL. So it's like iteratively constructing this recursive table and then it gives you the factorial. If you give to higher numbers, you see that SQLite is not Mathematica and at some point you just break out of integers and get reals. But such is the limitation of SQL. So this thing like recursive queries is again a very powerful tool that is not supported everywhere. And it's kind of difficult to expose natively in the warframe language, but it's useful to do things in a sense, we do expose it in SparQL because SparQL uses this kind of stuff to do the. I think it's called the transitive closure. So if you're asking question like, instead of asking the question like who's your manager? You're asking like, who's your manager? Manager, manager, manager. Until you get to the point of a person that doesn't have any managers, or you want to know the list of all the person that are under me in a tree, that's the kind of stuff you can do only with rehearsal like this. Okay, but if you suppose you don't want to construct a query that can then be fit into another query, because the whole thing about this is that once you have this expression, this is another Lego brick that has properties that you can use. And so if you wanted to then call filter density class on this, you can, if you want to call you, you know, you want to make a join with another table. This works just like any other entity class. So everything is composable here. And again, it also supports feeding into it template slots from data that comes from other queries. But if you want completely abstract, completely arbitrary stuff, there is external evaluate that. And this ties in with the workshop that was yesterday. And here I'm just constructing. So if you do SQLite of nothing, this is in a memory database. I create a table of developers. There's a name, there's whether or not they're awesome. I insert two values. I am awesome, Donald is not awesome. And then I'm selecting the ones who are awesome. And you get out this nice little data set. So this is like entirely arbitrary stuff. So if you are an expert on SQL, but you still want to get your data into a notebook, this is probably the best way for you to do it. Because you get the advantage of like a nice smooth conversion into the native types of Mathematica with the transport that is as optimized as possible. So we try to keep things packed in transit and so on, even if there are nulls. So we reinsert the nulls afterwards. But you can write any SQL you want and you can actually also get it. Isn't there like the cell thing? Yeah. So you can just do this, go to SQL and then you have a thing where you can type select, select star. I don't know what database this is connected to right now. So select two. Yeah, I don't have it registered. That's why I didn't want to evaluate it. So greater than questions. Not yet stored procedure was the question. No, not yet. Okay, thank you.