How to Get Structured Outputs with Ollama [Pydantic Models]

Ollama now supports structured outputs, making it possible for you to constrain a model's output that you generate, maybe a response from, to a specific format defined by JSON schema. So what exactly does this mean for your day-to-day development, especially using local models from Ollama, and how can you get started on it today? So in this video, I'm going to go over that, and let's get straight to it. Blog that was released announcing this new changes that they added to Ollama, which is really cool. And this was released on December 6th. So, and as I mentioned here, it really helps you constrain the output that comes from the model. So essentially, typically when you chat with a model, it gives you output usually that's not super structured, or sometimes it is structured, sometimes it's not. But maybe you just want it to be really organized in a specific format. You can use this Pydantic models and the JSON output to actually make sure that the output's exactly the way you want it all the Each time you interact with it, for example. And some of the use cases here of why you should care even trying using this is parsing data from documents. So if you have a document like a PDF or even images and you want to extract specific information from that image, then you want it structured and maybe it's receipts and you want all the receipts to be in JSON, for example, and strict JSON format. You can define these models using Pydantic. That actually parses through the document. It could be, for example, using the vision model like Llama 3.2 vision model, looks at the image, maybe a receipt, and extracts all the details from it and verifies using your Pydantic-defined models already and makes sure that the data corresponds to that and outputs it in JSON format. So it's really helpful. I've tried it a lot with AI agents and it works amazing. Real quick, let's just get to the code and actually you can see a full example of how this works. So in my code here, I have, uh, 3 files which I'm using it differently as examples. The first one is country, which is more of like a basic example that we'll go through and how to do this. And then we have one for art critic. So given, uh, you know, a few art pieces, uh, images, it should go ahead and do some critiquing on that, on those images, and give us a full critique. So we'll define what the models or Pydantic models would look like, especially what we need it to look. So we'll need a vision model to do this critiquing for us. And then we'll finally get a full report on the critique. So first of all, let's start with the country.py file. So here, just walking through this example. So I'm just going to make my page a little bigger so you can see it on your side. So here we're importing Pydantic. So from Pydantic, we're importing the BaseModel. Which is what we will need to create the Pydantic models. Then also from Ollama, we're importing chat. This will allow us to chat with the model. Here on step 1, we have defining the Pydantic model that we need to use here. Country, here we'll pass in the name of a country just like the one they had with Canada. But when we pass a specific country name, we should extract the name, it should have the capital, it should have the official languages, The currency, the population, the continent, the borders, which is essentially the neighbors for that country, the time zones that are available within that country, and also the region where that country is located at. So pretty interesting information there from our model. So that's our entire model right there. And now when we come to the chat portion, which is step 2, we would need to send the chat request with the defined schema that we just defined up here to Ollama to be able to get the response. Here I'm just using the walrus sign here from Python instead of creating a variable for country name and then passing it over. It's just one step run there. So if country name, then the input of the country name, which will prompt you to enter the country name in the terminal when we run it. If this is a country name, then it should pass that to this function here with a chat. And we're using Llama 3.2, which I have locally. And then the messages here, role user. And then the content here, I'm just saying, tell me about the country name. So you pass in the country name, it should tell you more about that country. And then we just on the format section here, of course we're passing in our model, Pydantic model that we just defined up here. We're passing that here so that way when it generates the output, it should also make sure that it conforms to the model that we've defined. We want it to structure the data according to the way we've defined our model. So step 3 here will be printing out whatever it's gone ahead and extracted from that country and actually parsed it the way we wanted to. So we should print that out on the terminal. And then what I'm doing here is an extra step. I'm just creating a folder output, and I want to save, uh, that file in there for each country that I generate so that way I can go look over it, uh, afterwards. So let's just go ahead and run this. I'm going to open my terminal here and I'm going to do python And then we're going to run in the country, country_file.py. So we hit that run. So it's already asking me enter country name. So I'm going to enter Kenya, which is a country that I'm originally from. And so this will take a minute here because the model might need to warm up first, uh, or boot up. But it's done now, so that was pretty quick. So here we have country name, as you can see, structured it. Kenya, Nairobi is the capital city, official language English and Swahili, currency Kenyan shillings. Population 52,000— 52,800,000. This seems very wrong because it can't be just that accurate with all those zeros, but it's a good approximation. I think it's about 54. Correct me if I'm wrong, is somebody from Kenya. But continent Africa, borders Ethiopia, Sudan, South Sudan. Well, that's not all the countries, but I mean, you got a few right. The model is hallucinating a little bit, but at least it's putting it in the structure that we need. To put it in. And then time zone there, East African time zone, East African time zone DST, and then region East Africa. So that should be correct, um, as the model could be. So here let's look at the output, which is already in here. And since it's not structured the way we need to, it's not formatted desired. So we just Option+Shift+F if you're on a Mac, and it should show you all the data now structured correctly in JSON. So you'll see the JSON here, and this is how it outputs the data, and this is what we want. So if you want to use this data afterwards, like, it's already structured the way you need it. So always output the way you need it, and this is the, the power of having that Pydantic model. So pretty cool, pretty quick. Now that's done. So let's go ahead and delete this because we'll need that— you will use that folder again. So let's come here to the art critic, uh, example that I have here. This is a little more advanced compared to the one that we just did. So for this, it analyzes images. So let's look at the regular stuff that we're importing here. So we're importing the typing, of course, there because you want to enforce some typing enforcement in it, which we'll use in our Pydantic models here. And then of course we're still importing the base model here from Pydantic to help us create these Pydantic models and then Ollama and then argument parse here. And this is just a command line to allow us to enter stuff within the command line. And then the OS, which will allow us to save files within our system. And this is the specific use case we're using this for. So our first pedantic model here is composition. And so I'm not an art person, or I'm not an artist, let me say that. So if you're an artist out there, this is the best I could come up with, with the help of Sonnet, which did help me a lot in kind of figuring out what composition is, what artistic techniques I should be looking for in terms of basically critiquing the art that I give it. Uh, essentially I wanted to grade it based on some of these features. So first of all is composition. So we have dominant lines— horizontal, vertical, diagonal— and then focal points, balance type, symmetrical, depth technique. And then artistic technique, which is a little different, which is medium brush techniques, texture style you know, you get it. And then the color analysis, um, you know, palette harmony, mood, light source, uh, emotional impact, um, narrative. Yeah, most of this stuff is foreign to me, but I really enjoy looking at it, um, and it seems to make sense. And probably I should be paying attention to art more than I, than I am. And then we have artwork analysis, which is the final model here, which kind of combines almost all the stuff that we've done up here and kind of creates a full analysis itself. So we have the title and then we have the composition, technique, colors, emotional impact, estimated period, artistic movement, and overall impression. So pretty cool. So those are the models. Now we get to the point where we actually pass it to our chat model. So we are able to chat with the image and get like— or passing the image and get analysis on some of these images that we pass into it. So here we have the analyze artwork function, which basically takes in an image path and it's going to do some analysis on that image and give you results based on what it sees. And as you guessed it, probably here we are using Llama 3.2 vision model to do that. So we pass the image to Llama 3.2 vision. It's going to look at it based on our criteria, of course, determine if what the— basically judge the image and give us the output, the judgment. And here we have the messages here and this is what we will follow from our content. So analyze this artwork as a critic would focus on composition elements, technical aspect, color analysis, emotional impact, artistic period, movement characteristics and all that. And then we pass in the image path for that image specifically. And then I set the temperature to 0.2. You can play with this a little bit more. If you put it a little higher, it's going to elucidate more or just become really too creative in a sense. If you put 0, then you really minimize the, you know, make it a little more deterministic in a sense. To run it, here's how we run it. You pass it in there like we did with the previous one and then extract the content from there. So once that's done, we want to be able to save the final— the analysis that generates, which should be in JSON format. We want to have that saved so we can look at it. And then the next stage here is just parsing through that entire analysis and printing out each section and making it look pretty essentially for us to be able to read it in a nice format. So that's what I've done here. And so it's just really outputting it at a terminal so we can read it and see exactly what it does here. And then finally outputting that result there. And in this one, you can pass an individual file or or an individual artwork and then get criticized on that one piece of artwork. But what I decided to do was more— well, it's not cool to just do one at a time. Why not do a comparison, right? And this is what I did up here. Basically, this acted as modules and I just imported them here. And once I imported them, uh, what happened here is I'm comparing the artwork. So I'm able to give it a folder. For example, here we have If we do this, so in these artwork samples, I have 3 files. I have Mona Lisa, Starry Night, and The Scream. So these are artworks. So if you look at Mona Lisa and then Starry Night by Van Gogh and then Scream, so those are my 3 artworks that I chose. So here I'm just going to compare all of them and it's going to do analysis, uh, based on the critic here, it's going to do like an individual one, but it's going to compare all of them together. So this is pretty cool. So it's going to get, compare all the artworks, going to take a link and then compare multiple artworks and display a summary table. And that's exactly what I'm doing here, just displaying a summary table of all of them so I can look at them side by side. So let's go ahead and run this and see what it'll output for us. And the goal here is really for it to follow Pydantic, use Pydantic models to make the output more structured. So python compare_artworks, run. So as well, this will take a little while, and I'll cut back to the point where it's finished and we'll be live. All right, so after several minutes, and we finally have a result. So let's look at it. So here you'll see it ran analyzing Starry Night, Mona Lisa, and The Scream. So it did analyze exactly what we need. Now If we look at the artwork comparison here, it did a comparison. I definitely need to verify these years and numbers, um, just because, uh, but let's look at— let's assume they're right for now, you know, we'll do verification after. So period, uh, Starry Night 1889, Mona Lisa 1503 to 1517, and then The Scream 1893. So pretty interesting, right? So we have style: expressive, realism, impressionist. Dominant lines curved horizontal diagonal, and then primary emotion serenity misery anxiety, which I think it got that correctly. And then color harmony contrasting monochromatic clashing, and then mood melancholic serene and anxiety. That's I think that's pretty accurate. So now it does a detailed analysis of each of the art. So first of all is Starry Night, and it says the title is The Starry Night. Period is 1889. Movement was Post-Impressionism, uh, pro— oh, Post-Impressionism. Yeah, you can tell I'm not— I didn't take any art classes. Um, and then composition, we have lines curved, focal point starry night, balance symmetrical, and depth atmospheric, uh, perspective. So we have technique there as well, oil on canvas, impasto. So pretty cool, right? You can see it's structured pretty well. And then color analysis, we have the color analysis there. emotional impact. And then we have the overall impression, hauntingly beautiful. And that is what is referring to the Starry Night. So let's look at the Starry Night. So this is the Starry Night, so Van Gogh. So looks pretty good. I think that captures it, um, exactly as it is. So it's hauntingly beautiful, and it is. Now let's move to Mona Lisa. So let's look at the analysis it did on Mona Lisa. So if you're not familiar with Mona Lisa, that is Mona Lisa. And so look, let's look at the critique analysis that it created here. So Mona Lisa there, it gives us the time and the movement is Renaissance. Composition, subject's eyes is a focal point, and we have the balance there. Uh, let's look at the technique, oil on canvas, dry brush. Um, and then color Gold, black. I think I see gold, black in there. Harmony, monochromatic, serene, natural. And then looks, uh, emotional impact, wealth, power. Oh, interesting. Italian Renaissance. And then if we look at overall impression here, it says a masterpiece of the Italian Renaissance. This painting is a testament of Leonardo da Vinci's skill and artistry. The use of monochromatic colors creates a sense of serenity while subject's eyes draw the viewer in. The composition is symmetrical with atmospheric perspective adding depth to the scene. The overall effect is one of mystery and, and power, making this painting a true masterpiece. Yep, sounds about right. All right, let's look at the last one, which is The Scream. Now let me pull up The Scream. Here it is. That's The Scream. It's a little haunting, a little scary. Uh, in a sense. But let's look at that. So The Scream, 1893, Expressionism. And, um, the screaming figure, that's the focal point of, of the art. And then we have the overall impression here: anxiety-inducing and unsettling. I mean, that is right. If you look at that, you feel anxious and very unsettled because somebody's screaming and they're making all the— yeah, you It just doesn't— I think you got that right. So that's just beside the point. But at the end of the day, I think the important piece here is it did structure it the way we wanted to. And we can actually look at the analysis of how it did this. And so let's do again Option+Shift+F on Mac and then save the file. And here we'll be able to see the full JSON. And this is the exact JSON that it generated. So for example, this is the scream. And it generated and then the exact examples and the entire file generated like that. This is pretty cool. The gist of me going over this is really to show you why you need to start using structured output in your LLMs to generate exactly consistent output that you need and also within your AI agents. Stay tuned for more information or for more videos like this, and please don't forget to subscribe as well. And see you in the next video. Bye-bye for now.