LlamaIndex and More: Building LLM Tech with Jerry Liu - Season 3, Episode #1 (January 2024)

Welcome back, graph enthusiasts, to 2024 and a new year of graphstuff.fm. I hope everyone had or is still having wonderful holidays, that you are recharged and ready to do graphy things this coming year. I'm your host, Jennifer Reif, and I am joined today by Andreas Kollegger, and we have a remarkable guest with us to talk about data and LLMs. Joining us is the innovative Jerry Liu, who is the co-founder/CEO of Llama Index. Jerry has worked in machine learning and AI and has long been broadly interested in generative models. Welcome.

Thanks for having me.

So just to kind of get us started and warm everybody up here, could you give us a brief overview of Llama Index and how it came about?

Yeah, it sounds great. So Llama Index is a data framework for building LLM applications. The core mission of the company is to build build the right tools to help developers connect their language models with their own private sources of data, whether that's personal data or organizational data. You know, if you're an enterprise, you have databases, you have a lot of APIs, you have a lot of documents that you're working with. So our goal is to basically give developers all the tools they need to unlock insights from that data and to really, you know, take that ChatGPT reasoning and, and generation power and apply that on top of your own data. So that was pretty much the motivation for the project. We started a little over a year ago, last November of 2022, and have been working on the project ever since and started the company around April. And so now we're one of the leading toolkits, especially around some of the core enterprise use cases for generative models, which is around stuff like retrieval augmented generation, being able to connect to your, you know, vector database, load in your PDFs, and basically start kind of asking questions and automating existing workflows. And so our goal is to basically make that very, very good, both on the open source side As well as we're building out an enterprise product right now. The open source project has been there since the beginning. It's basically one of the core toolkits that developers use to basically build these types of applications.

Very cool. It's crazy how fast this industry has moved. Your project's only just over a year old and the company's only, what, several months, less than a year old. It's just, it's been a crazy year. So what kinds of features and things are you currently working on? I know you kind of hinted at it a little bit before the, where we started the episode here.

Yeah. So from the open source side, I think it's very interesting. I think there's still a lot of very interesting directions to explore. The goal from the open source side, right, when we think about new features is there's always like a lot of different things going on and I can kind of outline the different categories. And the goal really is to stay on top of the latest trends as well as create better tooling both prototype to advance, uh, kind of LLM applications. Um, so we are simultaneously kind of keeping on top of what's going on or in the space, and we're also innovating ourselves, right? We push out new techniques and then everybody kind of adopts these techniques, uh, to use in their own software. So for instance, um, stuff that involves kind of just like keeping up with what's going on, this includes pretty much every new LLM release. Uh, now it's the holidays, you know, there's not as many, uh, LLMs going on, uh, uh, releases. But, you know, from everything from like new OpenAI models to Google Gemini Mistral to pretty much like every other like open source and multimodal model coming out. Um, there's basically a lot of just like new use cases, benchmarks, and kind of, um, we want to explore how they work, right? And, and to see what use cases they're good for. Um, so that's basically a one source of just like keeping up to date with latest, um, uh, advances. Uh, and then in terms of basically techniques that we try to push, there's a few categories. Uh, we focused a lot on kind of advanced retrieval and advanced RAG techniques for better search and question answering over your data. That is going to be— that is something we're going to continue to innovate on, both to improve performance as well as improve system-level metrics like cost, latency, dealing with production-level challenges. And then other categories in the space, like things that we innovate on, include How do we think about the relationship of fine-tuning and what's the relationship between fine-tuning and RAG? How do we think about better data representations? And that's precisely where, you know, kind of working with like Neo4j comes into play because these days a lot of the existing RAG stack depends heavily on just vector indexing and retrieval. And so how do you move beyond that and think about more sophisticated forms of both organizing as well as retrieving your data? For use with LLMs. Um, and then we're also thinking a little bit about kind of as people move on but beyond the simple stuff into more agentic behavior and using the LLMs to automate more and more hops of their workflows, how can we create robust, reliable systems there? Um, so these are examples on the open source side. Um, and then, you know, we, we haven't kind of released this publicly yet, but we're also working on kind of like an enterprise platform that, uh, um, some members of the engineering team basically are working on.

So just a few things. Yeah, it's pretty amazing. Like, so just in the things that you just talked through there, and like, you know, as paying attention to like sort of GenAI news, there's so much going on in the space all the time. And listening to you like talk through all the things that are in your mind right now, like things you're kind of trying to to keep track of and also work on. How do you, how do you sleep? How do you both keep up with like what's going on and think, oh, what are you going to do about that? How do you incorporate that? And also look into the future. How do you balance all that?

Yeah, I mean, there's kind of like the, the, um, actual answer. I'm not sure how politically correct it is, but the truth is it is, it is very hard work. I mean, I think everybody, uh, on the team has worked like incredibly hard, and they've done a fantastic job of just being able to keep up to date with everything that's going on, that balancing between kind of short-term urgent priorities, uh, with kind of longer-term priorities. Um, I think especially in this kind of chaotic environment, um, basically I, I've told some members of the team this too, it's just like there's always a chance, you know, that something big is going to come out this week and we're just going to have to drop everything and focus on it. For instance, like OpenAI Dev Day, there were a lot of releases coming out, so we basically paused other features of the open source roadmap to just focus on supporting whatever the latest releases that OpenAI put out were. There's always going to be unexpected things that pop up, and so it's good for everyone to be adaptable, resilient, and also be generalist, not get locked into specific things. Just so that everyone becomes this superpower engineer that can do a variety of AI full stack as well as backend work. I think in the end, there's not really an easy way out. I think everybody is trying, is just putting in a lot of work and hours into this. If you talk to other members of the AI space, it's similar. It's just like everybody is just always trying to discover new things, hack on cool applications. And I think the thing is, if you think about just this concept of burnout or working a lot, I think it really has to be something that you're motivated in. And in the end, I think Sam Altman said this actually, it's like if there's momentum, then it doesn't feel like you're just really trying to count the hours in the week. And so given the pace and velocity that the project has been growing, I think everybody does kind of feel that energy and excitement. I think that's an important ingredient to making sure that we continue to kind of feed that flywheel.

That's a fantastic observation. Like, you're right. Like, when you're doing something you love and you're really interested in, the time just flies. And sure, there's a lot of time that you spend, but it's not like you're exhausted by it. You're energized by it. It's actually— you're kind of part of the wave and you're in the wave, and it's all just a glorious mess of kind of thought and It's great.

Yeah, exactly. I'm sure you and other members of the team are feeling that too.

Yeah, it definitely happens to us as well. Thinking through the last year and with the continued pace of innovation that's happened over the last year and it hasn't slowed down at all, there's this classic design thinking like, okay, the diamond is expanding right now, the possibilities are growing, and then there's some convergence that happens later. That's usually kind of done or portrayed in kind of like this two-phase step, right? Where it's like, okay, diverge, diverge, diverge, then finally converge. I think in reality, while you're converging or diverging, some stuff is solidifying. You know some of that stuff is going to carry forward. You kind of have a core that's kind of solidifying as you're innovating, right? There's still things on the fringes and the details. There's still things that will change, but there's a kind of a central part that's there. Do you feel like you've identified that at this point? We're like, okay, this stuff we know, 80% of this is going to be solid, we're just doing 20%, or like, what are the ratios? Like, is there a core that you can identify?

That's a good question. I think it's kind of a complex one because I think there's nuances and some of it's kind of like probabilistic too. I can probably speak in terms of like confidence, like percentages, as opposed to like, this is like something that's— I think it's absolutely going to be true. Um, I think part of this— okay, um, maybe there's, there's different parts to this question. One is just like, um, do we kind of see the like overall hype and like, you know, rate of advancement for, for AI, like, continuing, uh, in, in the next year, right? And, and in 2024 and 2025 and onwards. Um, and then the other part is basically kind of like what abstractions have really solidified and which parts are we confident about, right? So that we can start like building more stuff around that. Um, so on the first part, I think I, in terms of rate of advancement, I think I'm reasonably confident that the rate of advancement will continue. Um, I actually, I, I think, you know, inevitably there will be like troughs of disillusionment. I mean, I think we already kind of saw that in last year when people started doing like, uh, like fancy agent stuff and then realized that wasn't like, uh, super, um, reliable. And, and then, you know, I think the hype around like pure agentic things like died down a little bit, but I can anticipate that picking back up. But just like, if you just think about the sheer scale of money that's getting poured into AI research, um, the, the competitive aspect of open source models, um, like OpenAI, um, basically just like everything in the foundation model layer to just this entire ecosystem, which includes us, of, you know, that have like techniques to orchestrate these different components as well as like new modalities, like, you know, speech, multimodal stuff like images, video. I think there's a lot of room to make a lot of innovation. And the thing is, right, like you actually don't need to go too much beyond GPT-4 level capabilities as long as you make something like GPT-4 a lot faster and cheaper. Right, and, and also multimodal, there's actually a lot of, a lot more use cases to discover. And so I think I'm pretty confident in the next year, like, just the rate of innovation of like discovery of new use cases, um, kind of making certain things that weren't feasible in the past year more feasible in the next year. I, I'm pretty certain that will, that will increase. Um, and then, and then the, the other part here is just from like the developer perspective, um, I still think that most people in 2023 in terms of like the AI engineer persona, we're probably more on the early adopter side. It's the people that like read Twitter every morning, you know, they keep up to date with what's going on. Like what did like Sam Altman tweet, right? Like what did kind of like Jim Fan or, you know, like just like AI kind of visionaries in the space. And they're really like heads down, like building different projects. But I think if you just look at the space of application developers and the promise of AI, which is that it really democratizes access to pretty much like anybody can just make an API call without training a model, without too much compute, and they can just like build really cool applications on top of this. There's a huge like, um, opportunity for just like increased education, know-how of both like kind of simple to advanced use cases. And I think we're just scratching the surface there. Um, and I think like as more and more developers join this ecosystem We're going to just have the rate of advancement of best practices, techniques, and should we do evaluations? What's the best practice for different types of data? What are the different things that we can create? That will increase too. I think from the side of the rate of advancement, I think that will continue to increase both in terms of education and the accessibility of AI, as well as just the sheer rate of new innovations coming in. And then I think on the second piece, which is basically about whether or not we, in terms of just like new innovations, or sorry, in terms of new use cases coming in, right? Like which abstractions would solidify as kind of like the innovations come in? I think in terms of the abstractions, it's very interesting. I think there are existing frameworks are popping up in terms of like RAG agents, um, and, and other use cases. And a lot of people are building production stacks which include LlamaIndex, right? You pick like an LLM, you pick a vector database, you pick a data source, and you start composing your own applications, and then you start iterating on that. I think that's starting to become relatively solidified. You know, people are kind of building existing use cases and chatbots. I will say there's still always like that 10 to 20% probability that some new model architecture comes out that'll, that'll like disrupt the existing stack. Um, and, and the reason I say that is there's some interesting research going on that, you know, might tie kind of this idea of retrieval and the model architecture more closely together. And if that's the case, then the existing way you kind of like, uh, orchestrate the interactions between like a vector database, right, and, and, uh, LLM, uh, go away. And so if that's the case, you might have to swap out the existing stack. That said, I think it's really a matter of time, right? Like, is that thing going to come out in the next year or in the next 2 years or 3 years? Like, either way, if people have a more immediate need to get value, they can use an existing framework like LlamaIndex to basically kind of get existing— to unlock insights from their data. And so they can start building up, like, or start utilizing some of the best practices that we've developed over the past year. Even if that will change, uh, potentially in like the next year or two. Um, so, so I think in terms of abstractions, I think there are some core components that are emerging, but I'm not going to discount the probability that, like, this stuff might change in the next 6 months or a year.

A lot of great observations there. I was thinking about what you just were saying about the LLMs and some of the services will start picking up retrieval as part of the core functionality that they offer. And there's an aspect of that that's, I guess, kind of classic. Whenever you have adjacent technologies, you start to bleed into each other, right? Like, so database— everybody in database world is going to start to pick up some amount of, you know, LLM capabilities, or like whatever they can. Maybe they'll host models, who knows. And on the LLM side, they'll start to absorb a little bit. And what the right balance there is, I think it'll be interesting to see how that plays out. And it's I think it's probably hard to guess until people just try it and see what works best.

Right, right. I also think it's, it's a very interdisciplinary field because, you know, as you said, everybody from the systems world is trying to learn AI. Everybody from AI is realizing the importance of like different systems concepts, especially as you like try to productionize this thing. Um, and so just in terms of just like, uh, greater awareness and know-how of like building robust systems, I think there's still a lot of gaps and a lot of opportunity there.

Yeah, it makes sense. If I recall, some of your background was in recommendation systems, is that right? You spent some time doing that, or—

Yeah, that was, uh, my first job out of school, uh, which was at Quora as a machine learning engineer.

How much of that has influenced how you think about this space, I suppose? Because there's a lot of— it seems like a lot of the sympathetic thinking in that area.

Um, well, okay, I, I think on a very technical level, I learned what embeddings were. So that turns out to be very relevant. But I think the idea of recommendation systems is actually pretty different than the existing RAG stack. And actually, I think, you know, like hot take, I think people maybe aren't thinking about it enough in terms of like consumer-related applications. I think like the way recommendation systems works is that the way the embeddings are generated is based on user behavior. It's not really based on the semantics on the content. I mean, it is too, like these days it's based on both, but like you know, just a lot of algorithms. It's more based on, you know, you're watching Netflix, you clicked on these movies, and so for another user that watched similar movies but maybe didn't watch this exact movie, like, would this, you know, would they likely watch this movie, right? That's basically how the recommendation, uh, system works. And so for a lot of kind of consumer-facing applications, um, like, uh, stuff with like a news feed, stuff with a lot of content, like shopping, uh, these algorithms basically power a lot of that. And so I think if you try to combine that with this reasoning capability of large language models, I think it could be pretty interesting because, you know, the embedding representation isn't generated by the semantics. The way you do retrieval is based on what you think this user would likely watch or, you know, consume. And so just the way you do personalization and the way you kind of like generate these embeddings is using a slightly different algorithm. And so especially as we think about like chatbots these days, right? Like if we think about like a personal assistant, um, kind of cap, like utilizing, um, like a user's previous behavior and, and they have awareness of what like the user's actions are. I think there could be interesting ways to like blend kind of recommend, uh, recommender system concepts into this idea of like memory or personalization of building a chatbot for users. Um, but that's basically the connection, but I haven't explored that too much yet.

And, and that sounds actually very similar to bleed into knowledge graph land.

Yeah, I think so. I think, I think, um, there's one idea of basically kind of, um, you know, you construct like a graph of like different relationships, uh, kind of within, uh, your behaviors, and, and then you can like somehow dynamically construct that over time. And then the other idea is just like the, the actual embedding you generate for each kind of piece of content might dynamically change and it isn't just based on the semantics on the content, it's actually based on your interactions and your pattern behavior. I think that's— yeah, there's a lot of interesting stuff there. I don't think people talk about it enough. I actually haven't thought about it as much as I should have, but I think it's very interesting.

Well, again, all of the space is relatively new. I think people are still trying to get their heads around what do we do with this. I know we at Neo4j have have kind of explored into that knowledge graph land, but even, even we haven't figured it all out yet, you know. So there's, there's still a lot to learn, I think, a lot of, a lot of progress to be made yet.

Definitely.

Yeah. So the exploration that we've had on that side for sure is exactly as you're saying, like with the personalization. For us, all that users like me ends up being a graph pattern match stuff because we, we know about the users already. We don't need to do an embedding for that. And it's as efficient or more efficient for us to do the graph pattern match for the, okay, you've LLM, you've never seen me before, but you've seen people, the app knows people like me. And so we can actually then actually refine some of the answers, get a better relevancy score based on that. So users like me who have relevant, you know, content kind of refine down the set and then provide that back in for the context.

Yeah, that makes sense. So basically you're using these relationships to refine the retrieval. Process and these relationships are like related to this idea of kind of whether a user like, like something or not.

Yeah, exactly.

Makes sense.

So through all this then, we, we've gotten through like, okay, and you've covered a lot of ground already, which is pretty fantastic. And you started to touch a little bit on like, you know, sort of the use cases. I think the classic one, like the kind of hello world of, you know, ChatGPT kind of apps is like, okay, just Talk to it, of course. But then it's the, with the RAG part, it's okay, let's talk to our knowledge base. We've got a customer support knowledge base, I think is the kind of classic one. Rather than having the current generation, the pre-LLM version of chatting with customer support, now you can actually talk to an LLM that has access to the customer support material. That seems to be like the easy, if for any company who's got any kind of customer support, that's your first step into, into, this world. Do you think that's right?

Yeah, I think so. I think typically for a lot of enterprises, they start off, one, I think a lot of companies are interested in RAG. To do RAG, they start off with an initial dataset that tends to be pretty small. So with a specific use case over a small amount of data, maybe like 5, 10 documents. And then, you know, they try it out. They, you know, use a framework like LlamaIndex, pick an LLM, pick a vector database, and then, you know, you like get something and prototype working. I think the, like, there's a lot of challenges in trying to scale that to larger amounts of data. And we've talked to companies, especially the bigger ones, you have like kind of diverse sources of data. You don't actually quite know how to parse different sources of data. Like sometimes you have very complex documents with a lot of embedded tables or charts or links or references to other documents. And you like don't really have quite the knowledge or the time to really just like parse apart every useful bit of information here and try to feed it into the LLM. I would say one of the biggest use cases these days, again, just from the enterprise side, when we talk about RAG, I think it's not super precisely defined because RAG at a very basic level, it's just a very simple framework. You just do top-k retrieval from a vector database and just put it into the context window. But there's really advanced RAG and then there's agents and the high-level use case that people care about, that enterprises care about, is kind of just good search and retrieval over your data. Like, how do I ask a question, get back an answer? And that data could be arbitrarily complicated. You know, it could be a single set of documents, could be multiple documents, could be multiple documents across like unstructured, semi-structured, structured data. That's really what they care about, right? And I think that part actually still isn't solved yet. I think there's a lot of challenges and, and, you know, we are kind of there trying to really establish the best practices for these different challenges. And trying to help people, guide people through the process of, you know, developing the best architecture for their use case. The kind of like ideal goal for this type of system, this question answering system, is that if you imagine just the spectrum of literally any type of question that you might want to ask, whether it's a simple one or advanced one or a multi-part one, you're able to get back an answer and you're able to get back good citations, sources, right? And so even on the search and retrieval side, I think there's a lot of challenges. I think when people think about like agents, um, they typically think about things that can not only do like search and retrieval and kind of like complex task handling, but also, um, so they not only read state but they also write state. Um, and that's probably the, the next step after this. It's like, you know, I think we, uh, a lot of our work has been thinking about agents with respect to search retrieval, so stuff that doesn't modify state. I think that tends to be a little bit more, um, appealing to people who, who are concerned about like reliability or safety because you're, you know, worse you get is you get back the wrong answer, but you're at least not like changing the underlying data. Um, but once you start like getting this thing to start automating the workflows, like send emails, like schedule calendar invites, you know, like look up people and perform actions, um, that's when you start getting into kind of, you know, the ADEPT style, like AutoGPT, like you go in and actually try to do things. Um, we'll probably see a bit more of that as these models get better, faster, and cheaper. I think, you know, as, as I mentioned briefly, there was a lot of like interest in this type of workflow automation type applications earlier on this year. But I think due to like reliability and cost issues, mostly reliability, it's— people started trying to really think about what are the best practices to actually constrain these models and make sure that they work well. So I can imagine that becoming a bigger and bigger portion of what enterprises care about going into next year.

Yeah, I can imagine on a selfish level, just like if the travel industry could just get on this and I could book my flight out, like I mentioned, later in January when I'm going to fly out to California. If I could just say, here's my budget, here's the dates, just go figure it out. You know what I want, you know what I'd like. And it's either me having to do it or me having somebody that I trust having to do it. This, it seems like just so perfect for an LLM-powered solution.

Yeah, 100%. And, and the other thing is, um, maybe, um, this relates to this idea of like prompt engineering. Um, I think either way, and I think, uh, Logan from OpenAI actually said this recently, um, you're going to have to communicate with this AI assistant agent, this LLM-powered application. And, um, the idea of prompting I think will never really go away. Because even if you, you know, the LLM is able to handle something under the hood, and it's able to handle a lot of stuff under the hood, in the end you still need to give it some task, right? You need to give it some instructions. And in the limit, it's kind of like communicating with a human or a super intelligent human. And so you still need to be able to guide this like assistant in the right way to kind of like achieve the goals that you set out for it. And so I think, um, like, I think, like, you know, in the past year, people are really trying to figure out, like, properties of different types of models. Like, um, you know, do I try to really, like, insert, like, the right tokens for, like, Llama 2 or Mistral to make sure that it's able to achieve this task well? Um, I think as these models get better and people, uh, and, and as these cases of having this, like, personalized assistant that can really do anything as that, uh, vision really takes off, um, we're gonna kind of start to see better practices of how people are able to converse with these assistants to achieve the goals that they want. Um, and I think that's gonna be a pretty interesting thing to, to explore because it's kind of like at the intersection of, it's not really code at that point. It's like, it's like how you communicate and how you interact with, with, uh, uh, an assistant.

I mean, to be fair, when computers first came out, you, there was a, a learning curve to figure out, okay, this basically a glorified calculator, you know, how I— to some degree you have to think like a computer in order to successfully interact with it and get what you need out of it. And that, that kind of to some degree has always been the case. And you look at, you know, code, you kind of again have to think in the computer's terms in order to get what you need out of it. And so there probably will be some of that, um, just more natural language now than, than what we've dealt with anytime in the past, but Um, but yeah, I think there still will be— you still need to, to somewhat understand where the computer is coming from in order to, to help it get you there, right?

I guess what I'm saying is I think it's starting to converge on kind of just like human levels of communication, or, you know, maybe like kind of some level of communication that's neither human nor, uh, pure code. Um, and, and, and then like people are gonna have to learn these skills of how do you actually properly use natural language to, to to converse with AI.

Yeah.

Oh, I see. But so with some of the prompt engineering where it feels like you're kind of trying to trick the LLM to do something, like that part of it will go away because the LLM will have a better sense of what you mean rather than you trying to figure out how to really tell it, no, seriously, don't do the thing. Don't do this, please don't do this. And you say it 12 times to make sure it really doesn't do that.

Right, exactly. It's like there's the hacks that people are discovering right now and then there's just this like longer-term kind of skillset of how do I actually just, you know, talk to this thing to actually, um, it's like having a, it's like having a kid, right? Or having like a sibling, or, you know, having a friend, or having a pet. It's like, how do I guide this thing to, um, work with me? Or I guess it's different than, you know, having a friend because you're actually just trying to guide this thing to do what you would tell it to do. But like, um, how do I properly converse with this so that it can help me achieve my goals, right?

Really cool stuff.

Yeah.

So, Jen, I was just to have a quick time check here. We're into half hour. Like, I'd love to keep this conversation going for a while, but like, how do you feel about our time so far?

Yeah, we can go ahead and, and kind of maybe segue into our Tools of the Month. Cool. So anybody want to go first?

I see in the show notes that I've got my initials up first, so I'll go. And I'm I've been in the last month rediscovering frontend frameworks despite wanting to spend my time on just LLMs and GenAI. I'm like, oh, I still have to write frontend code. There's a framework called Remix, which I've actually become quite fond of. Next.js and React are where people usually go for things these days, or Svelte if you're being super cool. But Remix takes a really interesting approach to how it puts together frontends. It's a very nice client, server kind of coalition that kind of comes together. And it lets you kind of have things that are much more— it feels much more data-driven. Even though we have like headless CMSs and other frameworks for building, you know, front ends, you know, today, like we kind of separate out the API, like you define your models and things. That's great. This feels much more like you are just defining your UI, where you want data to appear. And then you certainly have some loaders for like, okay, I don't care where that data comes from, something else is going to take care of this at some point. And then of course you can plug into GraphQL or whatever you want to, you know, eventually. But it's kind of a— that all gets handled magically on the server side. And then just on the client side, it's just, okay, here's just HTML pages. You end up basically serving up HTML pages that have been hydrated with all the data that they need. There's some nice magic that happens behind the scenes. You can keep things up to date when data gets invalidated. But it's a really nice framework for just building data-driven, you know, websites, I would say. Somewhere in between you know, like React feels like you're building applications and like Next.js is more like you're building websites. This is like kind of in-between world where you're like, it's not quite an application, it's like a data-driven website.

Is it, is it kind of like a Retool sort of thing?

Uh, I haven't spent a lot of time with Retool, so I don't know that I can say that. I thought Retool had more like of a UI approach to things. Yeah.

Like low-code.

Yeah.

Yeah.

Okay, so this is more code probably than that.

This would be like Next.js if Next.js was more focused on data. Okay, on just content. Okay, not, not quite fair to say, but it's something like that.

Okay, cool. Um, for mine, I, um, kind of going back, Andreas talked last episode about a terminal that was just fabulous and raved, raved about. So if you, if you haven't seen that episode, go back and check it out. Um, but it got me kind of thinking. I was playing around with some, um, API building stuff, applications and things, and I was just using plain old cURL, um, and realized how, how boring and drab cURL is in some respects. Um, and so I, I went back. I have used HTTPie in the past, and that's kind of my go-to. But I was trying to find something that, that would be applicable and everybody would have installed, so that's why I was using cURL. I'm like, you know, there's really some nice things about HTTPie that I just really miss. So first of all, the data results that get back are auto-formatted, auto-prettyfied, which is super. Um, the, uh, requests are very, uh, they feel a little bit more user-friendly, I guess, um, because you type HTTPie and then if you're using localhost, you don't even have to type localhost, you just put port number and then your, your API path at the end of it. So it's just, there's a few little things that it just simplifies and it just makes boring old command requests a little bit easier and a little bit cleaner, I guess, is, is what I would say. So that's kind of one of the basic things, but again, bring— brings me joy on a daily basis doing the same old commands over and over again. So HTTPy.

Cool. Nice. I'll just talk about two things. One is more related to actual work and the second really has nothing to do with it. The first one is I've been pretty interested in just like headless browsers for like that you can just import as Python package. And so I discovered like PyPetir, which turns out to be unmaintained now. I think they tell you to use like Python Playwright. But basically, you know, why, why, why do I care about this? It's because like It basically just gives vision capabilities to a multimodal model to browse websites. So if you have like a headless browser, you can go up to any website, screenshot it, and then feed it to like GPT-4V, Gemini Pro, and you can go in and actually just like have it not just comprehend the website through like parsing the HTML, but actually just parse the screenshot. And it turns out it's actually, if you like feed an Amazon page or feed it like a, you know, shopping page on Macy's or something, it'll be able to kind of actually understand that from like just the layout, right, of the screenshot versus actually having to dig into the URL. And, or sorry, not the, the parseHTML. So I think, I think that's been pretty interesting to explore. I think on a pure non-work-related, or yeah, non-AI-related thing, I like a kind of efficiency when using my computer. And so I really like Rectangle, which is on Mac OS X. You just use a keyboard shortcut and it's splits your screen into a bunch of different rectangles and then you can go in and just like, you know, you can have like two Chrome tabs side by side. I use it all the time when I'm on a Zoom meeting. And speaking of Zoom meetings, I've been using Fireflies.ai for just like transcription and notes. And so that's been pretty nice because then you can just go in and kind of like recap things without having to take notes afterwards. And then the last thing I'll say is I really like Video Speed Controller, which is a Chrome extension. I never watch videos on 1x anymore. So basically for anything, any video on the internet, you just like dial it up to like 2x and you can, you can basically just like speed up the rate at which you consume content.

So I didn't, I didn't used to, to speed up video, and, uh, Mark Needham and Michael Hunger have got me doing it, and now I can't go back. I can't listen to anything on normal speed. I always have to speed it up at least, at least a little, like 1.25, 1.5, you know, at a minimum. But exactly, yeah, that would be super helpful. I may have to look into that browser extension now because that's always the irritating thing. You can, you can speed it up on YouTube, and now I've noticed on some of the embed video embeds you can speed them up, but if it's like a Vimeo or some of the other links, you can't always speed up the video.

Yeah, this will let you do anything.

Ah, that's awesome.

Life hacks. This is how you keep up to date with everything. You've got podcasts at double speed, videos at double speed.

Efficiency.

48 hours a day.

Exactly.

Okay, so, uh, Jerry Liu, uh, this is where we can let you drop off if you would like. Before we dive into all of our, our Graph news. But thank you so much for joining us. I really appreciate all of your insight and I think this is a fantastic episode and to hear your perspective going into 2024 to talk about LLMs and kind of where everything is going as well as Llama Index. So thank you very much.

Of course. Thanks for having me. Take care.

Good to see you, Jerry Liu.

Happy New Year.

Happy New Year.

Okay.

I didn't have anything noted in community projects for this month, but we will hopefully have something headed your way for next month. As far as product updates, there's several things going on here in the last month, but most of them all center around the fact that there was a new Neo4j release of 5.15. And so there's a few things that are involved with that. There's actually a command now for node vector indexes. There's some changes to CDC, change data capture, and then some large transaction work being done there as well. And then of course you have things that go along with the Neo4j release like driver updates, APOC core updates, GraphQL releases, Helm chart updates, and then several Neo4j connectors were also updated. So again, I'll have all of those links in the show notes. But most of all of that centered around the new Neo4j 5.15 release. Anything else, Andreas, that you can think of?

No, I have to be honest, I'm still in the holiday mindset, so I'm trying to remember what was happening at work. Yeah.

Yeah, December, the last or the first, I guess, 2, 3 weeks of the month. Turning the dial back over to the work setting this week. Okay, so just looking at some articles, which again, I have not had time to go through everything yet, because again, we're on holiday time here, but just kind of doing some overview of some things. There is a new next-generation graph-native storage format in Neo4j, and so there's an article talking a little bit about that. I know internally we've seen kind of some things pop up, but this is now being talked about on an external level. So this is a blog post on the Neo4j developer blog talking about this. So feel free to check that out if you're curious about like database internals and how, how the engine works and all that good stuff. There should be some interesting things there. And then kind of going back to what we were talking about with Jerry Liu, there's some implementing advanced retrieval RAG strategies with Neo4j. So there's an article out there for that. There's some knowledge graph stuff, some more, some more RAG, a Neo4j and LangChain article as well. So those are the things that have all hit the developer blog this month. I did see there's been a couple of videos added to the Nodes 2023 playlist. So if you were a part of Nodes, didn't get to see some of the content or miss some things because you were in one track or time zone or something. There's new videos popping up all of the time. So be sure to keep tabs on that. That'll be probably a recurring thing for the next month or two, I would think at least. It's kind of a trickle of release there. And then the other video is RAG with a Neo4j Knowledge Graph, how it works and how to set it up. And I actually watched this video. It's It's not a very super long video at all. I want to say less than 10 minutes. And so interesting. He walks through kind of how to, how to get things started, what all that entails, and so on there. So good video, interesting and relatively short, as you maybe are kind of closing out holidays and want to kind of keep tabs on work-related things. Or maybe when you get back, you want to kind of ease that on-ramp into to work things, that would be a good starter thing to have. As far as events, we've got a few things rolling in January, but nothing too major. Again, we're starting the year, you know, after the holidays. Everybody's still in their holiday comas, right? So January 4th, we still have the Going Meta YouTube series that will be online, virtual, livestreamed. So check that out. There's— on January 10th, there's a couple of things. Virtual, and then there's, um, a meetup in Austin. It's also virtual as well. Um, January 11th, a couple of webinars happening there talking about trends, what data analytics need— leaders need to know. Um, and then January 17th, there's an O'Reilly Media, uh, Generative AI for Healthcare. That's also virtual, so check that out. Um, January 22nd, yet another virtual webinar, um, talking about chatbots, Gen AI, Then January 23rd, some more virtual things, so not a lot of travel going on, so you don't have to leave your comfy warm home in January. January 25th, another YouTube series for Neo4j Live. This is something that our colleague Alex runs, and it looks like they are talking semantics-based recommender systems for ESG documents. Check that out. It's a YouTube series. And then there is a conference in Bristol, UK, GraphTalk Government, at the end of January. That's January 25th. And then closing out the month, there is a meetup on LLM Knowledge Graph in London. And then another meetup, which I could not find a location on that one, but it's talking cloud-native geospatial analytics combining spatial SQL and graph data science. So those are the events that will close out our January. So a lot of virtual things. Be sure to check those out, um, and hope everyone has a nice, quiet, and productive January.

And I will give a plug for the London, uh, community. We've, uh, committed to next year we're going to be running meetups every month out there. A lot of it's going to be GenAI-focused. We're working with some GenAI groups both in London, also up in Cambridge, um, and sort of building the overall community in UK, which is going to be great. So if you happen to be in the UK, you're interested in graphs or GenAI, reach out to us. We'd love to have people who have great ideas they want to talk about, whether you're on the, you know, the enterprise side, if you're a developer. And we actually have some academics that we're reaching out to, some, some of the schools we're collaborating with that are going to come out to the meetups as well. So it'll be a lovely convergence in the community of research, practical applications with developers, and then of course business value from the enterprise perspective on stuff. So if you have any thoughts on any of that, please reach out to us. We'd love to talk to you.

That sounds like a great cooperation.

Should be good. I have high hopes.

That'll be fun. Good way to start off 2024.

Totally. Yeah.

All right. Well, I guess that closes us out for this episode of GraphStuff.fm, and we will talk to you all in the next month.

Good to see you, Jen.

Cheers. Take care. Bye.