Jerry Liu explains embeddings as dense vectors of numbers that represent text. Vectors for similar content cluster together in space, while unrelated topics stay far apart. This proximity lets systems measure semantic similarity, which powers modern LLM data systems.
Chapters
Embeddings is a part of kind of a very common stack emerging these days around this LLM data system that's emerging. An embedding is just a vector of numbers, usually like floating point numbers. And so, you know, they— you could have like 100 of them, 1,000 of them, it depends on the specific embedding model. And the way an embedding works, it's just think about this list of numbers as a condensed representation of the piece of content that you have. And so, you know, if you can somehow in a very abstract manner take in some piece of context. Well, let's say this paragraph is about the biography of like a famous singer, right? And then you get an embedding from that. It's a string of numbers. The embedding has certain properties such that this string of numbers is closer to other numbers that are semantically about like similar content, and farther away from other strings of numbers representing texts that are farther away in terms of semantic content. So for instance, like, if you look at, like, the biography of a singer, it's going to be pretty close to biography of another singer, versus if it's about, I don't know, like the American Revolution or something like that, that embedding will probably be a little bit farther away. And so it's a way of, like, condensing a piece of text into some vector of numbers that has some mathematical properties where you can measure similarity between different pieces of content.
