Ofer teaches a course on embedding models with Vectara, covering how they work and why they power semantic search. You'll learn contrastive learning and dual-encoder architecture, then build and train a simple dual encoder yourself.
I'm delighted to introduce embedding models from architecture to implementation. Built in partnership with Vectara and taught by OPA Mendelevic, this is a technical course that will describe the history, architecture and capabilities of embedding models. You may have seen embedding vectors used in many generative AI contexts. These vectors have the remarkable ability to capture the meaning of a word or phrase. Further, vectors with similar meanings are close to each other in vector space, which means that you can compare vectors with a low cost algorithm like Cosine Similarity to find nearby vectors. This revolutionized retrieval by enabling semantic search, meaning retrieving query results that have a similar semantic meaning to the query instead of just matching keywords. But what are the models that generate these vectors? To dig into this, I'm delighted to introduce Amin Ahmad, Vectara's co founder and ofa, who is head of Developer Relations. Vaitara builds RAG systems and they're therefore deeply familiar with embedding models.
Many LLM apps today use a single embedding model for both questions and answers. Say you have a question like where's the capital of France? Then in your vector database you have an answer like the capital of France is Paris. You expect your retrieval engine to return the answer and other answers that are relevant to the question. But what happens if in your vector database you also have a variation of the question itself, or even its exact match? You don't want to answer the question with itself, rather you want the best answers. So how can you make an embedding model that groups similar concepts and at the same time understands the difference between a question and its answer? You'll see that through a process called contrastive learning, you can train a pair of embedding models, one to embed the questions and the other to embed answers. To pull the relevant answers close to the question, you will build and train a small version of such a model known as a dual encoder.
In this course, as you see, a lot is happening inside embedding models, which are really a fundamental element of many of the OM based apps you might build. And I'm glad that OFER is here to teach us about it.
Thanks, Andrew. It's hard to overstate how big an impact semantic search has had on retrieval. This big shift has been overshadowed by the even larger impact of large language models on NLP as a whole. It turns out that the two trends are closely related. The transformer model has enabled both LLMs and accelerated what we can do with embedding models. In this course, you will see the progression of models that take you from word embeddings to contextualized word embeddings to sentence embeddings that can capture the meaning of the entire phrase. The course will dig deeper into one example architecture, the dual encoder. This architecture uses two transformer based encoders, one trained to generate question embeddings and another trained to generate answer embeddings. You will build and train a simple version of this model and then see it in action.
So this is a technical course that focuses on explaining the key concepts behind the models. Useful embedding we'll go deeper than how to merely use APIs to get embeddings, and I hope this course will help hone your intuition and therefore help you decide when and how to use embeddings. I hope you enjoy the course.
