Learn best practices for multimodal prompting using Google's Gemini model family!

I'm delighted to introduce Large Multimodal Model or LMM prompting with Gemini, both in partnership with Google Cloud and taught by Erwin Huizinger. You may be familiar with prompting a large language model or LLM by writing text prompts. LMMS Large multimodal models are a further evolution of that which accepts multiple types of inputs like text, images and video. For example, if you have a picture of a plate of cookies with some text asking how would I make this a multimodal model? Correspond by giving you a recipe and baking instructions for making those cookies. This is called multimodal prompting. In this course you learn best practices for multimodal prompting and hone your instincts on this important field. Using Google's Gemini model, for example, it turns out experimentally that if you input the image before the question, you get a better answer than if you order things the other way around. Erwin will share many practical tips like this in this course. The ability to have a model that can reason across images and text is quite new. Before multimodal models such as Gemini, one way to work with images and text would be to use an image captioning model to describe the image and then feed that caption into a large language model. But with a multimodal model, the same model can understand these different inputs directly. I'm delighted to introduce the instructor Erwin Huizinger, developer advocate for generative AI on Google Cloud. Erwin has helped many developers with their two chaining processes for building with generative AI and look forward to sharing his deep practical experience with us here.

Thanks Andrew. Multimodality has played a big role in expanding use cases such as enhancing customer service by analyzing and responding to text and image based inquiries and enabling developers to create applications that process both images and text. In this course you will explore various multimodal use cases including interaction with images containing text or tables as well as videos. You apply best practices for Gemini prompting with a mix of images and text and learn to choose a model parameters that influence the model's creativity or consistency. You will also learn how to enhance language models with real time data integration through function calling.

If you learn these skills as a developer, then hopefully many people will be look up to you as a role model. I hope you enjoy the course.

It.