Intro to Large Language Models | LLMs for Answering Questions

Now let's Explore question answering. LLMs, question answering or QA tasks can take several forms. Here are three of them. In extractive QA the model receives a question and a context and the answer is extracted directly from the context. This supervised learning task requires an encoder architecture. In open generative qa, the model constructs the answer using language generation based on the context rather than extracting it. This relies on an encoder decoder approach. In closed generative qa, the model generates the answer based on its knowledge without any context. This relies on decoder architecture. Let's focus on extractive QA using the extreme dataset. This dataset contains many cross lingual subsets. By using load dataset and the name argument, we load an English subset. The loadedsubset is structured as a DataSetDict object containing two dataset objects called Test and Validation. There are three key features within these context, question and answers. Let's access these attributes for one instance. In the test dataset, the answers attribute, when printed, shows a dictionary containing the answer's start position in the context and the extracted answer text. Extractive QA is a supervised learning problem, namely classification. The pre processed question and context are jointly passed as input to the LLM, which returns some raw outputs or logits. For each input token in the sequence, two output logits are produced indicating the likelihood of the token being the start or end position of the answer span. Post processing these logits determines the predicted answer span, a segment of the input sequence to find by start and end token positions with the highest probability of containing the answer. Let's see this process in action. First, we instantiate a custom tokenizer and model checkpoint. For extractive qa we opt for a version of the mini ML model already fine tuned on the squad dataset. Next, we specify the input example strings for the question and context, passing them to our tokenizer and requesting the results as pytorch tensors. The resulting inputs variable includes input IDs and attention mask tensors as well as the token type IDs tensor with zeros for question tokens and ones for context tokens. We instantiate it using the auto model for question answering class, passing the model checkpoint variable with the model's name from the hub. We do the forward pass for inference using a double asterisk to unpack the inputs dictionary. Next we pre process the raw outputs, analyzing the start and end logits for every input token, extracting the most likely answer location and decoding the answer. We added one to the end position index to ensure the last token in the answer is included. To accommodate context longer than the LLM's maximum sequence length, we can set the returnoverflowingtokens argument in the tokenizer as true to activate a sliding window that splits the context into multiple overlapping windows. The maxlength and stride arguments help set up a window size, for instance, 100 tokens, and the distance between consecutive windows, for instance, 25 tokens. Here's the size of each resulting window the last one is typically shorter, unless the original sequence length is a multiple of the window size. After decoding, we observe that the original input sequence is split into several windows with sliding context passages. Time to practice.