A short YouTube segment (Two Minute Papers #22) where Károly Zsolnai-Fehér explains research that converts lecture videos and transcripts into interactive lecture notes. The work aims to extract derivation milestones, segment drawings and formulas, group similar visuals, and interleave visuals with descriptive text so users can click through key points and full derivations.
Dear fellow scholars, this is two minute papers with where we expand our knowledge in science and research. Blackboard style lecture videos are really popular on YouTube nowadays. Khan Academy is an excellent example of that where you get the feeling that someone is sitting next to you and teaching you, not like someone who is addressing you formally from the podium. Without question, these kinds of videos can augment textbooks quite well, however, they are often not easily searchable. This piece of work tries to take this one step beyond the input is a video and a transcript, and the output of the algorithm is an interactive lecture note where you can not only see the most important points during the lecture, but you can also click on some of them to see full derivations of the expressions. Let's outline the features that one would like to see in a usable outlining product. It has to be able to find milestones that are at the end of each derivation to present them to the user. If you studied mathematics, you know how mathematical derivations go. Following the train of thought of the teacher is not always trivial. It's also important to find meaningful groupings for a derivation. This involves finding similarities between drawings, trying to find out the individual steps, and doing a segmentation to get a series of images out of it. And finally, the technique has to be good at interleaving drawings and formulae with written text in an appealing and digestible way. It is very easy to mess up with this step as the text has to describe the visuals. Even though I wish tools like this existed when I was an undergrad student, it is still important to just study, study and study and expand one's knowledge. If textbooks like this start to appear, I'll be the first in line and I'll not be reading, I'll be devouring them. Also, think about how smart the next generation will be with awesome studying materials like these. Thanks for watching and I'll see you next time.
