This content was created by LARRUZ. This content was uploaded by LARRUZ. This is a YouTube video (title: "RT-X and the Dawn of Large Multimodal Models: Google Breakthrough and 160-page Report Highlights", created 2023-10-03) presented as an alternating two-speaker transcript. The speakers summarize and highlight recent research: a ~168-page GPT Vision report and a new Google DeepMind robotics effort (RTX/RT1x/RT2x). They describe open-source data used (over 500 skills and ~150,000 tasks), the idea of training a single general-purpose robot model on diverse datasets (outperforming specialist robots), and example domains (kitchen manipulation, cable routing, door opening, wiping robot at Berkeley, quadrupeds). The excerpt reads like a presenter-style dialogue that walks through paper findings and upcoming implications for vision, video, and robotics, and references an exclusive report from The Information.
Chapters
Just as I was reaching the finishing.
Pages of this epic 168 page report.
On GPT vision, which showcased unexpected abilities, novel use cases and predictions for the.
Future, Google dropped their colossal RTX endeavor.
And the data Google used with over 500 skills and 150,000 tasks is open source. I've picked out over 75 highlights from both papers which I read in full.
And I'll also bring in an exclusive.
Report from the Information to give us an idea idea of what is next.
By the end of the video, I want you to have a great sense.
For what GPT with Vision can do.
Today and a more acute awareness of.
What is coming in Vision video and robotics tomorrow. But let's start with this Google DeepMind report released just a few hours ago. Essentially what it showed is that you.
Could create a general purpose robot from data from diverse robotic tasks. These were data sets from different universities.
On different continents, and Google wanted to.
See if this diverse data could improve.
Their famous RT2 model. I talked a lot more about RT2 in the video you can see on.
The screen, but essentially it was trained.
On web data as well as robotics data. That meant that it could understand questions.
Like pick up the extinct animal.
But the RTX series is another step up, even though it comes just two months later. The report highlighted that conventionally robotic learning.
Methods train a separate model for every.
Application, every robot and even every environment. Their big finding was that training a.
Single model on that diverse data enabled that robot to outperform even the specialist robots.
The improved version of RT1 became RT1x and RT2 became RT2x.
But here you can see RT1.x out competing specialist models. Weirdly, aside from this wiping robot at Berkeley, in a range of domains you've.
Got kitchen manipulation, cable routing, door opening and many more tasks that I'll get to in a second. The paper even demonstrated applicability to robot arms and even quadrupeds think four legged dog like robots.
And here you can see how it improved RT2, which was already.
