Bad AI Predictions: Bard Upgrade, 2 Years to AI Auto-Money, OpenAI Investigation and more

People are starting to get used to the breakneck pace of AI. So I wanted to show that developments in just the last few days fly in the face of what was predicted only a couple of years ago. Starting with the upgrades to Bard. Then a snapshot of Claude 2, the all encompassing OpenAI investigation and inflection. AI's predictions of a self guided million dollar making AI coming in just two years. But I'm going to start with this page from a book released in 2021 called A BR History of AI. Look at the tasks it says are nowhere near solved and at the bottom it says at present we have no idea about how to get computers to do the tasks at the bottom of the list. I'm not arguing that these are solved, but this year we are getting pretty close. Check out the second one down. Human Level Automated Translation I asked the new Bard to write a poem about machine translation and here it is. I'm not going to judge the poem, but then I asked now translate it into Spanish. Of course that's nothing new, but listen to the quality of the text to speech model used to read out this poem. Here's a snippet. O traducione automatica eres una amante in constante un momento eres bella al sigente un desastre. I don't know about you, but that is starting to sound awfully human. Like now. Yes, it could do this before for English, but now it can do it for even Swahili. And I know some people will say don't we already have Google Translate? But Palm 2, which is the model behind Bard and outperforms Google Translate. I covered this in the original Palm 2 video, but the conclusion was we observed that Palm 2 improves quality both over the original Palm and Google Translate. Okay, that is pretty insane, but what about the next one interpreting what is going on in a photograph? Are we nowhere near solving that? Well, I gave Bard the meme that you can see and I said interpret what's going on. And it said this. The image shows a pizza that has been shaped to look like the Death Star already. That is so savvy to me that knows it's a pizza despite it being so strangely formed. And it can interpret that the toppings make it look like the Death Star. Of course, as a bonus it can read the text and therefore understand the meme it says it's humorous because the Death Star is a symbol of death and destruction compared to a pizza, which is about food and enjoyment. A quick bonus tip by the way, is that you can scroll to the end of the response and click this modify response button and then you can adjust the output. For example, I'm going to make it do something shorter and you are going to see a shorter version of this interpretation. And here it is apparently reduced by about 25% now that Google Lens is incorporated into Bard. I use it daily on my walks to answer questions about things around me. Like maybe I see a butterfly and I ask what type of butterfly is or a plant. And one thing to caution you on is that if it sees a human face it will block that out and not answer the question. And one more crazy thing that I found and curious if anyone else found this is that I took a photo in my local park and it actually recognized the location of the park and it was not a nationally, let alone internationally recognized park at all. Now that didn't work every time, but it is something you might want to try on your next walk or run and you might have noticed that that's quite similar to the one second from last which is interpreting a work of art. Now again, I'm not saying that solved. And there may be some reverse image search going on here. I asked write a haiku where the stairs are going in this work of art. Famously this sketch is about the stairs going nowhere by Escher and it wrote an almost perfect haikyuu about the fact that the stairs are going nowhere. Let's now take a break from that book and look at what professional forecasters predicted that AI would be capable of for the math data set for this particular benchmark in 2021. They predicted a score of 21% in 2023 and 52% in 2025 and the predicted trend would hit 80% only in 2028. Five years from now, seven years from then. Well, regular viewers of my channel might know that GPT4 can already get 78% of problems from that data set correct already today. And you can see here that there is still quite a lot of room for further improvement. And honestly this is without code interpreter or Wolfram Alpha. So I actually ran hundreds of experiments of my own using GPT4 with code interpreter on the math dataset and was getting more like 86% correct. But back to predictions from 2021. Notice two of the other categories that are apparently nowhere near solved. Understanding a story and answering questions about it and writing an interesting story. Here is a 112 page novel written fully by GPT4. Now I think it meets the definition of being interesting, if not human level, but honestly When a model can be fine tuned on an author's work, I think it will get very, very close. Prediction is that we are less than a year away from a human level novel. Not as good as the best of course, but fooling many readers. But here is where I can explore a bit with Claude 2 which can take in 100,000 tokens. I was lucky to get early access to Claude 2 and ran hundreds of my own experiments, which I will talk about more in future videos. But for this video, remember the book's predictions about answering questions about such a novel or story. Well, I inputted that entire GPT4 generated novel and I said find 10 sentences whose vocabulary could be made less generic. And if you look at these suggestions from Claude2, it does indeed make the vocabulary more exciting. Definitely not perfect, still quite generic, but we have words like crystalline, ethereal and inocerable. I love that word myself. One of my favorites. Again, I think we're less than a year from a high quality full length novel being produced using one prompt. And if you wanted a more technical test of reading comprehension, don't forget that GPT4 got 99th percent percentile in the verbal section of the GRE which includes reading comprehension. I managed to get 170 on this test. And to be honest, when I saw GPT4 get 99th percentile, that was a huge wake up call for me. A true story actually is that when I saw that result I decided to make covering AI my full time job. And I think each person will go through their own Such moment when GPT4 or GPT5 or Gemini crosses some benchmark to do with their profession. It's like I didn't wake up much when it could create basic art because I'm not an artist. But when these models come onto territory that you yourself know about, that's when you realize how powerful they are. And I think a lot of people have gone through that journey. This is Metaculus, if that's how you pronounce it, and their predictions about AGI. And look what the predictions were two years ago, around the time of that book. They were late 2000s, even early 2000s and this was as late as October or November of 2021. You can see that they thought AGI would be 20 or more years away. What do they think? Well, 2026. That is quite a change in two years and I can understand why Mustafa Suleiman, the founder of Inflection AI goes further. They are building towards their own AGI and they plan to ask it to go make $1 million. Of course, it would need a strategy. It would need to research and design products, et cetera, et cetera, maybe with a little bit of human approval, but the work would all be done by an AI. And he says something like this could be as two years away. And those of you who watched my previous video on super intelligence will find that quite a contrast to the idea that superintelligence is 10 to 20 years away. I don't know about you guys, but I think if an AI can make money in this way, the entire world economy will be transformed rapidly. There is a curious possibility, though, that such an AI wouldn't be released to the general public. Of course, I was following the ofTC investigation into OpenAI. As you might expect, I read all 20 pages of the investigation, multiple times, in fact. Sam Altman openly grumbled that the FTC request started with this leak, enabling me to read about it. It is actually a crazy document and it feels to me a bit like some sort of strip search. Like they want all internal communications about the propensity of ChatGPT to produce inaccurate statements or reveal personal information. And they say at the top, do not delete any emails relating to this. In other videos, I've covered how any model can be jailbroken. And I've covered how Sam Altman said that it might be two years before models stop hallucinating. So if the FTC ends up finding OpenAI or the other companies billions of dollars as they've fined other companies before, I can see one result being much more reticence from these companies over publicly deploying their models. Of course, let me know in the comments whether you think that's a good thing or a bad thing. Speaking of money, though, I can't leave out that this week was the start of XAI. Elon Musk was able to promise up to $200 million as a signing bonus for some of the AI researchers that joined him. If companies are willing to promise that much to researchers willing to defect from DeepMind or OpenAI, I do find it hard to see the march to superintelligence slowing down much. So let's end with that famous scene from iRobot, although perhaps this time with a different ending. An Imitation of Life Can a robot write a symphony? Can a robot turn a canvas into a beautiful masterpiece? Thank you so much for watching and have a wonderful day.