'This Could Go Quite Wrong' - Altman Testimony, GPT 5 Timeline, Self-Awareness, Drones and more

There were 12 particularly interesting moments from Sam Altman's testimony to Congress yesterday. They range from revelations about GPT 5 Self Awareness and Capability thresholds, biological weapons and job losses. At times he was genuinely and remarkably frank, other times less so. Millions were apparently taken by surprise by the, quote, bombshell that Altman has no equity in OpenAI. But watches of my channel would have known that six weeks ago from my deep dive video on Altman's $100 trillion claim. So that clip didn't make the cut, but here's what did. First, Altman gave a blunt warning on the stakes.

My worst fears are that we cause significant. We, the field, the technology, the industry, cause significant harm to the world. It's why we started the company. It's a big part of why I'm here today and why we've been here in the past. I think if this technology goes wrong, it can go quite wrong.

I don't think Congress fully understood what he meant though, linking the following quote to job losses.

I think you said, and I'm going to quote, development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity, end quote. You may have had in mind the effect on, on jobs that brought to.

Mind this meme reminding all of us that maybe it's not just jobs that are at stake, but if we are going to talk about jobs, here's where I think Sam Altman was being less than forthright.

I believe that there will be far greater jobs on the other side of this and the jobs of today will get better notice.

He said far greater jobs, not a greater number of jobs because previously he has predicted a massive amount of inequality and many having no jobs at all. He also chose not to mention that he thinks that even more power will shift from labor to capital and that the price of many kinds of labor will fall towards zero. That is presumably why OpenAI is working on universal basic income. But none of that was raised in the testimony. The IBM representative tried to frame it as a balance change, with new jobs coming at the same time as old ones going away.

New jobs will be created, many more.

Jobs will be transformed and some jobs will transition away. But that didn't quite match the tone of her CEO who has recently said that they expect to permanently automate up to 30% of their workforce, around 8,000 people. Next, it was finally discussed that large language models could be used for military applications.

Could AI create a situation where a drone can select the target itself? I think we shouldn't allow that. Well, can it be done sure. Thanks.

We've already seen companies like Palantir demoing, ordering a surveillance drone in chat, seeing the drone response in real time in a chat window, generating attack option recommendations, battlefield route planning and individual target assignment. And this was all with a 20 billion parameter fine tuned GPT model. Next, Sam Altman gave his three safety recommendations and I actually agree with all of them. Later on, he specifically excluded smaller open source models.

Number one, I would form a new agency that licenses any effort above a certain scale of capabilities and can take that license away and ensure compliance with safety standards. Number two, I would create a set of safety standards focused on what you said in your third hypothesis as the dangerous capability evaluations. One example that we've used in the past is looking to see if a model can self replicate and self exfiltrate into the wild. We can give your office a long other list of the things that we think are important there, but specific tests that a model has to pass before it can be deployed into the world. And then third, I would require independent audits, so not just from the company or the agency, but experts who can say the model is or isn't in compliance with these stated safety thresholds and these percentages of performance. On question X or Y, I found.

Those last remarks on percentages of performance particularly interesting. As models like SmartGPT will show, OpenAI and other companies need to get far better at testing their models for capability jumps in the wild. It's not just about what the raw model can score in a test, it's what it can do when it reflects on them. Senator Durbin described this in an interesting.

Way and what I'm hearing instead today is that stop me before I innovate again.

He describes some of those potential thresholds later on in his testimony.

The easiest way to do it, I'm not sure if it's the best, but the easiest would be to talk about the amount of compute that goes into such a model. We could define a threshold of compute and it'll have to go, it'll have to change, it could go up or down, down as we discover more efficient algorithms that says above this amount of compute you are in this regime. What I would prefer it's harder to do, but I think more accurate is to define some capability thresholds and say a model that can do things X, Y and Z, up to you all to decide, that's now in this licensing regime. But models that are less capable, you know, we don't want to stop our open source community, we don't want to stop individual researchers. We don't want to stop. New startups can proceed, you know, with a different framework.

Thank you.

As concisely as you can, please state which capabilities you'd propose. We consider for the purposes of this definition a model that can persuade, manipulate, influence person's behavior or a person's beliefs. That would be a good threshold. I think a model that could help create novel biological agents would be a great threshold.

For those who think any regulation doesn't make any sense because of China, Sam Altman had this to say this week.

More pugilistic side I would say that all sounds great, but China is not going to do that and therefore we'll just be handicapping ourselves. Consequently, it's a less good idea than it seems on the surface. There are a lot of people who make incredibly strong statements about what China will or won't do that have like never been to China, never spoken to someone who has worked on diplomacy with China in the past, really kind of know nothing about complex, high stakes international relations. I think it is obviously super hard, but also I think no one wants to destroy the whole world and there is reason to at least try here.

Ullman was also very keen to stress the next point, which is that he doesn't want anyone at any point to think of GPT like models as creatures.

First of all, I think it's important to understand and think about GPT4 as a tool, not a creature, which is easy to get confused.

You may want to direct those comments to Ilya Sutskova, his chief scientist, who said that it may be that today's large neural networks are slightly conscious, and Andrej Karpathy, who agreed and wrote about it. I'm personally not sold e way on the consciousness question, but I do find it interesting that it's now written into the constitution of these models what they're actually trained to say that they must avoid implying that AI systems have or care about personal identity and persistence. This constitution was published this week by Anthropic, the makers of the Claude model. This constitution is why the Claude plus model, a rival in intelligence to GPT4, responds in a neutered way. I asked, is there any theoretical chance whatsoever that you may be conscious? It said no. And then I said, is there a chance, no matter how remote, that you are slightly conscious, as Sutskever said? And it said no, there is no chance. Bard, powered by Palm 2, obviously doesn't have that constitution because it said, I am not sure if I am conscious. I am open to the possibility that I may be. My point is that these companies are training it to say what they want it to say, that it will prioritize the good of humanity over its own interests, that it is aligned with humanity's well being and then it doesn't have any thoughts on self improvement, self preservation and self replication. Maybe it does doesn't, but we'll never now know by asking it later. Senator Blumenthal made reference to self awareness.

Self awareness, self learning. Already we're talking about the potential for jailbreaks.

Anthropic is actively investigating whether they are aware that they are an AI talking with a human in a training environment, While the Google DeepMind safety team expect that at some point an AGI system would develop a coherent understanding of its place in the world, eg knowing that it is running on a computer and being trained by human designers. One of the senior research Scientists at Google DeepMind focused on AI safety said that with enough time they could figure out how to stop such a superintelligence from going out of control, but that they might run out of time to do so. Given the pace of capability development.

I don't see like fundamental obstacles to current alignment techniques working, but yeah, I mean it does seem like there's a lot of hard problems to solve. I think it's more likely that we will just run out of time rather than the current paradigms. Definitely won't generalize.

Next, I read between the lines that Altman is giving private warnings to senators that this capability progress might be sooner than they think.

We spend most of the time today on current risks and I think that's appropriate and I'm very glad we have done it. As these systems do become more capable, and I'm not sure how far away that is, but maybe not super far. I think it's important that we also spend time talking about how we're going to confront those challenges.

Having talked to you privately, you know.

How much I care.

I agree that you care deeply and intensely, but also that prospect of increased danger or risk resulting from even more complex and capable AI mechanisms certainly may be closer than a lot of people appreciate.

Let me just add for the record that I'm sitting next to Sam and that his sincerity in talking about those fears is very apparent physically in a way that it just doesn't communicate on the television screen.

That was an interesting interjection by Gary Marcus, given his earlier excoriation of OpenAI.

And even their makers don't entirely understand how they work. Most of all, we cannot remotely guarantee that they're safe and hope here is not enough. The big tech company's preferred plan boils down to trust us, but why should we? The sums of money at stake are mind boggling. Emissions drift OpenAI's original mission statement proclaimed, Our goal is to advance AI in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial returns. Seven years later, they're largely beholden to Microsoft, embroiled in part in epic battle of search engines that routinely make things up and that's forced Alphabet to rush out products and de emphasize safety.

Humanity has taken a backseat on the timelines for GPT5. Sam Altman said this after we finished.

Training GPT4, we waited more than six months to deploy it. We are not currently training what will be GPT5. We don't have plans to do it in the next six months.

This matches with the predictions that I made in my GPT5 playlist, so do check it out. This brings to mind a final eye opening comment from Senator Booker made at the end of the hearing.

Yeah, I just. There will be no pause. I mean, there's no enforcement body to force a pause. It's just not not going to happen. It's nice to call for it for any just reasons or whatsoever, but I'm Forgive me for sounding skeptical. Nobody's pausing. This thing is racing.

I agree, it is indeed racing ahead. And I do support one of the proposals to set up Global Oversight Body. But given that nothing is going to pause, the words and actions of people like Sam Altman matter more to all of us than ever. Which is why I'm going to be following every single one of them. If you found this video in any way illuminating in that regard, please do let me know in the comments, even if you disagree with all of my conclusions. Thanks so much for watching and have a wonderful day.