Is It AI? One Tool to Tell What’s Real with Truemedia.org CEO Oren Etzioni

Hey, everyone, this is Tristan. We have a special episode for you today with computer scientist Oren Etzioni to talk about a new tool to detect AI generated content. As AI technology is getting better, the ability to tell reality from unreality will only grow. Already, deepfakes are being used to scam people, extort them, influence elections. And there are lots of sites out there that claim to be able to detect if a piece of content was created using AI. But if you've used these sites, you know that they're unreliable, to say the least. There are also folks out there who are working to build better tools, people who understand the science of artificial intelligence and want to see a future where we can actually know what's real on the Internet. And one of those folks is Oren Etzioni, who's the founding CEO of the Allen Institute for Artificial Intelligence. And his nonprofit truemedia.org has just launched their state of the art AI detection tool. And I'll just say that here at the center for Humane Technology, we think it's critical not to just point out problems, but highlight the important work that's being done to help address these problems. So I'm super excited to have Oren on the show today to talk about that work. Oren, welcome to your undivided attention.

Thank you, Jose. It's a real pleasure to be here. And as I'll explain in a minute, it's particularly meaningful to have this conversation with you personally.

Well, let's get right into that because I believe you and I first met actually at the meeting in July 2023 with President Biden about artificial intelligence. Could you tell us that story?

With pleasure. I suddenly got an email and it was an invitation to join President Biden, Governor Newsom, some key members of his staff in a small meeting in San Francisco. And the idea was for a few of us to get together and share with him our thoughts and ideas about AI, to give him a sense of what is most important. And I'm probably one of the more optimistic AI folks that you would have on the podcast. So I came in to this small meeting, all kind of, I wouldn't say guns blazing, but all bright eyed. And we had a really interesting conversation. A number of us spoke about moonshot projects, but also about concerns. The thing that is amazing is I came out of the meeting particularly worried about the scenario that you brought up. The concern that you highlighted was the potential of deepfakes to suddenly and sharply affect our society, whether it's economically where there was a sudden drop in the markets with the picture of the Pentagon being bombed, which you highlighted, and other potential economic scenarios. And in the context of meeting with the President, I naturally thought of the political scenarios where we've seen things like the fake robocall from President Biden Hampshire primary. We've seen an example in Slovakia that was released two days before the election. And I became obsessed with the concern. What happens if in the 48 hours, 24 hours before the election, somebody releases a fake that will tip the scale, particularly in our highly contested, very narrowly divided electorate and election that we're having in November.

Let's actually just talk about the tool that you are launching and what the process was actually like to build it because you're a nonprofit and you had to raise independent money to do this, which is kind of one of the interesting things about AI is there's trillions going into increasing how powerful AI is and all the things that it can do, but there's not trillions going into making it safe.

That is very, very true. I think our investment in AI is really quite unbalanced. And I came out of our meeting, as I mentioned, with this concern in my head and I said, okay, let's see what the available tools can do. And the first thing I find out is there basically aren't any. So my first realization was that there's a huge gap in the market. I was very fortunate to be able to meet with Garrett Camp, the co founder of Uber, who funded us out of camp.org, his philanthropic organization. And then we set out to build this tool to make it available to media organizations, to fact checkers, but ultimately to the general public, to any concerned citizen to enable you to do a very simple thing. Take a social media URL from Facebook, from X, what have you TikTok, paste it into our search box and say analyze. Assess whether it contains something that's fake, that's been manipulated by AI or that's real. We have this tool. It's freely available@truemedia.org welcome everybody to check it out.

How actually does this work? Oren, what is the way that you train a model to detect a deepfake?

I think there are two really important points to explain. The first one is just the mechanics. And I want to highlight that interacting with our tool is super simple and you can do it either by taking an image that's on your computer or video, audio, and just uploading it to our site and within a minute or so you'll get our assessment, or you can just take a social media URL from TikTok from Facebook, Instagram, all these places, even we just True Social Trump's network, which has seen its share of fakes. You just paste in the URL into our search box on our homepage@True Media.org, you hit Analyze and you'll get the results back. So the user interaction is very simple. But now let's go under the hood and talk about what happens when you do that. So what happens when you do that is, first of all, conceptually, two things. First of all, we extract the media, then we send it to a whole bunch of vendors. We just send it to them and say, what do you think? Reality Defender, Hive.

So these are existing deepfake detectors, and you're trying to get what's called kind of a mixture of experts, a mixture of detectors. Synthesis.

Exactly, exactly. And want to be as comprehensive as possible in doing that. While they're doing the analysis, we also have our own models that look at various technical characteristics. For example, the distribution of visual noise or areas of blurriness, all kinds of telltale signs that we've developed that assess this. I'll tell you one very cool idea just to give you a sense of how deep this goes. And by the way, we. We don't just analyze the signal. We analyze the semantics. We get transcripts of the videos and assess them. We look all over the web using something called reverse search to say, has this image occurred elsewhere before? Is it a modified version of something that we've seen? So we use a lot of tricks under the hood because there's no silver bullet. We use every trick that we can find to do that. But I want to share with you something that just gives you a sense of how involved this can get. So there's a technology called lip syncing, which is where you take a video of a person, then you lay down a different audio track. So now they're saying things they didn't actually say. And that's really weird, right, because their lips aren't aligned. Right. That's like very bad dubbing in the old days. But now with lip syncing technology, they can actually modify the lips so that it looks like the person. There's a famous example of this with Obama a few years back. It's gotten much better since then. It looks like the person is actually saying what you're making them say. Right. This is terrible. It turns out that there are subtle discrepancies between the audio track and the video track. So actually, Hanif and one of his students, right, he's a Berkeley Professor, a major authority in the field of forensics, had the idea, well, what if we analyze the transcript, right? We record the audio and transcribe it and then we use lip reading software to analyze what the lips are saying. Now, because of these discrepancies, what you see visually with the lip reading and what you hear in the transcript is going to be quite different. And when that's the case, that's a hint that this is fake. So it shows you the creativity and the great lengths that we can go to to try and find this smoking gun that tells you, aha, this is a fake.

Could you talk a little bit about the performance or the sort of accuracy of the existing systems and why the system might be more accurate?

Sure. So first let's, let's clarify what we're talking about very specifically, and that's images, videos and audio. We don't deal with factual questions which can be subject to interpretation or with text which can definitely be faked. That's a whole nother arena. But in these three things, there are not really tools available where you can do this. And actually we've gone even further and put a bot on X, right, where a lot of this stuff is rampant, where you can just tag Mediabot and it'll take what's in the thread, the media that's in the thread, analyze it, and post its analysis in response. So we are democratizing the ability of anyone and everyone to use this technology. But now to go to your question about quality, it's extremely important. So as an academic, I started looking at different models and different vendors and assessing the tools. I very quickly determined that there's no silver bullet here. There's some very strong claims made by different people, and there are also some very high quality technologies. We have a number of partners, including Pindrop on the audio side, Reality Defender, Hive, Sensity in Europe, a number of organizations that do a good job in doing the analysis. The first thing we did is we said, when we get a query, why don't we send it to all of them in parallel to hit their APIs and collect their responses and form an analysis. So we did that. And naturally, when you can consult all the experts simultaneously, you tend to get a better result. We then went further and used open source models, ones from academia, ones that we've developed ourselves, and we fine tuned it on the data that we find in social media. But the bottom line is we sit at comfortably above 90% accuracy, which is very good, but also, you know, full disclosure, error prone. Right. That means if you do 100 queries, 10 of them we can make mistakes on. And we do various things that I could talk about in the user interface to address that, so you don't get the wrong impression of what our assessment says.

So, obviously there's kind of a cat and mouse game, because people remember famously, that when you generated images of people in the last few years, what it was bad at was getting the hands right. So if you looked closely, there was always more fingers on the hands than the. That wouldn't be there on a normal human being. And that's a signal that a human being can detect. But as AI gets better and better, those signals that are visible to human beings go away. And instead you have to look for more of these invisible signals. I'm just curious, was there anything surprising about what you discovered about the signals that a machine can pick up that a human eye or human ear cannot?

Yes. We found that the technology advanced to a very key point, and in fact, now people can no longer tell. Actually, a lot of people think, oh, whether I can see the hands or not, I can squint and glance and I can tell. So we launched a quiz taking social media items, only political deepfakes that have been posted on social media. And we found that people typically cannot tell. The New York Times did multiple quizzes, a very recent one with videos, previous one with faces. When you take these quizzes, you are quickly humbled and you cannot tell. So the fact of the matter is, even in the current state of technology, and as you pointed out, Tristan, it keeps getting better, people are deluding themselves if they think they can tell.

Yeah. And I think it's just so important people to remember that. I remember the days when I would see deepfakes, and, you know, it causes alarm when you see where it goes, but you would always say, but at the end of the day, I can still tell that this is still generated by a computer. And I think in many areas of AI, whether it's, you know, AI capabilities in biology and chemistry and math and science and, you know, generating fake media, we look at the capabilities today and we say, oh, but see, it fails here, here, and here. And then we say, so see, there's nothing really to be worried about. But if you look at the speed at which this is going, we don't want to have guardrails after the capabilities are so scaled up. We want to really get those guardrails in now. So I'm so grateful that you've been doing this work. I thought for a moment what we do is Just set the table a little bit, because there's a whole ecosystem of players in this space and there's different terms people throw around watermarking media, the provenance of media disclosure, direct disclosure, indirect disclosure versus detection of things. Could you just sort of give us a little lay of the land of the different approaches that are in this space? I think President Biden's executive order called for watermarking of media.

So all these terms like provenance and watermarking and others refer to technologies that attempt to stamp to track the origin of a media item. I'll just use image for simplicity and to track changes to it and to give you that information upfront. That's very important technology. And it only has one major Achilles heel, which is currently it's completely impractical. And it's impractical for two key reasons. The first one is, turns out that these watermarks, and there's visible ones, invisible ones, are relatively easy to remove. The second one is, even if somehow we were able to successfully insist on watermarks on all AI generated media, and that's, as you point out, a big if. It makes no difference unless the app that you're using to consume the media looks for it, right? So if it's your browser, if it's your Facebook app, if it's your TikTok app, if it doesn't insist on detecting it, then it doesn't matter, right? Because nobody is going to go through a bunch of machinations. Not nobody, but most people just consume what they're given. They're not going to turn various things on. So unless the app that you're using as a consumer reflects the watermark, it doesn't do anything whether it's there or not. And for that reason, right, if you remove it, nobody would even notice. And if it's there, nobody would notice either. We have to reach the point for this to be practical, we have to reach the point where the way that we consume media tells us always, right? Whether this is fake or real, shouldn't.

It be the case that Facebook and Twitter and even dating apps like Tinder sort of implement this deepfake detection directly into the way that all these products work. Like, shouldn't they just implement true media embedded as checking all the media that's running through it?

Absolutely. If you want assurance that what you see is real, that content that you record isn't stolen basically in various ways, then we need to do something like that. I did have conversations with the major providers, as you'd call them, social media networks, generally speaking, I wouldn't say that they're rushing to do this. They don't have the economic incentive, and that's why legislation is appropriate. Of course, there are free amendment issues and so on. And so starting with certain cases, child pornography is actually one that we've done better on than others. And certainly politics is a key one. You can't just have political ads. We're such visual creatures, right? All kinds of political ads or social media posts that appeal to people's base or instincts, that confuse people, all that without any kind of appropriate analysis and enforcement.

Oftentimes in our work, we repeat the line from Charlie Munger, who is Warren Buffett's business partner, if you show me the incentive, I will show you the outcome. And as you said, the companies don't have an incentive to implement this, especially if it costs them money. And that's one of the questions I wanted to ask you is I'm assuming one of the reasons that they don't just implement deepfake detection is that involves running more servers, more compute. You know, every piece of content that gets uploaded cost them some server cost to, like, process the image, process the Tweet, process the TikTok video, put it onto a server. And what this would involve is doing an extra check that would cost them some extra money. So what is this cost that's getting in the way of that?

There is a cost here, but really, given the tremendous abilities of these organizations, the cost is very small. I think the concern is of a different sort. And you're absolutely right. There are disincentives to do this, but it isn't so much that cost. So what are the disincentives? First of all, this stuff, stuff, kind of the worst stuff, in some sense, plays very well, as you've documented. Right. The algorithms spread that stuff. People click on it. So the first and foremost incentive is revenue, not cost. This stuff makes money. And we have a kind of tragedy of the commons here. They don't see that it's their responsibility to make sure that we have access to the truth or even information about. Right. Something could be true, but still manipulated. Right. It doesn't mean that it's wrong, but they see a responsibility to let the public know this was manipulated by AI.

Yeah, well, I can also see how with the 10% failure rate, if you were to have the tech platforms like Facebook or TikTok, you know, be forced to implement this deepfake detection, and they got it wrong 10% of the time, that would leave them victim to all sorts of rightful attacks. That they're dismissing content that's actually real.

I think the concern about accuracy is a very fair concern, but I also think that having information, particularly educated information, is always a good thing. So I would not suggest that anybody toss anything out, take down requests and so on. That's where we get into First Amendment issues. What I would suggest is adding a label that this seems to be automatically synthesized or this may be suspicious, according to an assessment Created by True Media.org, which is of course a nonprofit, nonpartisan body, or by some other one. And I do think that we can drive up the accuracy above 90%, but it will always make mistakes, which is why I actually think that the most important thing is to when something may be suspicious and it may be due to its origin, it may be due to our analysis or a different tool. You just need to take an extra minute before you Forward it to 100,000 followers. They forward it and it spread so virally. And the biggest thing we can do as this technology plays out is to take a moment and say, wait a minute, am I sure this is true? Where did this actually come from?

You're highlighting a really important point that I really want listeners to get, which is something in our work we call perverse asymmetries. You can create a deepfake 100% of the time. You cannot accurately detect a deepfake 100% of the Time. And to get it to 100% takes years and years and years of research that now you, Oren, have signed up for your organization and nonprofit having to do all this work to get to closer and closer and closer. So wherever there's these asymmetries, we should be investing in the defensive first, rather than just proliferating all these new AI offensive capabilities into society. So, Oren, I think what we really want to do in this podcast is paint a picture of how we get to the ideal future. And, you know, you're working hard on building one tool, but as you said, it's not a silver bullet and it's just one tool of a whole ecosystem of solutions. Do you have kind of an integrated idea of the suite or ecosystem of things that you would love to see if we treated this problem as kind of the Y2K for reality breaking down?

I think ecosystem is exactly the right phrase here because I think that we all have a role to play. I think regulators, as we just saw in California, and hopefully will become federal or in many states, follow California. And then when you have a patchwork of regulations at the state level, Sometimes it's elevated to a federal and even international regulation. So I think that's an important component. It needs to be done. Right, right. There's a balance here of the burden on the companies on protecting free speech rates, but at the same time creating an appropriate world for us, particularly in these extreme harm cases like politics, non consensual pornography, et cetera. So I think that's incredibly important. Once you have those, you need the tools to then detect that something's fake, whether it's watermarking post hoc detection like we do@truemedia.org or a combination of both. You can't have regulations on the book without enforcement. So I think regulation and enforcement go hand in hand. I really hope that in that context the social media companies will step up and realize that we're encountering what I think of as really an Achilles heel for democracy, the combination of AI and social media in a way that can disrupt elections. And I really believe that they can step up more. And then the last and in some ways the most important thing is what we're doing today in this conversation. Raising awareness, increasing media literacy, making sure that everybody exercises common sense that is appropriately, everybody's appropriately skeptical about if you see something, are you sure that it's real? And even today we do have organizations like Reuters, AP News, others who have extensive fact checking organizations. And sometimes we just need to take a little bit of time to make sure that what we're seeing and having an emotional response to really is real. If we work together across all these different elements of the ecosystem, I do think that things will improve and I am worried that it's going to get worse before it gets better.

Well, with that somber note, that was a good note to end on. Thank you, Oren, for coming on your undivided attention.

Thank you, Tristan, and thank you for inspiring me to engage in this work. I'm both inspired, terrified, but also pleased that we are giving it our best shot in 2024.

So before we go, I just want to say this is obviously a massive problem and it's going to take a whole suite and ecosystem of solutions and new tools from things like proof of humanity verification to requiring that we have whole new cameras that have cryptographic signatures embedded within every photo that we take. And these things would require some new laws that are being proposed right now. And I want to give a shout out to the huge ecosystem of people who've been hard at work on this problem for a very long time, from the partnership on AI and the ontology of different sort of issues in the space to the Coalition for Content, Provenance and authenticity, or C2PA, which is a collection of companies that have been working on these issues for a while, non profits like witness.org and companies like Trupik. I want to make sure people go and check out their work because we need all of these initiatives to be successful. And one last thing, please don't forget to send us your questions. You can email us@undividedumanetech.com or tape a voice memo on your phone and then send it to us. One of the weird things about recording this podcast is our little team sits on a Zoom call and we do these episodes and we can't feel the millions of you that are out there for whom you know you want to go deeper into these topics. You have questions, you enjoy certain episodes, you don't like certain episodes, and we really do want to hear from you. So please send us your feedback and your questions and we can incorporate them into a future. Ask us anything. Your Undivided Attention is produced by the center for Humane Technology, a nonprofit working to catalyze a human humane future. Our Senior producer is Julia Scott, Josh Lasch is our researcher and producer and our Executive producer is Sascha Fegan, mixing on this episode by Jeff Sudeikin, original music by Ryan and Hayes Holliday and a special thanks to the whole center for Humane Technology team for making this podcast possible. You can find show notes, transcripts, and much more@humanetech.com and if you like the podcast, we'd be grateful if you could rate it on Apple Podcasts because it helps other people find the show. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.