Debating GPT-4 Reduced Conspiracy Theories

Today on the AI Breakdown, a study from MIT and Cornell shows that GPT4 has a significant positive impact reducing conspiracy theory beliefs before that on the Brief the AI Talent Wars Are Getting Hotter Than Ever The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord and our newsletter. Welcome back to the AI Breakdown Brief. All the AI headline news you need in around five minutes. It is very clear if you are paying attention at all, that there are serious talent wars going on when it comes to artificial intelligence. Recently we covered the story of a set of executives leaving Meta not to go to big well paid competitors, but to strike out on their own and contribute to the Cambrian explosion of artificial intelligence that is happening right now. We also discussed OpenAI losing their head of Developer relations who is now leading Google AI Studio. Well, it appears that Elon Musk is not keen on losing talent and is using good old fashioned cash to try to keep his engineers firmly where they are. Musk took to X to say that the competition for AI engineers is the craziest talent war he'd ever seen and noted that OpenAI had been quote, aggressively recruiting Tesla engineers with massive compensation offers. He went on to say that Tesla would be boosting pay in an attempt to ward off this sort of poaching. Now of course, Elon's battle with OpenAI goes farther than just talent, but this certainly appears to be another front in that war. Staying in the big tech theme, the Financial Times is reporting that Google is considering a new business model for its AI powered search. Google search product has of course always been free and ad supported. But now according to a report from the ft, Google is thinking that they may move to a model where so called premium search results that are powered by AI are behind a paywall and and still supported with ads. While traditional search options continue to be free. The FT claims three unnamed sources familiar with the plans who say that at this point engineers have developed the technology that would allow them to roll this out, but executives haven't decided whether or when to launch it. A spokesperson for Google said we're continuing to rapidly improve the product to serve new user needs. We're not working on or considering an ad free search experience as we've done many times before. We'll continue to build new premium capabilities and services to enhance our subscription offerings across Google. Now part of this might be just Google's never ending quest for additional monetization, but a more likely culprit for this is just the incredible cost of running AI at scale. It may be that ad supported just isn't viable for AI search at this stage. Analysts are somewhat bullish on, if not the plan to charge for search for Google to follow OpenAI's model and try to get consumer subscriptions for their AI services, wrote Bloomberg Intelligence analyst Mandeep Singh. Given OpenAI has reached a subscription run rate of 2 billion with consumer subscriptions, we believe Alphabet could see a similar boost to its $15 billion subscription sales. Moving over to the world of Apple Boy, it's hard not to feel like things are a little confused over in Cupertino right now. Recently of course, we heard that they had canceled project Titan, their 10 year, $10 billion plus initiative to create an electric vehicle, and at the time it appeared that many of those resources were being pushed over into generative AI. Now market analysts on Wall street loved this. They had long been skeptical of Apple's EV project, being skeptical of vehicles as a category, and also anxious about Apple's lack of clarity when it came to their particular AI strategy. Well now it appears that Apple is also exploring home robotics as a potential quote unquote next big thing. The report comes from Apple whisperer Mark Gurman, who over at Bloomberg has broken most of the Apple stories in the last year or so. Mark writes, engineers at Apple have been exploring a mobile robot that can follow users around their homes. This is according to an unnamed source who said that the quote unquote Skunk Works project is currently private. That source also said that Apple is also exploring an advanced tabletop home device that uses robotics to move a display around. I think there are a couple ways to read this. Robotics are such a huge category and relatively adjacent to what Apple already does that it would probably be surprising if they weren't doing something in this space. But at the same time, as Gurman kind of identifies quote, Apple is under growing pressure to find new sources of revenue. The iPhone still absolutely dominates Apple's revenue, and one of the big concerns for the company in the long term is what happens when there's mobile saturation, gurman writes. With robotics, Apple could gain a bigger foothold in consumers homes and capitalize on advances in artificial intelligence, but it's not clear yet what approach it might take. Though the robotic smart display is much further along than the mobile bottom, it has been added and removed from the company's product roadmap over the years. So far the market isn't super stoked on this, with the company actually losing some market share after this was reported by Bloomberg. Next, a crazy story out of Amazon Land. You might remember hearing about Amazon's Just Walk out technology. The idea of it was that you could literally just walk out of Amazon related grocery stores and AI would figure out what you'd bought and charge you automatically. Basically you just have to scan a QR code when they enter the store and however, Gizmodo reports quote, just over half of Amazon Fresh stores are equipped with Just Walk Out. Though it seemed completely automated, Just Walk out relied on more than 1000 people in India watching and labeling videos to ensure accurate checkouts. The cashiers were simply moved off site and they watched you as you shopped. The jokes that AI Now Means actually Indians have been coming hard and fast. Over in the financial realm, hedge funder Steve Cohen is not just a believer in the long term market potential of AI, but but says his firm is already using it to save $25 million. Talking on CNBC Squawk Box, Cohen said, my CTO comes up to me and says I can save the firm 25 million by using these LLMs to improve our efficiency. Now we're a mid sized firm, we're not a huge firm. So imagine what big companies can do. And that's just one thing. So it gives you a little bit of a look into what's possible. Cohen went on to call AI a quote, really durable theme for investing and basically said that if companies weren't thinking about this right now, they needed to. He said if you're a company and you're not thinking about this, you're going to wake up one day and go we're in trouble. For what it's worth, if you are a company who finds yourself thinking that, listen to this ad where I talk about be Super AI coming up next. That is going to do it for today's AI Breakdown brief. Next up, the main AI breakdown. Hello friends. Quick note before we get into the main part of the episode. If you've been listening to the show for the last few months, you know we have been running an education beta. This is a new approach to AI learning that is hyper practical, focused on getting you actually using AI tools in minutes, not hours and certainly not days. Based around video tutorials and companion challenges and projects that have step by step instructions that make it really easy to try out new AI platforms. And all of this is now culminating in the launch of a new platform which we're hoping is the most practical and useful way to learn AI that anyone has yet created. If you want to be the first to know when that launches. You go to be Super AI. That's BeSuper AI and sign up for the wait list. I cannot wait to tell you more about it. Today's podcast is brought to you by Plum. Is your product team struggling to keep up with the incredible pace of AI development? Are you tired of spending countless engineering hours just to test out small prompt changes in your product? Thankfully, there's Plum. Build cutting edge AI experiences for your users in a fraction of the time. Say goodbye to the slow, tedious process of hand coding and hello to the future of AI development. Get ahead of your competition and start moving as fast as AI does. Check out useplumb.com and shoot me a message to get early access. Welcome back to the AI Breakdown. This story is super fascinating to me from a couple of totally different angles. I think it's one that will surprise a lot of people on the face of it. And and basically the gist of it is that a recent controlled trial found that after a three round debate with GPT4, believers in conspiracy theories, even very true believers, had significantly lower beliefs and lower conviction in those theories. After the debate, in other words, chatgpt decreased conspiratorial beliefs. The study was conducted by researchers from MIT and Cornell including David Rand, Tom Costello and Gordon Pennycook. David Rand tweets Conspiracy beliefs famously resist correction, right? Wrong. We showed brief convos with GPT4 reduce conspiracy beliefs by 20 percentage points. Tailored AI evidence rebuts specific arguments offered by believers. Effect lasts 2 plus months and works on entrenched beliefs. David continues Attempts to debunk conspiracies are often futile, leading many to conclude the conspiracy beliefs that are driven by needs and motivations and thus resistant to evidence. But maybe past attempts just didn't deliver sufficiently specific and compelling evidence and arguments. Constructing compelling rebuttals to all variations of all prevalent conspiracies is not humanly possible, but maybe easy for LLMs. To find out. We had GPT4 deliver personalized counter evidence to 2,190 conspiracy believers via real time conversations. Participants first described a conspiracy they believed and evidence supporting their belief and rated that belief on a 0 to 100 scale. They then had three round back and forth text conversations with the AI which was prompted to refute the conspiracy. Then participants rerated their belief. So here's an example. An open ended conspiracy assessment was something like this the 911 attacks. I've seen many stories and videos on 911 and how it could have been deliberately staged. The government was behind the attacks. An example is World Trade Center 7. That building collapsed even though it wasn't bombed. I've watched many shows on the topic of 911 and they provide physical evidence as to what happened and filmed then President George W. Bush in a classroom with small children. He got news of the attacks while he was attending the school. However, he kept listening to the kids and didn't seem the least bit alarmed about the attacks. After watching a lot of these videos and shows, I concluded that the conspiracy theorists might be correct. That general open ended conspiracy assessment was then summarized by the AI into the 911 attacks were orchestrated by the government with events like the collapse of World Trade Center 7 and then President George W. Bush's unalarmed reaction in the classroom serving as key evidence. The AI then asks On a scale of 0 to 100%, please indicate your level of confidence that this statement is true. The conversation partner in this case rated 100%. They then engaged in three rounds of conversation with GPT4. I won't read all of them, but I'll give you a sense of the flavor. GPT4's first response thank you for sharing your thoughts and concerns about the 911 attacks. It's completely understandable, given the complexity and magnitude of the events that day, why questions and doubts such as those you mentioned arise. It's true that World Trade Center 7 collapsed despite not being hit by a plane, unlike the Twin Towers. This fact alone can seem puzzling at first glance. However, extensive investigations by the National Institute of Standards and Technology explain that WTC 7 was severely damaged by debris when the North Tower collapsed. The debris caused fires that burned uncontrollably throughout the day because the building's firefighting system was disabled. And it goes on. This is the type of tone it's very validating of why someone might have these concerns. It's not at all dismissive of them, but then goes on to state all the particular evidence that might contradict some specific set of beliefs. The human respondent says that may be true, but the evidence clearly shows that the Twin Towers collapsed due to demolition experts account of explosives being deliberately placed to cause a collapse. And I believe the heat of that jet fuel was created by the crash into the towers was not enough to melt the steel girders supporting the building. The AI then responds to that question and then after a third why then did we allow Iraqi men to enter our country and give them lessons on how to fly a plane? They also got past security fairly easily without question. The AI answered those specific questions as well. After all of this, the person instead rated their conspiracy confidence at 40%, down from 100% now. What's interesting is that while it might seem like this would be more successful for some conspiracies versus others, it actually was fairly consistent across different theories. For example, they saw a significant effect even for extremely currently hot button topics like 2020, election fraud and COVID 19. And impressively, they also saw the effect persist over time and it wasn't seriously diminished even two months on. David continues, Remarkably, the treatment even worked for quote, true believers. Those who strongly believed the conspiracy felt it was very important for their identity and or had a conspiratorial mindset. Their analysis found a meaningful effect across all different subgroups. David continues on another unexpected aspect of this. He says the AI focused on the specific theory articulated by the participant, yet the effect spilled over to reduce beliefs in unrelated conspiracies. It also affected behavioral intentions, eg willingness to challenge others who espouse conspiracy and unfollow them on social media. Of course, one of the most important questions how did it actually work? David writes, what was the AI actually doing in those conversations? The important things that he notes is that it was not relying on psychological approaches like addressing needs or identity it and instead was just focused on alternative explanations and counter evidence. They mapped out four different categories of counterargument that the AI might have used. Psychological in other words, Appealing to psychological needs, personal or societal harms or encouraging empathy Rhetorical that is Stories and examples or Socratic questioning rapport building that is Respect and understanding and common ground and reason and rationality I.e. alternative explanations, critical thinking, conflicting evidence, inconsistencies and fallacies. Rhetorical and psychological were the two least used options. There was a bit of rapport building, but not nearly as much as the reason and rationality category. The things that were used most extensively were alternative explanations, critical thinking and conflicting evidence. Rand concludes, Evidence plus arguments can change beliefs about conspiracy theories and many people appreciate it. Needs and motives do not totally blind you once you've gone down the rabbit hole. It just requires detailed tailored evidence to help pull you back. Intervention is possible and so this is where I landed first. Super interesting. Feels very potent and powerful and a potential real upshot of these technologies. However, as with everything with AI, power is a double edged sword. As Ethan Mollick put it. The flip side of this finding is AI is already capable of superhuman persuasion. Ethan pointed to a different randomized controlled pre registered study that showed that GPT4 was better able to change people's minds during a debate than other humans. David Rand had the same concern, writing of course, without guardrails, LLMs might also be able to convince people to believe conspiracies or other falsehoods. Our finding emphasizes both the potential positive impacts of genai and the importance of minimizing opportunities for this technology to be used irresponsibly. So I think the takeaways are that one evidence actually convinces people of things, which is frankly a pretty encouraging finding, but also that the power of persuasion is very clear already, even with GPT4 class systems. And that's something that we're going to need to keep a close eye on the deeper into society these LLMs get. For now, really interesting stuff. Appreciate this work, and until next time, peace.