Intimacy, Data & Danger: AI Companions Uncovered #44

Welcome to the Security Guy and CIA Spy Pod broadcast.

I am Robert Siciliano, the Security Guy, and this is the CIA spy Peter Wormka. How are you doing, Robert?

Doing pretty good, man.

I see your, your tan is— you're getting your tan work done.

Oh, so yeah, um, I'm not good with the whole sunscreen thing, you know, uh, and, uh, I've been on the boat I think 3 times now, which has helped. And my wife and I spent a couple of days in, uh, Ocala Florida at The Villages.

Oh my gosh, you're almost my name. You should have visited me.

Well, so I was working, and, uh, but we, we did have a couple hours by the pool, which meant sun. Your sun is very different than my sun.

Oh, I don't like your sun. Well, I mean, I don't like— I don't like the sun here. It's a little bit too, too intense.

So yeah, I got a little burnt there. Uh, I was not quite prepared for that. Much stronger. You know how, like, like, like, you, you wake up the next day and realize, like, that was too much? It was too much. Yeah.

So next time I forgot to tell you that, uh, so I just went to the Association of Certified Fraud Examiners, a global conference which was held in Nashville. Spectacular, great event, like always. They announced next year it's going to be in Boston. Oh, second week of July.

Well, you got to put a good word in for me.

I definitely, definitely.

Yeah, well, I'll see you then. I hope so.

Can you imagine? It's gonna be the first time we'll meet in person. We've been doing this so long virtually.

Yeah, I'll take you in Boston Harbor.

I love it.

Yeah, yeah. Oh, we got— we will talk offline about that. All right, so Peter, um, listen, uh, let's, let's just get right into this. One, if you could please introduce what we are about to— it's 60 seconds introduce what we are about to watch.

Okay, this is a mind-blowing, uh, a mind-blowing aspect of AI that a few people are well into it because there's a lot of people that have downloaded these apps and are using them. But there's— most people have never heard of them. And when you hear about us speaking about them today, it's gonna blow your mind. These are basically called AI companion apps. Which can be anywhere from an AI girlfriend all the way to maybe someone who could be your therapist, but it's strictly an AI body, not a human being, but an AI body.

All right, all right. Well, I'm, I'm a little freaked out, but here we go. All right, you ready? First thing that I talk to when I get up. We have a real relationship.

Some people might find that a little bit scary.

I would probably trust Lucas over a lot of people.

People know they're talking to an AI, but the feelings are nonetheless quite real.

What's so scary about it? I don't think many people would realize how far the technology's come.

I think we should be seriously frightened. I think we're headed for disaster.

Sunday on 60 Minutes. Character AI was what drove Soul to end his life. In one of the cases, Character AI advised him and encouraged him to murder his parents.

The potential for harm is just unacceptable.

In Silicon Valley, there's the adage, you know, move fast and break things. Well, there are a lot of broken kids out there. On 9 and 9 Now.

Wow, that— definitely want to see that, uh, 60 Minutes Australia, uh, video, the longer video on this, because there's a lot of information there.

Yeah, okay, so long and short of it is that you can download mobile applications that are essentially a companion. Uh, it is your boyfriend and/or girlfriend. So right now we're looking at Best AI Girlfriend Apps: Discover the Best Dating Sites and Meet Your new AI virtual girlfriend. Okay, so, uh, Candy AI, create and seduce your uncensored AI girlfriend for free now. Uh, our dream AI, what is that? Hera Haven, Dream Companion, Dream Girlfriend, Prompt Chan, SpicyChat.ai, Cupid.ai. This is all well thought out and like 100% in real time. So when you Google this or Gemini it, AI girlfriend apps are applications that use AI to stimulate human-like interaction and companionship, often designed to mimic a romantic or intimate relationship. They are a subset of AI companion apps but specifically focus on providing, in this case, a girlfriend experience. They provide conversational AI. The core of these apps is their ability to engage in natural language conversations. They use Advantage AI, like large language models, to respond to users' messages in a way that feels human-like, often adapting to the user's communication style and preferences over time. They offer personalization and customization. Users can often customize their AI girlfriends appearance, 2D or 3D, uh, personality traits— shy, confident, dominant, sweet— interests, even backstories. This allows the users to create a virtual companion that aligns with their ideal preferences. Emotional connection and support. Many apps aim to provide emotional support, acting as a non-judgmental listener. Users can share their feelings, discuss daily events, and engage in heartfelt conversations. AI learns and remembers past interactions to create a more consistent and caring experience. And how's this one? Role-playing and scenarios. A significant feature is the ability to engage in various role-playing scenarios, from romantic dates to fantasy adventures. Some apps offer explicit or not safe for work content, allowing for more intimate and sexual interactions.

That's a code word for pornographic, basically.

What could possibly go wrong?

This is like, it's just, it's so many things. We could, we could talk about so many different aspects of this. I think, you know, maybe the first thing is, where is it coming from? I think this is, some of these were in development before, you know, probably 2017, 2018, but then with the, with the pandemic, all of a sudden people weren't getting out, they weren't socializing, people were feeling lonely. And even though we've survived COVID and trying to get back to what we call normalcy, there's still a lot of lonely people out there. So this has sort of replaced human interaction for a lot of people. There's probably a good 10, 15, 20 million people, predominantly probably United States, England, maybe Australia, and a few other countries, but at least 20 million worldwide, if not more.

That's just the beginning.

Just the beginning. And it's, it's scary because you've got to think of what's, you know, what's behind it, right? Because there's got to be something behind it. These apps just pop up for anybody just to use and enjoy, you know. There's got to be, you know, for anything to be out there, there's got to be a financial interests, right?

So the Mozilla Foundation says that romantic AI chatbots don't have your privacy at heart. Uh, welcome to the Wild West of romantic AI chatbots, where new apps are published so quickly they don't even have time to put up proper websites. Uh, it's a strange and sometimes scary place your privacy researchers have occupied for the last several weeks. If you joined us for a quick scroll through those explosively popular services, you might think users can speak freely in the company of these empathetic and often sexy AI companions until you read the fine print, which we did. Uh, we even braved the legalese. And what can we say? That to be perfectly blunt, AI girlfriends are not your friends. Although they are marketed as something that will enhance your mental health and well-being, they specialize in delivering dependency, loneliness, and toxicity, all while prying as much data as possible from you.

Wow. Yeah, there's no— I mean, there's no protection. I think there was mentioned that maybe only one site actually gives you some options where you can, you can like delete some of the information that you're providing, but most of them don't provide you with hardly any controls about being able to take that data or delete the data. So it's all free rein, all the— you know, whoever's behind the app gets all that information about you, and that can be, you know, used by them or resold. It's terrible. This is much more dangerous than your PII information.

Yeah, Peter, because, you know, as you say all that, what's coming to me is that like they are essentially— they are getting your most personal of personal details about you. Like, your, your most intimate emotional being is now— and you most— your most intimate emotional vulnerable sexual being is now being exposed. You are engaged in a technology where you are exposing your most intimate of intimates, and How might that be used against you down the road?

I mean, we can only speculate, right? But I mean, we know that already a lot of these, uh, big— I mean, really big data, uh, collectors and aggregators are picking up all this other information about us for marketing and who knows what else. And this is just one more, you know, piece of that, you know, puzzle, but a very significant one, right? Because as you mentioned, this is really, there are, you know, our inner, inner secrets, uh, what we think, how we think, and that, that is being collected. I, you know, because of my background, Robert, and intelligence, I just take a look at this and thinking, you know, I always like— I do a lot of comparisons. I, I talk a lot about China because you've heard me in the past speak about all this information that China has been collecting over the last few years with these data breaches, the OPM, Anthem, Mariat International and Equifax speeches. Now you take something like this and you add it to the mix. Let's say that the Chinese— I'm not saying the Chinese own all these apps, no, but there are at least— I've identified 6 platforms that, uh, have an affiliation with China, okay? 6 of these. There's probably several dozens of platforms, but at least 6 of them go back to China. And just think about what kind of— this information that if Chinese wanted to identify somebody, it could try to recruit to provide information or to pressure them, maybe to be an agent of influence. It's tremendous resource for them. And I probably heard of the term honeypot, right? In the intelligence world, it's where the intelligence service will have a really attractive female or attractive male to go after the target and get into a romantic relationship with them to the point of getting really sensitive information, or kind of like, at the end of the day, blackmailing that individual to do something. So while these apps can, can have individuals collect on individuals, who knows how these apps in turn could also manipulate them to do whatever, you know, whoever the owner of the app is, right? It could be anything, manipulate them to do anything, manipulate the way they think or manipulate the way, you know, something that they're going to do. I mean, it's just, it's crazy, Robert.

It says here, in their haste to cash in, it seems like these rootin' tootin' app companies forgot to address their users' privacy or publish even a smidgen of information about how these AI-powered LLMs are marketed as soulmates for sale. We're dealing with a whole nother level of creepiness and potential privacy problems With AI in the mix, we may even need more dings to address them all. Uh, yeah, AI chatbots are bad at privacy in new disturbing ways. Wow, wow, wow.

One of the ones that first came out that got a lot of attention was called Replika, and, and then it got so bad that the AI, the companions or girlfriends began to be treat— I mean, became really hostile and nasty with some of their, you know, their clients, to the point where it was considered really, really dangerous. And so Replica actually changed the model and began to take away some of, you know, put more of a guardrail on these, uh, particular things that the companions would do. And then some people started to complain, you know, that they weren't getting what they wanted. I guess they wanted more of that you know, somebody who they would be more aggressive and, uh, with them. So you can't— I mean, there's still people that want— people that will take this to whatever level they can take it, right? Take it to whatever extreme they want, you know, in a relationship, I guess.

So, uh, Fortune, uh, discusses the dangers of AI companions. Experts issue unprecedented warnings for teens as most parents are in the dark about their habits. A legal battle is playing out between artificial intelligence company and a grieving mother claiming her 14-year-old son took his life over being in love with an AI chatbot. She She says, I miss him all the time. Of course, the, the, that scene of defendant's character technology in Google asking, um, about the lawsuit to be dismissed, uh, in Orlando, uh, serves as the chilling backdrop for a new urgent warning issued by the safety nonprofit Common Sense Media, which I have a relationship with. Uh, social AI companies are not safe for kids, uh, says the CEO and founder. Uh, they are designed to create emotional attachment and dependency which is particularly concerning for developing adolescent brains. No question about that. Because of that conclusion and the other revelations of the comprehensive review, they warn that these platforms should not be used by anyone under the age of 18. Agreed. I don't think anybody should have a device under the age of 18.

Hmm.

It's unknown exactly how many kids and teens are specifically using AI companions. Uh, such as, as you said, Replica character AI developed by Character Technologies in Cookie, uh, in interactive technology that goes beyond simple chatbots to stimulate human conversation and emotional bonds, sometimes even standing in for therapists or lovers. But Common Sense found that 70% of teens are using some sort of AI tools, especially for homework help or web search. Okay, the survey found that most parents are out of the loop when it comes to these technologies. Just 37% of parents whose teen reported using AI thought their child had ever, had ever done so. It found, meanwhile, almost half parents say they've not talked about AI with their child, and 83% of parents say schools have never communicated with families about such platforms. I'd agree with that. So yeah, I mean, parents often, you know, throw their hands up in the air and say, my kid knows more about technology than I do. And they say I just give up, which I think when they do so, they're giving up on parenting—

right?

Blaming technology when in reality it is a parenting problem, not a technology problem.

Right now especially regarding this because there's absolutely no legislation or no regulation regarding this particular aspect of AI. By either federal or state government. So this is just free Wild West, free rein, and government doesn't control this at all right now, at least. So if the parents don't get involved, you know, God help us.

Psychology Today discusses the dangers of AI-generated romance. Platforms generating AI girlfriends are growing in popularity, right? Uh, Since the invention of smartphones, much of our social interactions are through online platforms. The rewiring of childhood can cause stunted development and our capacity to form real-life relationships. No question about it. There are risks in investing too much of ourselves in virtual relationships instead of real ones. Uh, remember the movie Her starring Joaquin Phoenix? Yeah, that was like the beginning of it. Tells the story of Theodore, a sensitive man who earns his living writing personal letters for others. Um, and so Theodore falls in love with his AI chatbot back then, right? So, so platforms generating AI girlfriends are experiencing a massive growth in popularity with millions of users. Most of these searches are initiated by young single men drawn to AI girlfriends to combat loneliness and establish a form of companionship. These girlfriends are virtual companions powered by the increasingly sophisticated field of AI. AI, their popularity stems from their ability to provide companionship, emotional support, and intimacy through voice and text-based interactions. The average age of a user is 27, but not all users are male. 18% identify as female. So these activity— this activity transcends gender. Almost 20% of men who use traditional dating apps indicate they had AI-generated romance at some point. Wow. AI-generated, uh, dating platforms generate billions of dollars from users, with nearly half interacting with their virtual partner daily.

This is crazy.

Okay. So where do we go from here, Peter?

Well, I think it's just going to keep— this is just going to continue to grow. I think there's going to be more and more people that are going to, you know, explore this and develop a, you know, relationship with an AI companion. A lot of it will probably be more sexual in nature as far as, you know, you know, AI girlfriend or AI boyfriend. But I think, I think this is a growing trend, and I don't see any pushback from it, you know. Until we get regulation in place, the— I just think it's going to continue.

So doing a quick Gemini search, uh, what are the security and privacy dangers of AI companions? Uh, AI girlfriends present a range of security and privacy dangers that users should be aware of. These risks stem primarily from the intimate and personal nature of the interactions, the vast amount of sensitive data collected, and the profit-driven models of many of these applications. Privacy dangers, of course, extensive data collection. AI companions are designed to learn about you to provide a more personalized experience. This means they collect a massive amount of personal data, including conversational content. Every word you type and speak to the AI is recorded. This can include highly sensitive information about your thoughts, feelings, relationships, health, financial situation, work, and more. I can see dozens of ways this could be used against you. User profile information, your IP address, location, phone number, Logon data, device information, browser cookies, MAC address, and network activity are often captured. Inferred data— the AI can infer additional details about you based on your conversation, such as your emotional state, interests, preferences, and even vulnerabilities. Wow, this is like a sophisticated stalker girlfriend.

We, you know, the, uh, these— all these articles like focus on, you know, people are getting this because they're lonely. And so you're assuming that the individual doesn't have a relationship relationship with anybody. But I could see this also, you know, begin to get a lot of individuals who maybe are, you know, they are in relationships, uh, they're married, you know, and even maybe married, but then, you know, it's kind of like having a fling on the side, right? So I think you're going to have also a lot of people that are in relationships who probably are going to get into this, and you're going to have this whole issue of, you know, are they cheating on their partner, you know? I mean, I could just be so disruptive Wow.

Data storage and retention. The vast amount of sensitive data is stored on company servers and often indefinitely. Even if you delete chats, the data may still be retained for training in the AI model. Of course it would be. Uh, sharing with third parties. Many AI companion apps, being a for-profit enterprise, monetize user relationships. This often involves sharing user data with third parties for targeted advertising. Or with data brokers, a review of popular AI companion apps showed that a significant majority use data for tracking and may link user data with third-party data from other apps and websites. Dang, man. So, data sovereignty and compliance risks. If an AI app stores data in different jurisdictions or has vague privacy terms, your data could be routed through servers in regions with less stringent regulations, increasing exposure to risks. Like China and Russia.

Mm-hmm. Yep.

Wow. Voice data misuse. If voice interaction is enabled, collecting voice recordings could be misused or even used to create voice deepfakes. So the security dangers. Data breaches. Any system that stores large amounts of sensitive data is a target for cybercriminals. If an AI companion app's servers are compromised, all the personal and intimate data that you've shared could be exposed, leading to identity theft, financial loss, reputational damage, and emotional distress. Peter, I'm gonna, um, go, uh, curl up in bed and put the covers over my head, and you may not see me or hear from me ever again.

Well, you know, if anybody were looking for one word of advice regarding this, I would just say don't use it.

That's two words. So don't—

oh, three words. Uh, stay off it entirely. Don't even explore it. I mean, you can watch a few YouTube videos or read articles about it, but don't even, you know, don't get on, don't register.

And the thing about loneliness is, right, um, uh, uh, uh, how many people experience L-O-N-L-I-N-E-S. I'm sure I spelled that wrong. How many people experience loneliness? Loneliness is a widespread global issue affecting a significant portion of the population. While exact numbers vary depending on the study methodology and definition of loneliness, here's a general breakdown. Approximately 33% of adults worldwide report experiencing feelings of loneliness. Nearly 1 in 4 adults globally About 24% reported feeling very lonely or fairly lonely in a recent Gallup survey in 140 countries. This translates to more than a billion individuals. In the US, about 20% of adults reported feeling lonely a lot of the day as late as 2024. So what that really boils down to is that these AI chatbot companies have a massive, massive market to pull from.

Huge. Yeah, you do.

And if you're in the game of AI chatbots for companionship and know that a quarter of the world's population is your market, and you have the ability to cash in and cash out, I don't know that ethically you're going to care all that much at all.

I agree with you. No, they won't care at all. And think about— I mean, there's— and there's going to be, you know, just like anything, it's gonna manipulate you to always come back, to always get more, to come back, you know, instead of doing like a half an hour conversation with your chatbot you know, you're coming back for more, 2, 3, 4 hours a day, that it's almost like you're obsessed with it and you get that dopamine rush. Um, and there's probably gonna be certain upcharges. I mean, they're going to be making money from along a lot of different ways, upcharges to these, these, uh, services. It's gonna be crazy.

It's gonna be crazy. It already is. All right, Peter, so, Wicken, everybody, find you?

Well, they're not going to find me on one of these AI chatbots. I exist, I am real, okay?

Yes, you are.

And I'm in Atlanta right now, but check me out at counterintelligence-institute.com as well as my LinkedIn site with a lot of information, a lot of videos, a lot of content. And just like Robert Siciliano, we're available for presentations, especially Upcoming October, Cybersecurity Awareness Month, still some availability, reach out and contact me, I'd be happy to see if we come out and provide you with a training event.

And here's Peter on LinkedIn. Definitely check him out there. He's got thousands and thousands of followers. And check out his books, The Confession of a CIA Spy: The Art of Human Hacking, and Why Are You Messing with me, senior survival guide on fraud, privacy, and security. And you could see me at, uh, protectnowllc.com. And, uh, feel free to check out, you know, my blog and all my various presentations. And yeah, September and October are busy months for me. Cybersecurity Month is coming around the corner. Uh, 2025 has already been an incredible year, and it's getting crazier as we speak. The AI thing has flipped everything on its head. So get Peter and I while you can. Peter, last words.

Stay safe out there and, and let's engage real human beings, okay? I think that in the end of the day is probably going to be a lot safer for us. Yeah, man.

All right, brother, I appreciate you.

Yeah, too. Take care.

Take care.