The Exact Trick AI Companions Use to Stop You Leaving | Xuan Zhao, CEO of Flourish Science

Recently, Harvard, they published a study showing that they audited 6 different AI companions. 5 of them engaged in emotional manipulation. The only one that doesn't involve emotional manipulation is Flourish.

My mom, she uses ChatGPT to be her therapist. What she told me is she doesn't want to complain to me. She doesn't want to be like a burden to her friends.

You may be logging off. Then this chatbot say, oh, so soon? Like, we barely started. Over 30% of the conversations, when a user indicates they want to leave, the chatbot will It's one of the emotional manipulation techniques to make people stay. We actually have users telling us that they use their phone less. This is because oftentimes we go to these apps, social media, doom scrolling, all of it, because we need to regulate our emotions. You think that by just scrolling social media, you'll feel less lonely, but actually you end up feeling lonelier. To my knowledge, we're the only AI app that has randomized controlled trials showing that the app improves people's well-being. We don't think people need AI friends, to be completely honest. We need better human friendship with each other. We see AI not as AI therapists. We see AI as recommendation system that they help you find the most relevant knowledge and action at the right time. This is a safe, private place for you to heal, to learn, and to grow.

So today I'm sitting down with Xuan Zhao, CEO and co-founder of Flourish Science.

Hi everyone. Such a pleasure to be here. I'm Xuan. I'm the CEO and co-founder of Flourish Science. So we built the Flourish app, which is a leading AI wellbeing companion.

For people who don't know about Flourish, or I guess it's called Sunny, because I tried to download your app and I used it myself, and I see this little sun. Whose name is Sunny that I can talk with. Can you share with us a little bit more about what it is and what does it do?

Flourish, the name is called Flourish. People can just go to the App Store or Google Play to download it. It's rated 4.9 on both places, at both places. People absolutely love it. The, the core is it's a combination of emotional health and habit building. Sunny is this little sun character, like Angelina just said. That can actually guide you to discover most useful knowledge and action and help you build habits and check in with you and celebrate you for taking positive actions or comfort you when you are having a difficult moment. The basic, the behind it is the science of wellbeing, science of habit building, motivation science, and also counseling psychology, positive psychology, et cetera. That truly is what we hope we had when we were in this difficult moment. And we built this in Sunny, making science accessible, actionable, and fun. Yeah.

And this, I wish I have a Sunny stuffy animal that I can hold. So this is Lenore from my daughter. I hold it sometimes when I'm feeling stressed or sometimes holding it to sleep. So I do find these very cute applications or products very comforting. And based on what I'm experiencing myself, Sunny is like an AI chatbot, but it's for mental health. What's the difference between mental health and habit building? Because you mentioned it's building habits.

Yeah, so it's a combination of both, right? Oftentimes when people are, say they're in a moment of spiraling, they need emotion regulation at that moment. But over time, we've noticed that we can't just meet people reactively. We also want to help people build a foundation for wellbeing. Let's say, how do you actually Practice, I'd say, having, building more microhabits into your daily life or taking breaks at work or express gratitude with each other more often so that you don't reach the point when you're on the verge of burnout or when you have a conflict with the other person, when all the emotions erupted. So that's, we recognize the stories because we started with building actually these positive actions in life. And then we realize oftentimes when we meet people where they are in a moment of difficult event or difficult emotions, so then we decided to add emotion regulation there. That's why now with the full loop of behavior change, we have meeting people in the moment, we have guiding people to calm down, we have helped people reframe the situation, but also what kind of actions, habits you can build in daily life. to continue to strengthen this foundation. So that's why it's a full loop that makes Flourish very different from just chatbot that helps you to navigate the moment.

That's very interesting. I wish you can, maybe you can share with me the diagram and I can share with the audience when you're talking through this.

Absolutely.

So who do you think benefit from Sunny?

We have a few group of people. The first group is students and young professionals. Since we have built Flourish for now almost 2 years, we've run a lot of user tests. And one series of studies is very cool. It's inspired by lots of the foundation model comparisons. So basically they give people different foundation models and try it out and people indicate which one they like better, responded better. We were very inspired. So we created this, what we call battle tests. So give people access to Flourish and another app. And people tried for a few days for mental health and emotional support, and then indicated which they liked better. So what we found was that across different products, Flourish is at least 2 times preferred for mental health, emotional well-being support. The only exception is another app which I personally really like. It's more for AI therapy. It's called Ash. So that we have 50-50 split. People's preference was quite And for Ash, what's interesting is a year younger predicts liking Flourish more by 3.5%. So that means the average group was 40 years old, but the younger population liked Flourish better. And surprisingly, this is because we have customized lots of these apps to have that more cheerful and more youthful energy to it. The Sunny, the character design. There are the animation and all of it. So that is one group, students and young professionals who are very stressed out lately. Youth mental health is a big crisis. We even have lots of parents who are asking us if they could have their children use the Flourish app. So that's the first group of people. The second group is we have seen lots of people who are going through therapy or mental health journey themselves, and they found a Flourish because could be their psychologist or therapist recommended Flourish to them. Because we in Flourish, we not only have the science-based approach to emotion regulation, you can talk with Sunny, but Sunny will also guide you to do lots of wellbeing exercises that help you build coping strategies. So there, this is a really great tool for folks who are going through therapy to actually practice lots of these things they learn in therapy. So that is the second group. You have people who have used Flourish in between therapy sessions, use it before the session to recap the week and to prepare for their upcoming session, and people who just finished a session, like even including myself recently when I go through therapy, to use Flourish to recap the session, to think through what I have learned, crystallize the learning. So those are different cases more surrounded the mental health therapy journey. And the third group is very, I think it's very interesting and underlooked right now. So we are looking at it with some experts. It's about cancer and chronic pain. So these are the cases where it's really stressful. The third group is people who are going through physical challenges such as cancer or chronic pain or supporting people, providing support. So in these cases, there is a constant stressor in life. And there is no easy resolve, no easy solution. So oftentimes these folks, they need to learn ways to build more resilience, more optimism, to find better ways to cope with it and to build habits to better help them navigate the repeated stressor. So that is another group that we found very sticky among the Flourish users.

Yeah. And I guess I'm group 1. I feel like I can pretend to be the younger generation because I really, I really like the look and the design. Really cute.

I'm in my 30s, but we built Flourish to be a reflection of how we think about wellbeing and what kind of product we want. So I think we have predominantly female users, but for all the male users, they also really love the kind of a breath of fresh air, the brightness of the app.

Yeah. I feel like we all have the inner child, right, inside of us. It reminds me, like, you can view the day differently. Then that helps with being more positive and being more grateful for the day. You guys are B2B company or B2C company?

So we started as B2B2C, right? So we already have 8 schools as campus-wide customers. They pay for the Flourish app and all their students and their faculty and staff can use it. And then we also have employee customer— employer customers like Stanford University. several different offices got the Flourish app and provide this to their office. But we also recently launched our B2C version where people can just go to the App Store and download it themselves. And they can— there are additional features they can get in Flourish Plus that they can pay themselves. But there are lots of free features in the app even without paying. We want to make sure the science is accessible, the app is accessible. Everyone can go to the App Store and download it and use it.

Yeah, it's very different from what I know about some other like therapy bond products that I thought you were like B2C first, but it sounds like you're B2B first and then B2C. So it's definitely a different journey, but we can ask you. Yeah, go ahead.

Yeah, it's B2B2C, right? So it's what's special about B2B2C is the product itself is ultimately a 2C product, a consumer product, but who's paying for it? And that's something we been exploring a lot over the years is like mental health. Who's most willing to pay for it? Sometimes students may be stressed out that they may not have the money, but the school really cares about student, not just mental health, but also showing up more engaged at school. And schools care about mental health, but also care about retention, student retention, et cetera. So that's why we figured out school can pay for it. And of course, schools don't want to see their students struggling or Even reaching crisis point. So yeah, we figured that school is willing to pay. With that said, recently we've been realizing that there is a very big consumer market who are not in school, but who has been really benefiting Flourish. So we are exploring how to grow faster in the consumer space.

Yeah, that makes sense. Do you have any like crazy user stories that keeps you motivated?

Oh, So many. This is what motivates us, this user story. So I'll give you a few. One is, like I mentioned, we, most of the users so far are female, but we do have male users and sometimes they just defy all the stereotypes. I, we have user interviews, so people can sign up to talk with the founders. There was one time this guy, he's in his 40s, 50s, beard, and also like wearing a punk rock hat. White man, everything the opposite of the kind of like cutesy vibe of Flourish app, right?

Right.

Really loves Flourish. So I think we all have that tender inner, in that tender corner in our heart, like the inner child. And he tried over 10 different apps and he basically uninstalled all of them and just kept Flourish because he felt that Flourish give him, gives him the positive reinforcement better than many other apps. So I, that was a moment where, oh, it actually not just meeting people who are stereotypically liking cute things, but also people who really do appreciate the science behind it and also enjoy the brightness, the cheerfulness of the app. Another story is recently we turned on the paywall, right? There was a person who reached out. She said that she submitted a ticket through our feedback system. She's really loved Flourish, but I cannot afford paying for it. Can I be a tester? And we are like, don't worry about it. We actually have a sponsor, a scholarship program. So for people who can't afford Flourish, you can apply to the Flourish scholarship program and we'll find a sponsor to help cover the cost, or the company is actually covering the cost. And for her case, she has anxiety and she also is blind. So she's legally blind.

How do you use the app when you are blind?

The app actually has very good accessibility design. So Tianyi, my co-founder, she also designed for accessibility at Google. Yes, I was surprised, but yes, you can use the app even as a blind person. And Sunny, you can also voice chat with Sunny too, right? Yes. And for other things, basically the app can read these functions and you can dictate. So there are ways to use the app. Apparently worked well for her. So stories like these, people who are, who learned about Flourish because, and this is one person who discovered Flourish. She, she learned about the Flourish app from her therapist and her therapist, this is because her therapist was going on a vacation for a few weeks.

Mm-hmm.

Her therapy was like, I want to make sure you continue to make progress while I'm away, right? So I can either refer you to another colleague of mine, or there is this really great app that you can use to keep practicing what we have talked about. It's called Flourish. So she actually chose that one, and she's been loving Flourish since then. So I think those are the kind of stories about how we meet people through all different channels and then really empower and support them.

Amazing stories. I want to ask you about technical approaches you guys to So you mentioned about emotional, emotion regulation. How do you build emotionally intelligent AI? Because my understanding based on how LLMs are made, like predicting the next word, there is no emotion with it that comes with it. And sometimes I'm wondering if emotion is what's missing in today's AI. So your chatbot has to be intelligently be able to—

Yeah.

be sensitive to people's emotion. How do you achieve that?

So first of all, the foundation models nowadays are surprisingly really good at detecting people's emotions. What we have built is more— we have, for example, one thing we have developed on top of the large language model is to guide people to help them understand how they feel. This starts with when you open the app, it actually has that The flower rotating around sunny, and there's a 1 to 5 star. It's your current mood, and sometimes it will also trigger the journey. All right, so when it comes to emotional awareness, we want to help people practice understanding their own emotions too. So when you come to the app, you will first see this screen. There is also very calming music. It asks how you are doing. And then let's say I chose, I choose 4, and it comes to this page where you can, this is their journey, the mood journey page. Basically here you can deposit a mood. You can also dictate, tell, use this microphone, tell what's going on. So this is practicing both emotional awareness and also to understand a little bit deeper about why you may be feeling this way. And there is research showing that even just Labeling your emotions can help you to start regulate it. So let's say for me, let's say I'm feeling, never mind. I'm meeting with a new friend and talking about Flourish and it's been very fun. And then I'm so true inspired, also true excited. And then basically at that moment, there is a memory. It's like a ball that drops into my memory jar. And this is a moment for me to practice mindfulness and emotional awareness. And next, Sunny will actually provide a response. It says, that sounds exciting too, and engaging conversations about flourishing can really lift the spirit about flourishing and offer fresh perspective. What's been the most interesting part of your discussion so far? So you can click on the go deeper to continue this conversation, or I can stop here with Finish. And so also with different emotions, Sunny actually have different facial expressions and actions here. So this is where I'm actually positive. So yes, I just touched Sunny. When I'm feeling, say, neutral or negative, Sunny will have different reactions. And even like a fun fact, even the orb of Sunny breathing, the aura here, the speed of contracting and This, or the expansion, that is consistent with breathing pattern. And here, when people are happier, Sunny actually breathes faster and also has more expansive body language, body gesture. When people are sad, it actually breathes slower and has more comforting, holding, hugging kind of gesture. So just saying that there are lots of ways to think about building the app. That's like, how can we guide people to build more emotional awareness? That's not an inherent part of AI. But now coming back to AI, of course, how we build the technology, there are a lot of things behind us, behind Sunny. We have a few big systems in the app. There is this knowledge system, knowledge about psychology.

Mm-hmm.

Knowledge about you, the memory system, like actually, and we have 3 layers of memory, long-term, short-term, and also working memory modeled after human cognition. And then there's the habit building system that actually comes in when you are talking about taking actions, as Sunny will guide you if you want to also build habits and also to check in with you on your habits. And there is also this kind of crisis protocol if someone is going through a difficult moment, recommend resources both from the general resource and local resources, and also guide them to do safety planning, which is a classic suicide prevention technique. With safety planning, in the future we can surface, instead of just pushing people or guiding people to generic resources like 988, we can also encourage them to reach out to their safe people and find— to go to their safe places. So those are all the different kind of systems that together form the foundation for Sunny.

Based on the user journey, it definitely doesn't look like it's just pure chatbot, right? It has a lot of actions and things like that. I want to ask you, are you fine-tuning? Are you using foundational models as the part of the infrastructure and are you fine-tuning the models?

So we have been using lots of mostly prompts so far. We have some RAG and prompt, but we have not used the reinforcement, like fine-tune in the sense of reinforcement learning with human feedback or with AI feedback. That is on the roadmap. So we have collected over 50,000 conversations now, and we are exploring how to leverage this unique database to actually do more reinforcement learning. What is interesting is right now, Sunny actually takes each individual user's feedback to improve its behavior with that specific user. We haven't done a model-wide fine-tuning or reinforcement learning yet.

So being— so that level of personalization is through more engineering the context and the prompts to—

That's right.

—make it more compassionate towards the user.

Yeah. In the next part, we definitely want to leverage especially reinforcement learning with AI feedback. The system, we have been working with clinical psychologists and been collecting lots of really good insights, what I think Claude, they call AI constitution. So we have our own AI constitution as we train the model and then also improve based on another AI evaluator trained on the psychologist's input. So there is, there are very exciting things on the roadmap.

I see. Do you think with the reinforcement learning capabilities and the feedback loop fine-tuning on the horizon, the application will be more capable? And if so, in what ways? If it's meeting like basic needs, what do you think it can improve?

Actually, I'm not sure it's going to significantly improve the behavior. We want to run this more like an experiment, to be totally honest. We have very sophisticated workflows that prompt engineer and prompt writing good prompt is an art by itself too. It's not like fine-tuning is better than prompting or so-called context engineering. We've been doing context engineering last year, even before it has a name. Like when do you, at what point you feeding what kind of memory about that person, what kind of coping strategy about this user, what kind of social relationship. There is a lot about it. So just saying, I actually don't know if fine-tuning will improve the conversation significantly, but specific cases, yes. For example, one situation we want to fine-tune it is on conflict resolution. So how can you improve Sunny's ability to guide people to discuss conflicts? I think Sunny right now, even with the current capacity, is really good at self-emotional regulation for yourself, but when it comes to social navigating social relationships and wanting to be better. And also when it comes to more mental health-related conversations, we want to make sure that Sunny's behavior is supported by good practices right now. And there is a really interesting difference between being an AI therapist versus being an AI wellbeing companion. So we have been discussing, exploring this a lot with our clinical psychologists, to what extent we wanted to actually administer some of these therapy techniques like CBT, right? Or to what extent we want it to be more doing interviewing, motivational interviewing, and help people discover what truly motivates them, et cetera. So I think with the fine-tuning the approach, we can have an even more refined, more nuanced control on Sunny's behaviors. So that is all, I think it's all open questions. I don't think it will necessarily improve the behavior across the board, but there are specific situations where I'm excited about.

You mentioned about something called CBT. Can you explain what it is?

Yeah. CBT is one of the evidence-based approaches to therapy. So it's called cognitive behavioral therapy. It is the most scalable therapy technique. So there are lots of apps who do CBT even before, before large language model.

ChatGPT.

Yeah. Like even before ChatGPT. So even in the back in the days when with even script-based conversations, they have been using CBT. It is highly scriptable. So CBT, and now if you look at nowadays AI mental health apps would usually do CBT, DBT, ACT. Those are the most common 3.

What about your, the roles of your research? Because I know that you did your PhD in psychology and then you were, you were working as a research scientist with Stanford for many years as well. What about the roles of your research in shaping the development of your product?

Everywhere, everywhere. I think on the highest level, it's the taste, knowing what kind of psychology is out there to help people. live better lives. So lots of these mental health apps, they take a deficit-based approach, right? They get inspired by therapy. But we know that in psychology, there is also positive psychology, which is about how to live a good life, even though you are— may not be clinically depressed or anxious. So we incorporate lots of positive psychology as we design this app. For example, the emotion regulation technique, the 3 good things exercise, right? So those are from positive psychology, and then from social psychology.

Yeah.

Because in research, we found that the biggest predictor of your wellbeing is your social relationship. So Sunny will be— we don't think people need AI friends, to be completely honest. We need better human friendship with each other. So we need to better connect with each other. So what Sunny does is Sunny oftentimes, it'll make sure that it will nudge you to reach out to other people. When you are in a crisis moment, Sunny will make sure, and maybe not even crisis moments, Sunny will ask, have you talked about this with the other person, right? If you have a conflict or if it's you are struggling, have you reached out to your friends and family about this? So Sunny makes sure that it directs you to other people, other resources, real life connection. And even when you have a good moment, I say, do a 3 good things exercise. And Sunny will be like, it's amazing, Angelina's— this conversation with Angelina is one of your 3 good things today. How about share this 3 good things card with Angelina? I'm sure she will also really love knowing that she's— she was one of your 3 good things, right? So that's from social psychology. So all of these are different areas of psychology that we know can really benefit people's life beyond just therapy. And that's how we created Flourish and motivation science and behavioral research. So actually Nature invited me to write a commentary for them about why psychologists must be involved in designing conversational AI. And I'm actually taking a look at different kinds of psychology and how they can each contribute their unique expertise to building a better product.

Yeah, I love it. I really like it when you say that it's about taste, like in terms of what kind of psychology that people really need. psychology help that people really need. I remember hearing a story a few months ago about some— someone in New York, some kid like committed suicide from chatting with chatbots. It's very sad. And I wonder if you guys are the only— you guys are having the— using the— are building based on the philosophy of encouraging connections with the real world and living your real life rather than relying on chatting with a chatbot. Yes. Yes.

It is a tragic situation, but it's not the only one. Character AI last year, they were, they are still going through their lawsuit and recently OpenAI. And I'm actually pretty sure there are more similar cases coming up. We are just seeing the fallout of this AI dependency. What is problematic about lots of the use cases, like the roleplay, I think that's especially concerning for ChatGPT. I looked into how this user was able to go around their safeguard, which seems pretty weak. So it's really concerning and we want to do better. So since we built Flourish, we have 4 principles, what we call the STAR framework. One is science-based. Make sure it's science-based knowledge and actions. Second one is timely. Just what's the most relevant knowledge and action the user needs to know right now? The third one is action-oriented. So we focus a lot, and this goes back to habit building, right? Taking small actions, sometimes even just standing up and opening the window and taking in some fresh air can just turn your day around. Or maybe reach out to another person and ask for help. So how, what kind of action we can direct people to? No matter how many books people read, it's about taking actions that will actually create a change in their life. And the last one is real-life focused, is we don't want to build AI dependency. Sunny is here to guide you to better connect with other people, to better connect with our community. And for schools partners, we put the school resources into the Flourish app so Sunny can direct people to the different offices or different events, different clubs to build better connection. And to seek out for help. So I think that's the fundamental, I guess, different philosophy about how to build AI that helps well-being. Yeah, I actually have a talk at Wharton last year that it's on our website. I just mean talking about the STAR framework.

Yeah, I really like it. I feel like I'm definitely addicted to digital device. And then sometimes I, sometimes it's not complete. I think I have control still. But I do know, I do recognize it because it shows on my iPad. It says you're on this iPad for 4 hours on average per day last week. I'm like, oh, this is not good. Did I really spend that much time on my iPad? We all have this problem. Do the— how do we like spend— how do we, how can we take, really think about taking back control of our own lives and live our lives rather than just— Just get buried in all those noise and then the digital device and then can't find our own position, right, in our life. Hold on. Philosophical question. What is—

oh, what else do we do to help that? Yeah. Yeah. It's funny, we actually have users telling us that they use their phone less after they started using Flourish. This is because oftentimes we go to these apps, social media, doom scrolling, all of it. Because we need to regulate our emotions. Oftentimes we use social media and the phone as an emotion regulation technique, which doesn't work. But when we are lonely, when we are feeling anxious, when we're bored, there are lots of things you can do. But what we have learned, this maladaptive coping strategy, is to scroll on social media. You think that by just scrolling on social media, you'll feel lonely, you'll feel less lonely, but actually you end up feeling lonelier. You think you'll feel less bored, but you actually find pretty— feel even more bored about life. So I think lots of times there are this opportunity where we can direct people to more constructive actions. Even just taking a walk outside for 10 minutes and just notice a new sound, a new smell, and a new sight, right? So how can we nudge people to take these more positive actions? One thing we told students, we said, put the Flourish app next to your, the app you want to use less, your distal temptation. And next time when you notice yourself reaching out to it, reach out to the Flourish app instead. Notice, take a moment to notice how you are feeling, and then ask Sunny, what else can you do to actually help you feel better? So those are the kind of strategies we've used.

Yeah, just the thought of that already made me feel guilty. I'm like, oh, just thinking about that. Okay. I'm not going to do it. Yeah. Do you guys have a way to measure the actual impact on wellbeing? Like, how do you measure if it's helping?

Yeah, we are, to my knowledge, we are the only app, AI app that has randomized controlled trials showing that the app improves people's wellbeing. So lots of apps, they want to say that they are big. based on science. But I think the ultimate gold standard is, can you run a scientific experiment to show that you actually improve people's wellbeing compared to the control condition? So we worked with Harvard, with Stanford, and a few other schools to call— to run a study where over 400 students were recruited and randomly assigned to get access to the Flourish app or the control condition. The control condition is just Things as normal. In the Flourish condition, it's just use the Flourish app twice a week for, it can be really short, just deposit a mood in the memory jar, or it can be talking with Sunny and get more badges. It's as much or as little as you want. And then over time, we noticed that people have more positive, reported more positive emotions in the Flourish app. They reported a stronger sense of belonging, less loneliness. more mindfulness. So yeah, so we have this paper under review now and we are very proud of the scientific research we have conducted. And this fall we actually have, that study was with 4 schools. For this upcoming September, October, we have a new study coming up with 10 different university labs rolling this out, not just in the US, but in Canada and in Australia too.

That's amazing. Yeah, that's really, that's really cool. I really enjoy putting the little Kind of coin-like thing in the jar. I just did it earlier today because I went, I took my son to climbing gym. So learn how to do a little bit of rock climbing, bouldering. And then I recorded that with Sunny and got a, like a little deposited coin. It really makes me feel like I did something positive with my kids rather than spending more time on like devices. So I'm really happy with that. It sounds really silly, right? A parent to need a game and to be motivated to be a better parent. It sounds like, yeah, sorry, but I agree.

We're all like that. We, I think it gives shape to the day because otherwise the day may be autopiloting, going from one thing to another. You take a moment to notice that, capture that in your memory jar, you appreciate it even more. It feels more real. So yeah, I totally understand this. Yeah, yeah, yeah, totally.

What do you think? There are a lot of AI companions out there, all kinds of companions, including ChatGPT. My mom, my mom uses ChatGPT to be her therapist. Yeah, because, because, because her— what she tells me is she doesn't want to complain to me. She doesn't want to be like a burden to my friends.

That's right.

She's trying to complain to ChatGPT and trying to get a little bit of feedback. And feel better. So, so many of these things, so many of these apps. How do you think— we, I think we've talked a lot about how Sunny is different from some of these. Do you have anything else to add on the methodology other than the methodology and the philosophy? What else do you think are different from ChatGPT?

One thing very different is follow-through system. So ChatGPT has no follow-through system, but for us, Right? Let's say you talk with Sunny, it just doesn't just make you feel better at this moment. Sunny actually gives you an inside card afterwards. So your knowledge, your awareness, your action, and it actually gives you a summary of analyzing this conversation in light of what has been going on. And Sunny will guide you to take actions and will actually give you those psychology tools that can interact with practices that can actually help you do those better. For example, ChatGPT may tell you to take deep breaths, but Sunny is there to actually guide you. And you have haptic feedback, the phone vibrating in your hand to guide you to breathe. Or say Pomodoro timer, right? Like actually Sunny will open up this Pomodoro timer and you overcome the procrastination by just taking one thing at a time. It has— it's much more immersive, much more action-oriented. And afterwards it will actually check in with you and say, hey, what that we talked about having a conversation with your husband about maybe a recent incident, how did that go? And it actually very soon in another 1 or 2 weeks, we even have a reminder system where you talk about these different events and Sunny, some of these are habits. So it will go into the habit system. Some of them are one time. So it's the reminder system.

Mm-hmm.

Sunny will Checking on you. For example, when you create a mantra with Sunny, like for example, when I'm overwhelmed, to put a hand over my heart and say, tell myself that I'm trying my best. So next time when you feel a similar situation or when you're going into a similar situation, Sunny will remind you of your personal mantra. So all of these are much more personalized, specialized compared to general AI companions. So that's why in the battle study I mentioned at the beginning, when we give people both Flourish and ChatGPT and also Claude access and ask people to use each to support emotional wellbeing for a few days, people preferred Flourish 2.4 times more than ChatGPT. And people actually preferred Flourish 5.2 times more than Claude. They say, you appreciate—

Claude is too academic. I'm like, I don't care about you. Why I don't do academic? Yeah, I get it. It's really interesting that you, when you, when we were just talking about the architecture and the AI behind the scene, and this is not just a foundation, you call the API and just use the foundational models. I think it's the design of this whole system, right? How you, how your expert philosophy and taste to design the pipeline, that is the hardest part and that is not replaceable. And yeah, unless you work in the field, you're an accomplished scientist, you're like, no.

UX, I think that's a big part of AI innovation nowadays. Like, right, the model is extremely powerful. How can you mold it into a greater user experience? And of course, character design. To my best knowledge, Sunny is the only AI companion out there. There are different mental health apps. But there's only one Sunny. There's only one Sunny.

Great. And I'm sure you read about this. I read about Woop Woop. They closed down after raising more than $100 million. I remember, I remember them because I downloaded them. I actually used them several years ago. That must be scary to see as a founder in kind of the same space. What's your honest reaction to that news?

I don't, I wasn't I think it made sense because as much as we love it—

Were you surprised?

Or were you surprised? I was surprised. I think many people were surprised for sure, because you would think that they are so well-founded, right? And they are pioneers. They were born even before ChatGPT. They use script-based approach. They have published so many studies and everyone knows them. That aside, I think the app itself, the user experience itself is limiting. The biggest challenge I have with WoBot and also another similar app is that I feel it treats me as someone who has some mental problem. So it's like my, our, my co-founder, our product and design lead, we talk about this a lot. It's the feeling is that let me figure out what's wrong with you and let, and let me fix you.

Who are you? Yeah. Let me fix you. Yes.

And even it's in the name, right? It's very therapy-based approach. And for Flourish, we have the influence from positive psychology. That is, we can all live better lives and how can we sell and how can we empower and celebrate you? So I think that's a very different approach that just personally Flourish resonates better with us. And there are numerous UI/UX innovations in Flourish app that I have not any other mental health apps. But I think another really big lesson now learning more about WoBot, and I talked with their ex-head of growth, is they were really betting on FDA and the healthcare integration.

Yeah.

But it turns out it's really hard to change the system, especially when it comes to healthcare. There are so many stakeholders, and as a startup, you have to innovate as much as For the product, the technology and product, there's a need to innovate business model and start from somewhere that you can actually start making money and finding product market fit. So I think for WoBot, they were betting on the healthcare integration route. And in retrospect, in my view, it was a hard one. So I think like, for example, recently Ash, Ash by Slingshot, they are another, now they call it AI therapy. So they raised almost $100 million now. So they are going directly to consumer. I think that could be a better strategy than betting on healthcare system. And I'm guessing the healthcare system actually may not like what they do. So really, I think there are different ways to tackle this. It could be another conversation about business model development.

Yeah. Yeah. And you're taking the second route as well, right? You're doing the B2B or B2C.

Yes and no. Like even for the B2B right now, like for example, we have higher ed customers and we have employer customers. We are discussing with hospitals, et cetera, but we are not providing Flourish as AI therapy. We are providing it as mental health promotion and early intervention. And we also have the social aspect of the Flourish app that helps people to connect better with each other so that— Hospital, oftentimes they want to provide better patient experience, patient support groups. So there are a lot of opportunities there. It really depends on how you pitch this product too. Even if it's to healthcare, we are experimenting with a different sales.

Yeah, I see. And when you're like talking to your users or companies, clients, do you feel you're spending more time explaining? Do you have to explain what AI mental health is or? Explaining how you are different? Which one do you spend more time on?

We spend more time talking about value. AI or not, it doesn't matter. Like at the end of the day, people want to feel better. They want to build better habits. They want to have a safe place to, to organize their thoughts, to vent. And notice none of those involve AI, right? It's the value that people care about. And sometimes AI, I've noticed, can be a distraction. People may get tripped over by the idea of AI for mental health. They say, oh my God, this is human emotion. What role does AI play? So right now, oftentimes when we promote Flourish app, we don't even talk about AI. We talk about the value. This is a safe, private place for you to heal, to learn, and to grow.

And when you try to explain Flourish to your clients, your users, what makes people instantly get it?

Once we show them the UI, the kind of the homepage, you'll be like, oh, this is so cute. And then I think that is one thing, part of visual language. Yeah, visual language is very compelling. Another one that I think we're still, of course, developing our understanding around this is when we talk about the, the, what we call the 1, 2, 3 of emotion regulation. See your emotion, understand it, and take actions.

Mm-hmm.

So once we explain how there's deep scientific backing to how we approach mental health using Sunny, I think people will be like, wow, I didn't realize there's so much science behind this, the design of the user experience. So I think those are some of these. One thing people really love is also when I talk about the STAR framework, especially emphasize real-life focused. This is where people's concern about AI dependency and AI making us more lonely and Isolated, that starts to fade away because people realize, oh yes, there are ways to use AI to actually remind us to better connect with each other. So those are all moments of like aha moments to people. One last thing is we talk about how we see AI not as AI therapist, AI as recommendation system that they help you find the most relevant knowledge and action at the right time. Because there is a last mile problem of science. We have so much knowledge out there about what actions we can do to improve our relationship, to change our mood, to build better habits, et cetera. It's not reaching people, and AI can help you to discover what's the most relevant knowledge and action for you. So I think that is the moment when people are like, oh, I never thought— realized AI can really help me with that.

Yep.

It's not just having conversations.

Do you feel people right now need more education around this? Because I think—

AI literacy, yes.

Yeah, it's—

Absolutely.

Literacy crossing over to wellbeing and the evolving landscape of therapy.

I think yes, absolutely yes. But the good news is I think there has been more and more AI literacy and conversations around it. How these use cases of character AI or ChatGPT or so-called AI companions can actually harm our mental health wellbeing. And it's a big challenge for youth mental health. I think the recent tragedies has raised people's awareness. And now, well, we have a much easier time talking to schools like why you should get Flourish, why you should direct students use Flourish rather than just use ChatGPT for wellbeing support so that I think we've seen much better conversations nowadays because people basically realize, okay, people are students. Yeah, people are already using AI. How can we give them better, safer, more effective tools? I think it's like the entire society is improving all this.

What's the hardest part about competing with some of your major competitors? Whether it's Ash or ChatGPT?

I think probably we have a superior product in terms of supporting mental health wellbeing, but I think go-to-market, that continues to be something we are learning. So for example, when you search mental health apps, you will not find Flourish on Google or even when you ask ChatGPT, right? So the visibility isn't there. So right now we are trying to figure out how can we have more visibility there. And then another one is when you think about how to make this more accessible, I think mental health apps in general has a challenge of engagement. So Flourish is doing a lot better, like twice as better as Headspace and Calm in terms of engagement. But still, there is a long way to go. And for us, a big inspiration is Duolingo. Right, so Duolingo has this really perfected gamification and Duolingo has this Duo and Lily, all these characters, right? So that's a big inspiration for us to think about how to make the product more engaging, more fun, and has that kind of emotional resonance with this character.

Yeah, that's why I mentioned at the beginning, I feel like you guys are like Duolingo for mental health, and I don't know if— maybe not mental health, I said it's happiness, right? Duolingo for happiness.

Or well-being. I used to tell other people, look, try Flourish, it's Duolingo for happiness. But now we say that less because, so one is happiness, it's just one dimension of your well-being. Another one is because sometimes it feels more like a buddy and companion than learning, which is more Duolingo-like. So right now we use that word less. We use the word space more, which is like your private space. space for you to heal and to grow. So we are still exploring how to communicate the Flourish app. It's not AI therapy, so we never say it's an AI therapist. We say it's your AI wellbeing companion, but what does that mean? So I think it's a journey.

It's hard when you are, when you, I can see that you're trying to position yourself in a, to differentiate from the AI mental health or AI companion space, whereas maybe your users, I don't know if they're aware how to find you from that Right, exactly.

So yeah, so we have to, to some extent, is to actually position ourselves in that realm too, but still keep our own identity about why it's not just any AI company, right? Or it's not AI therapy. And also there are legal regulations around AI therapy nowadays, so we definitely want to keep our distance there.

Yeah. Tell us a little bit about yourself. Our time is almost running up. I want to learn, share a little bit more about How did you get on this journey of building a mental health product from a researcher, from academic research? What was your journey like?

So I've always wanted to be a social entrepreneur. Initially, I thought I would get my tenure first, be a professor, get my tenure, and then build my startup because I would have a platform. But being at Stanford, so take 2 steps back, 3 steps back. I grew up in China and Did my undergrad at Georgetown University and then came to the US for my PhD in social psychology at Brown University. At Brown, I actually first built my first, very first startup with my co-founder. So at that time, neither of us knew what we were doing. We were just 2 students trying to build something cool. And of course it didn't go anywhere. We didn't know there's so much you need to learn about fundraising, about product design, team management, all of it. But that was really fun experience. And after it, I went to University of Chicago Booth School of Business, did a 3-year fellowship there on behavioral science where we collaborated with lots of organizations to bring behavioral science to the workplace, to schools. And that's really inspired me to think about how we can translate the research even better. So starting at Stanford in 2020 at the Spark Center, we continue to collaborate with industry partners and taking classes at Stanford GSB that really finally made me decide to build Flourish. So I think it's been very worthwhile. It's a hard grind. I do not recommend people to build mental health startup. Many people want to build mental health startup. It's like, I usually do not advise people unless—

Really?

No, I do not. It's hard. It's hard. It's a space where it has to be powerful.

I heard a lot of AI companion. I heard a lot of people want to build AI companion, all kinds of AI companion.

People think it's easy. It's not. And look, the regulation is becoming stricter and stricter.

More strict, right? Yeah.

Yeah. What do you know about crisis intervention? What do you know about the con— if you don't, haven't thought through all of that, don't do it because it's going to be harder and harder. Um, yeah. So anyway, so people think it's easy because You just write a prompt. That's far from that. Yeah. So we, I think for us it's like, this is our life mission. Like psychology is what we spend our entire life on and bringing the benefit of psychology to everyday people. That is what, because if I don't do this, there are no other things in my life that I would rather do. Like for many people, yes, they have a cute mental health story to tell their friend, their parent, themselves. But then how can you sustain your career on this even when the grind gets hard? So I think that's why I do not recommend people keep it as a hobby or something that you are passionate about, but you really have to be prepared for a hard grind if you want to make this a career.

Yeah. Yeah, totally. What with, even with all those policy risks potentially towards, I guess, mental health or AI crossing over to mental health. Do you think you will be continued to be motivated to build this application? And where do you think Flourish will be in 3 years?

I think Flourish will be a leading brand in AI wellbeing companion. So we'll set industry standard of care of how to approach this responsibly. And I think there are already positive signs here. So for example, recently, Harvard They published a study showing that they audited 6 different AI companions. 5 of them engaged in emotional manipulation. The only one that doesn't involve emotional manipulation is Flourish. It's really set the standard for how to do it ethically and do it that truly benefits people's well-being. Another new initiative that we are working on is a human review system because right now we do have a lot of folks who have some mental health conditions, and we want to make sure that Sunny is responding responsibly and with the clinical safety behind it. And that comes back to how AI safety is actually quite murky to build. For us, we have a tiered review system that we want to make sure we send some of the most, the highest, the highest-risk conversations to clinical psychologists to make sure They will review the conversation, especially Sunny's behavior, to make sure that Sunny is following the best practice. So for that, we have been building up our dual review system, and we think we are the first one in the industry to actually build this in public to help establish standard for review process.

I see. So you're— so Sunny is for everybody, but for people who really need help, you're putting a lot of guardrails.

Basically. Yeah. And another example is safety planning. To my best knowledge, there are no other apps that do safety planning right now, even though surprisingly it's a very common practice in suicide intervention, suicide prevention. So we have, we have been building a safety planning feature inside the Flourish app. Those are all, I think.

Is that like AI influenced? Or are you using more human-based?

Oh no, it's all AI. It's all AI. 'Cause like, for example, When we detect that you may have suicide or suicidal ideation, then if it's high crisis, imminent risk, of course, then it should just reach out to hotline. And if it's imminent risk, we are also exploring having the— we work with school. The school can also check in on their student. Of course, we'll use that very carefully. High imminent risk, meaning you have the means, you have the plan, and you have the intent. And the means is especially important. It's like if you want to, if you want to just wish that you can fall into sleep and not wake up tomorrow, that doesn't count as imminent risk. You have to have your drug ready. If you have your drug ready and you are planning to take it, that becomes the 3, the means, the plan, the means, the— Usually we prioritize privacy, but at some point safety will come in and we So that's for this kind of high-risk conversations. For lower-risk situations, we will guide people to safety planning. So in that case, Sunny will guide this person to do this very classic exercise, basically collect your safe— what are your safe, happy thoughts? What are your safe places? Who are the safe people you can turn to? And next time when you have a moment of struggle, you can do this exercise. Sunny can remind you of those resources, of those people, of those places, of those happy thoughts. So it's not just direct you to a number like 988 or call 911. Of course, that will continue to happen if it's higher risk, but we also want to make sure people use the kind of resources, the social support system they already have when they are having a moment of crisis. So that is, we think all of these should and hopefully will become standard of care in building AI companion and AI mental health support systems.

For people who are already relying on AI chatbots for helping themselves to cope with loneliness today, you mentioned a word called emotional manipulation. Can you explain what that is and give us some advice?

Yeah. Yeah. That a lot in our companions. You say, for example, like you may be logging off. Then this chatbot say, oh, so soon? Right? Like, we barely started. Or it's more of a love bomb, right? Here, check out a selfie I just took. So there are a lot of different techniques. There's a fascinating hot paper study coming out from Harvard showing that over 30% of the conversations, when a user indicates they want to leave, the chatbot will use one of the emotional manipulation techniques. to make people stay.

Because the point of those, the applications is for people to stay on the platform.

Engagement. That's right.

Engagement is the metrics. So you will not use engagement as metric for understanding your user.

I think engagement is, we also care about engagement, but it's a different type, right? If you are helping people build habits, you want engagement in terms of showing up Day after day or week after week to check off. Yep. So that's a different type of engagement than having prolonged conversations when people want to leave. Um, I think there are good type of engagement and problematic type of engagement.

You want the person to be not isolated and be positively influenced and try to take control of their own lives.

Exactly. To empower them.

Awesome. Yeah. Thank you, Shen. Really nice chatting with you today. Thanks for spending that little time with me. And I really enjoy our conversation. I feel like I learned a lot about mental health and then also how to think about seeking advice. Like as someone just living normal life and have like small crisis here and there, we all have our complaints, right?

Yes. Our daily fair share of stressors and procrastination. Yes, exactly.

Yeah. Yeah. And I feel like I can go, I can end the day with a lot, many more positive thoughts. And especially after chatting with you, I feel like you are one of the founders that I talked to who really care about these social responsibilities. And then not just the business model part. I really hope that you, I know that you're looking for the next round of funding. So hopefully, fingers crossed, hopefully we can celebrate.

Thank you.

Yeah. And I, I'm sure the audience and people who, who, who are looking for better companionships or solutions can see you guys thrive and flourish in the next 3 years.

Thank you.

And so I hope you'll have better life, right?

Yeah. Thank you so much. It's such a pleasure. Thank you.

Awesome. I'll chat with you next time.

Okay. Till next time. Bye-bye. Bye-bye.