Notes of my AI psychiatry therapy

Okay, so.

Wow.

So what you're looking at right now, this is, this is my interface, this is my electronic medical record. And so over here on this far side, this is the transcript that was happening in real time. You'd asked us before, you know, is it keeping notes in real time? Yep, yep, it is.

And it looks fairly accurate.

It's. It's. Even when it doesn't look to my eye like it's accurate, it's able to infer, I guess, between the lines or collect information in a way that I wasn't confident it could, just based on what it says here. But. So, for example, it's caught you saying, I'm wearing a black T shirt right now. I bet it's going to include that in your examination findings that you were, you know, a guy wearing a black shirt today. We'll see what it does. But. Okay, so you can also see here that we have the option to anonymize it. What that's done is it's removed our names. That's the first step of the HIPAA process, is making sure that anything that leaves our system cannot be identifiable. Then we can sanitize it as well. This is the new piece. This sanitize feature is what we've had to consider. Now that we've recognized the LLM will censor certain topics, I will be surprised if the note generates for you today because of what we talked about. I think the LLM's not going to be happy with it.

Oh, really? The church lady's gonna get involved.

The church lady and all of her friends, I think, are gonna show up to this party.

Oh, really?

We'll see.

Just. Just because we talked about sexual abuse and things like that, right?

Yep, that's what I expect. And so the.

Even though I'm public about it. I know, right?

I know. But it's because this is. Yeah. The good thing is I know everybody. The first question on their lips when they ask about this system is HIPAA and privacy. And so. So we have a couple of different steps we take to protect that. The anonymized was the first one until we realized, oh, my goodness, we gotta sanitize as well because of what the LLM does and doesn't do with information.

Got it. Okay.

So what I'm gonna try and do is we're now after the appointment, so I'll just show you here quickly. These are all the prompts that we've engineered to have it consider everything that it knows today about you, but only from. From you here today. Because once we send this to the LLM, it won't know. It's. It's Robert Scoble. It'll know. It's just. Just a psychiatrist and a guy talking.

Got it.

So those are the prompts. So I will. Like I said, I'll be amazed if this works. I think it's going to generate an error. So right now it's considering my prompts as well as the anonymized transcript. And it is the thinking. I've noticed it takes about 60 seconds. If it doesn't generate within 60 to 75 seconds, it's not going to happen.

Got it. So it doesn't. You. Yeah, it's crazy. And it doesn't. It, it's global. It won't work on any of the notes if that's true. Right. It. It doesn't just turn off for a paragraph that it doesn't want to deal with. Right. It. So.

Right. Right. It's your entire transcript. It rejects. And so that's where the sanitization process is going to come in. We're going to have to go through and try and understand when it's not happy and sanitize those items so that we can generate a transcript. But there's the bigger question, which is that these are important topics we talked about. They shouldn't be sanitized. From your. From my understanding of your situation and your story. This is your story.

No. Psychologists and psychiatrists have to talk about harsh topics all the time with people. Right?

Yeah.

Sexual abuse.

It works.

That's cool. See, I tried to keep it fairly clean. You know, we didn't go into a lot of detail about what my sexual abuse was about. Right. So.

Yeah, yeah.

But so what?

You can see this first one here. This is the subjective section. And so what this is, is it's meant to reflect your. Your telling of your story.

Yeah.

So it's not my interpretation. It's not the LLM's interpretation. It's just the facts that you told us. And.

Wow.

It's. Yes, it's. These are very common topics.

It's really interesting. It's really interesting to see it. Your own words brought back to you in a completely different form.

Yep. Yep.

Wow.

And this output is partly based on the prompts and largely based on what it heard you say. So that's your story. Now, this blows my mind. This is objective. So this is meant to be where I have observed you and I describe what I think I saw today. And it gets us a little bit wrong. Well, maybe it thinks you're a 24 year old man. Are you 24?

I'm 58. Yeah.

So that's something where I'd have to go in and just edit it like that.

Yeah. This is a good point. These large language models still make mistakes. Right. They generate. People say it hallucinates. Right. It just hallucinated that I was 24. I don't think we ever said the number 24 in our, in our conversation. Right.

I don't think so. And that's, that's an interesting question because what one of the superpowers this now gives me as a clinician is, is I stop and think, I did say 24. And then I wonder, was there 24? That was an important piece that went over my head today.

Yeah.

So I love that. I love that.

Makes you look at a conversation in a different way. That's interesting.

Sorry.

This is why I want to do AI first. It causes you to think differently. Right. It's like, oh, absolutely. Never thought about it that way. But maybe you're wrong.

Yeah, yeah. So this is absolutely fascinating that this, this is even possible.

Oh, I'm not suicidal today. That's good.

And see, I don't think that I asked that phrase today because I know it tends to get blocked by the LLM, but it's able to infer and based on probabilities, I suppose make that comment, which I would probably agree with. If I didn't agree again, I could.

Go in, look at this one. Judgment appear sound, recognizes challenges and seeks support.

Do you agree?

Yeah. But the fact that it got that out of our conversation, like, wow, you know.

Right.

What has it been really? And this is out of a 30 minute conversation. What if it listened to all my Twitter spaces? I've given Twitter spaces thousands of hours of Twitter spaces.

Yeah, it's pretty eerie, isn't it?

So I could go through all of my Twitter spaces and come up with a psychological profile on me. That's going to be pretty deep.

I guess. So ours will not be doing that, but I think they're. That is how possible. Yeah.

Oh my God. Elon Musk build a nice AI Truth.

Seeking AI Truth.

Oh, it'll tell you the truth of me for sure.

This is fascinating up here.

Does it give you a narcissism score? Does it know, you know, I, hey, this guy is a little. Talks about himself a little too much or something.

I never prompted it for that, but if I prompted it, I bet it could. Up here, this is the assessment. So it's based on prompting and based on what it knows about you just from today. But it's concluded that you likely have these labels for your symptoms. So adjustment disorder means that you've got a ton of stress that you're not coping very well with. Yeah, ptsd, that's the life threatening trauma that affects you even to the present day.

Yeah.

And then ADHD, is that dopamine hunt? Just that sort of like 10 million thoughts at once and you, you're just busy, busy, busy and. Yeah, active, active, active. So again, I can, and I, I do go in here and I adjust this if I disagree with anything. But as far as my superpower, it gives me food for thought if I see, oh, it thinks you have adhd. I never thought that. Yeah, I can sit, step back and think, what did I miss? Or what is it missing? What is it hallucinating? Yeah, it just gives me pause for thought and I think it makes me a better clinician for, for you.

This is crazy. I mean, by, you wrote all these prompts, right? You went, you worked with the company that made this tool for you.

That's right.

Learn how to prompt. What have you learned? What is its understanding of psychology? It seems pretty deep to me. I mean, what have you learned by talking to the machine from a psychological point of view? Because I don't talk to the machine this way.

Yeah, I think. Sorry, just clicked a button here. One of the best things I've learned before starting the super prompt engineering was that the LLM has read the Internet. And so I'm mindful of what I know is on the Internet about mental health and psychiatry and psychology. And so, for example, when I go into one of my prompts here, I tell it, you know, LLM, you know about these two major texts of mental health. Draw on that information as you consider this person that we just talked to today. Yeah, that's, that's been, I think the most powerful feature of the prompting that I'm using is, is reminding it what it's learned from the Internet. And because of that prompt there, that's how we get this prediction over here of the potential diagnoses. Wow. It's. Wow.

Does this blow you away?

My jaw drops every time I run a note. I've run probably 10 notes today from other patients. And I'm just, I'm, I'm, I'm in awe every time I do this.

Yeah, it's crazy. All right, well, into the AI world we are in now, GPT. Now, what if you actually hooked up GPT. And if GPT knew it was talking to me because GPT already knows a lot. OpenAI already knows a lot about. Know is a funny word because the singer actually doesn't know anything. But you know, to a human, it looks like it knows me very well because it read all my vlogs and knows all my patterns and can, you know, make, make your essays sound like I'm talking. I'm telling you them, right?

Yeah, yeah.

If you hook this up to my, up to, you know, Robert Scoble, all of a sudden the LLM would get a lot smarter about me. Like, oh my God. Yeah.

At the moment it's up to you. But you know, when people ask me about the drawbacks, I think we've got a responsibility, especially when using it in healthcare, to always protect your information and let it know about you only what it should know about you and nothing more. And that's where the anonymization and sanitization becomes so important.

If the machine knows you at this kind of level, which it did out of a 30 minute conversation. Right. Could it figure out how to addict you to things, how to trigger you, how to control you, how to change you? Right. How to brainwash you?

I think so. I mean, that's the joke that people always say. If you go to a dinner party and they find out you're a psychiatrist, people take a step back, they say, are you analyzing me? Can you read my mind? No, I can't do any of those things. But I really believe that an LLM could. If we don't supervise it, it's obvious it can.

I mean, look at what it did out of a 30 minute conversation. Right?

Yeah, yeah. So it must be a partner. It can never, in my opinion, should never work independently. There always has to be a human partner with it.

Yeah, that makes sense. Yeah, but you know, you know, there's some entrepreneur going, well, we're going to make a automated psychiatrist. All.

That'S the thing. I've got this, you know, young adult son.

There are rules against that.

What do we tell him? What do we tell him for a career? I don't know. Because the jobs that I thought would always exist are not going to exist in the very near future. You don't need a psychiatrist anymore if you can do this, you know.

Well, you still need somebody to talk to. It's hard to talk to your virtual. Well, yeah, and there's virtual. There are virtual beings coming that look like humans. Right. So you could have your human therapist and in your home. Right.

Yesterday I saw a tweet from. I don't know if it's you know, real real. But the, the tweet was. A woman has created a emotional support chat bot, and she's earning $17,000 a month from talking to people at a dollar a minute.

Yeah, I need a bot like that. But my bot has a bad reputation, so, I don't know.

I just, you know, I. I didn't know I would. A month ago. I didn't know I would ever see something like this. This is moving so fast, and our team is so responsive. They're literally programming day and night to make this happen. Yeah. I don't know where we're going to be a month from now, a year from now, 10 years from now. I can't imagine someone like you needs to imagine because you're the, you know, you're the evangelist for AI, what's coming.

You know, I wrote four books on tech about things that, that are coming, coming over the next decade. Right? And this one's going crazy. This is getting harder and harder to stay ahead of because we're in an exponential growth curve, right. And programmers all got tools to make them really productive. And so we're speeding up and it's hard to see where that's all going. But we're soon going to have a holodeck, right?

Yes, yes, yes.

I've seen pieces of it. I interviewed somebody this morning, so. But this is crazy, you know, I.

Mean, well, and I never thought. I never thought mental health could be incorporated with AI the way I've seen that it's now possible in other parts of medicine. You know, you have to see if someone's got a rash. How big is the rash? What does it look like? That's very concrete information that I think an LLM could interpret. I never thought an LLM could tell me that. I just talked to a guy wearing a black T shirt who was insightful about his condition and navigated the telehealth appointment. Never expected that. And you just saw it happen.

It's insane. Well, thanks. It's 46 minutes since the beginning of our call, Right? So in less than an hour, GPT figured all this out, right?

Yes, it did. And that's with. With restricted information. We didn't tell it most things about you that I guess could be known about you. It just for 35 minutes of sanitized information.

That's what I'm saying. If it ever hooked this up to what it really knows about Robert Scoville, you know, first of all, does it already know about this? Right. Has it gone through all my blog posts? And done an analysis. You don't like this underneath. I don't know. You don't know.

Yeah, yeah.

And so if you hook up your name all of a sudden and if, and therefore it figures it out who it's talking to. Right?

Yeah. There's a major responsibility now on anybody who's using this. And there's always been a responsibility as a, as a doctor to a patient, but it's, it's just gone into, you.

Know, because we talk about things today. Even if I didn't use my name, it would have figured out it, it could in potential figure out it's talking to Robert Scoville. Because there's not many human beings who do what I do. Right.

So it could. I mean, it got things wrong. Like, is there a 24 year old out there who, who. Well, you know.

You know, maybe it's just being flattering.

It is very lovely. It wants to please.

It does what it does do that, particularly if you're nice to it. You know, if you say please and thank you, then it seems to be a little nicer to back, you know, but maybe it's all in your head.

So the first time I ever saw chat gbt, I was asking really critical questions about itself and it actually stopped and said it sounds for getting very frustrated. Are you okay? I thought.

Whoa, whoa. Okay, well, thank you so much.

Thank you very much.

We'll talk on Monday. We're going to do a Twitter space. Well, this video, I'm, I'm tempted to do this video on Sunday and sort of use it as a pitch, but thanks so much. I'm going to turn off the video now.

Okay?