THE DARK SIDE OF OPEN EVIDENCE

Okay, that's 13 messages a day from chat.

That's a lot.

How many messages do you have with your mom? Not 13 a day, that's for sure.

Okay. Hi, everyone, and welcome to the podcast. This is Generation with mg. My name is Stephanie and I'm joined here by my colleague Dr. Sarah Davis, who's also my mother.

Hello. Hello. How's it going?

Good, good. How are you?

I'm good, I'm good. You've been studying and studying. Yes.

Step one coming up, winter break. So just going back to the basics. Open Pathoma Chapter one, Learning about the cell.

The cell. You're back to the cell.

We're back to the cell.

Oh, the mitochondria, the powerhouse. Powerhouse. Isn't that like sixth grade biology?

I. Yeah. And just, you know, cell damage.

Oh my God. Seriously?

Yeah, I do. Yeah.

Just I feel like we should be on. Be beyond the Cell by now.

Apparently it's really emphasized.

Apparently it's important, that cell.

Apparently so. But yeah. So just grinding away. Grinding away.

And then you take the test in like spring or something.

March 30th.

Yep, March 30th. So you're practically ready.

You don't know that literally at all.

Well, I just like to pump you up.

Well, that's what moms are for.

That's right. Yeah. Well, I know you're gonna do great, but some people, you can't. Some schools take it earlier than that. Is that right?

Yeah. So my school has. Does it traditionally. This is how they've always done it is that you do the first two years of pre clinicals and then you have some time for dedicated and then you take step one and then you start clinicals. Since step one went to pass fail. Now a lot of schools switched it around where they go straight into clinicals and then you take step one and step two after you finish clinicals back to back.

That's a terrible idea.

I don't know. It's not it. Because yes, like you forgot some of that basic science stuff and you have to go back and review it. But also, also it's just pass fail now. So it's like you should. You should be able to pass and then since you're already studying all that stuff, then you don't have to like restudy for step two as much. So. Yeah, I don't know. I don't know.

I just feel good getting it knocked out.

I agree. Some people even do that. They'll just take step one early even though their school doesn't. That's not how they Their school does it.

So you could just take it on your own.

Yeah.

Go rogue.

Yeah. So, yeah, I don't know. There's. I think there's pros and cons, but I will say that is a newer system. The step one, step two, back to back. And I don't think there's a lot of data on how students are doing. So I feel like I would feel better just doing it the way that.

People have done it, the way it's always done, because that's what your stats are.

Well, you know the interesting. This is some step one talk. So if you're uninterested, deep dive into step one talk. But this is the step one T. So it was about four years ago that it went to pass fail.

Okay.

So the first class that took it, I guess they are the class of 20, 25 that just graduated, I believe.

Okay.

So they were the first class to take it, pass fail. And they didn't really know how it was all going to go. So that class studied the same as if it were a graded exam. Okay. So, so then they studied and then they all did really well and passed with a breeze. So then they were like, they told the class below them, don't study. Who are the fourth years now? Yes. They said, you don't need to study that hard. We took it way too way, you know, too extreme. You guys just like, blow it off. It's really not that important. Whatever. That class, ton of people failed. So then that class who are the fourth years now then told the people who are the third years now, gotta study hard. They said, you gotta study. Don't listen to people to say, blow it off. You gotta study. So many people failed. So then the third years all studied a ton and they were all very nervous about it. So then that's still kind of getting passed on to our class as well.

Well, I was thought they were gonna say, oh, well, we studied too hard, you guys. Now you should blow it off. Maybe.

I don't, I haven't really heard. Like, I, I think it's a little bit mixed now. It's kind of like they all were. I, I see. Because I just remember them last year studying and talking about how hard they were studying and how nervous they were for it. And now they're in clinicals and I don't see them ever. So I didn't see them after they took step one. Yeah, they disappeared.

So.

Yeah, so that's the T for that.

Well, I wonder like how many people actually failed.

Like, I think it was, it was like 20 or something. Yeah, kind of a lot. Yeah, exactly. Um, so that's why also my school is very like, we have a lot of meetings and a lot of stuff because of that fourth year class.

Yeah, that's fair.

Yeah.

I mean, people, you know, you want to know that when you're signing out of med school, you want them to.

Have high pass 100. Yeah.

But you can kind of tell if you're on track. Right. I mean, you take these practice tests.

Yeah.

So I mean, if you're failing all the practice tests, you're probably not entrepreneur. Yeah.

I don't, I don't know what happened exactly with that. Maybe they told them to ignore the practice test. I don't really know.

But.

Yeah, so just, just.

Well, it is interesting though, because things do change, like studying. Studying for something that's pass fail is very different from studying for something that is. That you're trying to get every single little minutia correct.

Yeah.

Yeah. And I've heard about that too, that they kind of. Of thought, okay, well this would be better because people were just wasting so much time studying all this minutia that wasn't going to make them a better doctor.

Yeah.

And I think that people thought that was a waste of time, but now people just spend a bunch of time doing research that they also don't need to be doing, but it. To get a better residency.

I will argue though, I think it studying minutia versus doing research. And you know, research is important, so getting people to do that, I think is for the betterment of society, I think is better than having people studying these tiny little cytokines.

Yeah. Well, and the other thing is that at least when, you know, when I took those tests, it was, I mean, it would like. So it was your percentile, so you really were competing with everybody else taking the tests. It's not that. And so that to me, never. So it's not like, you know, everybody can't be in the 99th percentile. And so when you, when you take a test like that, it's. The more people know. Well, then the more you have to know to stay ahead of everybody. And then they have to make the test so much harder all the time to try to like, weed all that stuff out.

Yeah.

And you know, and the purpose of a test like that is to make sure that you know the material. It's not to try to separate out the 95th from the 99th percentile, as opposed to something like the SAT where you are trying to. That's part of the purpose of the test.

Yeah.

So anyway, I support them in their decision. Yeah.

I mean now all kind of. It's the, the same thing for step two though, now.

Yeah, yeah, now everybody's got to study for that.

Yeah. So I don't know. But whatever.

Yeah, whatever.

How it goes, how it goes.

But really one of the things I want to talk about today is AI in medicine because you really don't need to know any of this stuff. Just put it all into open evidence.

Yeah, put it in, put it into chat, put it into open evidence.

No, there's no reason to know any of it. But that's. So open evidence is. It's basically the chat GPT for doctors and med students and other medicine healthcare workers.

Yeah.

You do have to register and then you can use it for free.

Yeah. So you just put in your credentials or whatever and then. So for any healthcare provider I believe not any, but a lot of healthcare providers, you can put in your info and then be able to. It's a database for peer reviewed articles. Which sounds great, right? So like you can ask a question about what is the standard dosage for whatever IV insulin for somebody who comes in for.

Well yeah, and you can even give it like a, you know, a case and things like that. But what I do like about it is that then it cites all the references as opposed to chat. That just tells you you're doing great sometimes.

I like that.

Well, okay, when you need that, go to ch that. But when you actually need the real evidence, go to open evidence. But it did, it started a conversation in our family because of course your brother is, you know, an AI engineer and he sort of brought up the, the dark side that he's like, well, you know, can't open evidence do things on the back end so that certain articles are going to float up to the top and be cited more so than other published articles. And could it be that, you know, pharmaceutical companies can pay money to open evidence to get their stuff to be at the top? So what do you think about that?

Well, you know, I, I hate like conspiring against science in any way, but I will say something like open evidence is. I mean, I don't know exactly how they make their money. With ads, I guess.

Yes, with ads.

So it is kind of. There's no overlying like a governing body. There's no governing body above this. There's no governing body above open evidence. So you're not able to check and make sure that there's not conflicts of interests and all of those Things we're just kind of trusting it blindly. So I think that, I mean, I, it's also not perfect and I think people need to understand that it's not perfect, um, because it, it might show you some study and not give you all the nuances of the study and everything. Um, so. Yeah, yeah.

But yeah, you're not using it. Like every time I see a patient going to open evidence, what do I do? I mean, you're really just using it as like something you're thinking about and you, you can ask a question and it gives you, you know, it gives you some information and then you can read those studies and. But you could always go to Google Scholar and do a search yourself if you wanted to. So I don't think, I mean, yeah, you, you do not use this as just to tell you what to do, but you do use it just for like a little bit more information. I also think that, you know, their entire business model would collapse if we don't have trust in them. So I feel like they are going to make an effort to make sure that they're not being bought by the pharmaceutical industry. And I mean, there's lots of things, I mean, there's lots of ways that even somebody who doesn't work for Open Evidence can start to fact check. I mean, we can do studies looking at that and those probably will happen. So I think there's lots of ways to kind of keep an eye on the whole thing. I mean, just like anything, it's another, it's another tool. But you, you don't use it for, you know, you don't use it for everything. But it's, it's pretty handy when it's there because I've done lots of those Google Scholar searches and. Oh my God, I mean, you can spend just hours and hours coming through just trying to answer one question.

Yeah. And I mean, you can say the same thing for Google Scholar or whatever the search engine is. They could be paying to get up on the top whenever you search a certain question. Same thing with the Internet. So it's, you know, you can always say these sort of things. But I think that's the point of, the point of using it isn't to get an answer or to figure out exactly what to do. You should be able to do that. It's just to be able to keep up with information, ask question research stuff more. So I think it is smart that they have it for just healthcare providers and people that understand that this isn't a. Yeah. That there's nuance There with whatever it is.

Yeah, it's. Well. And so the kind of competitor to Open Evidence is up to date, which has been around for a long time. And basically it's a subscription service that they would. I mean, a long time ago, they would just basically put out a newsletter and say, like, okay, here's the. Here's the newest stuff that is out. We've looked at all these journals and here's some things that you really need to pay attention to. And I never subscribed to it because I was too cheap. I was just like, I don't know, I'll figure it out myself, what the important things are. But they, you know, and then they. They've kind of grown and they've, you know, then became more of a searchable index. And now it uses AI as well. But it's still a subscription service, so you have to pay for it. So, you know, somebody has to pay for this stuff. And so it's. In some ways, maybe it is a little bit more pure if you are paying it for it yourself rather than having pharmaceutical companies ads pay for it. You could look at all that. Funny story about up to Date. One of dad's doctors, one of the hospitalists that worked for him a long time ago was this guy who was just very smart, very much like the internist kind of brainiac guy. Also super, like, ethical, moral. He played in a Christian rock band. Anyway, just sounds like a guy I'd.

Want to be my doctor.

Yeah, yeah, exactly. Well, he's very ethical, moral. Like, what do you mean? Because the story that I'm about to tell is that up to Date contacted him and said, hey, we think you have been sharing your subscription with other people because you were on this for hours, hours and hours and hours. And we don't think any normal human would be on this that much. And he was so offended because he had not shared his subscription with anybody. He just literally was on up to date for hours and hours and hours and probably logging in from the nurse's station at the hospital and maybe from his house and maybe from a friend's house. And so they thought, oh, well, this guy's got to be up to something. And he was the one person who.

Doesn'T share his subscription. Everyone else does that.

That's what one of the other doctors was saying. He's like, I share my subscription all the time. And they've never contacted me, saying I'm on there too much. That's funny. Wow. So anyway, so there's the. There's the different, the different models with that stuff. But, but I don't know, I, I'm, I'm a fan so far of this open evidence. I think we just gotta, we just gotta make some, you know, do some checks and balances on the whole thing. Yeah.

I've been writing a research paper and it's really handy to be able to do that because I can kind of. It's handy. And also it, I do think just sometimes will reinforce biases that are already there.

Yeah, it actually, yeah, I saw something in there that it does do that.

Because I, there's so many studies that will show you exactly what you want it to show you.

Sure.

And so if you frame it in a way of saying, because I'm talking about substance use in India and I wanted to make a point that substance use is higher in rural areas in India, which is kind of nuanced and not necessarily true, but I was like, give me a study that shows me that substance use is higher in rural areas in India. And it popped out with something. But then if I asked is substance use higher in rural. What do, what does the evidence show of substance use in rural India? Then I'll be like, well, this study says this, but a lot of other studies say this.

So.

And as I'm writing my paper, you know, I have to be. You're kind of, as you're writing and you're putting all the pieces together. Even before open evidence, when I was writing and had to look into stuff, you do kind of look for things because you have this idea and then you want to look for a study that kind of like validates that a little bit so that you can put that in, you can like make it fun flow, but it's just, just kind of makes it easier to be able to do that.

Yeah.

You know.

Yeah. And that, that's true in science with lots of things. Right. I mean, you can, you can find studies that prove lots of different things that you want to prove now. A lot of times you'll look at those and you're like, well, that was an itty bitty study done like 20 years ago. Yeah. That probably wasn't, probably wasn't the best study. And if you look at more recent things or bigger or meta analysis, you're going to find something a little bit different.

But yeah, just talking about AI and research too. I saw something that was talking about how people are just using AI to write all their research paper. Well, using AI to analyze their data, using AI to write all their research papers. And then the Peer review. People are using AI to review the research studies. And now we have stuff like open evidence that is then giving us the information AI that is reading the papers for the people reading them. So who's reading research papers anymore?

Wow, that is kind of scary.

I just hope that it ends up being a way for, I mean, we were talking about this on the last episode to be able to facilitate that communication between scientists and person and just, you know, lay person, because I, I think that that is such an important thing and there's this kind of gatekeepy idea of reading research papers or whatever. And I think scientific literacy is, is so, so important. So I mean, hopefully it can be a tool to be able to facilitate that. But I don't know, just like everything.

You know, it's good and bad, right? It's. It also just like you're saying if you want to find something that proves whatever it is you, maybe now it's easier for you to come through there and find something that tells you that, that you know, just some sort of crazy idea that you have. Okay, so speaking of that, just along with ChatGPT and I happen to read this one article about this woman who felt that ChatGPT kind of caused her psychosis.

Oh, is this delusional? AI delusion or whatever? Yeah, oh yeah. I mean, 100% because it'll just validate. We're talking. I was shadowing a psychiatrist in the psych wards and we were talking about this.

Yeah, it's really in. This was this woman who, they called her a healthcare worker. They didn't say what she was, but they also said she had done a 36 hour shift. So she sounds like a resident. She was taking Ritalin and an antidepressant. But she asked Chat GPT to help find the digital footprint of her brother who had died three years before. And it started off sort of like she just wanted to see just stuff out there on the Internet that he had posted and things like that, just kind of for memories of him. But then she also asked Chat to look for it in a magical realism sort of way, which seems like she was kind of asking for stuff. And so she kept. So basically she started like being convinced that her brother, that she had brought her brother back to life. And she ended up in the inpatient psych ward with delusions. And she did not have a history of delusions, although she was taking Ritalin and amphetamine, had been awake for 36 hours, and then was using chat GPT. So I don't know, it Seems like it doesn't. I don't feel like we can pin that all on chat.

Yeah, I mean, it's called chat box psychosis.

Okay.

So, yeah, and like, we were talking about that, and we didn't have any patients that had that, but the psychiatrist was just talking about how. Yeah, that is a thing. And people come in, say, well, like this. AI told me that this thing is true. There are people following me. There are. There is a chance. Because that's what Chad's gonna do is just say, well, there could be a chance.

Yeah, Chad's very affirming. Yeah, whatever you say. So, yeah, that. That's not great for somebody with I. With psychos.

I could see myself falling down that track. Did you get your chatgpt wrapped?

No, I didn't do that. I need to do that.

Well, yeah, I could definitely see myself falling into a chat box.

What did chat have to say to you?

Psychosis? Well, it just is your wrap. So let's see. Yeah, I don't know. I had like over 5,000 something messages.

Wait a second. Okay, 5,000 in one year? Yeah.

5,000 messages divided by 365.

How many messages per day are you messaging with chat?

I don't know the exact accuracy of this whole thing, but.

Okay, that's 13 messages a day from chat.

That's a lot.

How many messages do you have with your mom? Not 13 a day, that's for sure.

Well, you can't answer all of my questions.

I can try. I can tell you how great you are, and that's what chat does.

Can we talk about being old?

Okay, sure.

Well, did I. I feel like you have been an influence on me in the way that, like, I talk about my age and stuff.

In a good way.

Yes, yes. Like, you have always just been like, oh, I feel like my life has gotten better and better every decade of my life.

Yeah. So.

And I just get. I talked about this on Tick Tock and it got some traction of people in my generation, people in their 20s always talking about how old they are.

Yeah, that's super annoying.

Did have people always done that?

Yes, they have always done that. I mean, I. I think back of so many times. Like, I remember. Or like, we were 30 and people were like, oh, we're 30, we're so old. Like, give me a break. But yeah, Even in the 20s, people are like, oh, yeah, you know, 20. I'm 25 now. I'm old. Like, give me a break.

Yeah. I mean, what, like, do you feel like being a doctor has influenced that at all just be seeing actual really old people or very unhealthy people and just being like, oh, I'm, I'm so able bodied right now.

Yes, it could be. And I do think like the, the older you get, the more you're like, well, that's not old. Like each age is like, oh, I used to think that was old, but now that's not old. At the closer, whatever age I am, I'm like, well, okay, I'm 57 and that's not old yet. So I don't know at what point. But I also like, I see these patients who sometimes are in their, you know, 80s and like that one patient, I was telling you who went to Rwanda to see the gorillas. Was I telling you that?

No, but wow. Yeah.

Isn't that incredible? And she's 84.

Yeah.

Now she had a lot of help in, in getting there and getting, she actually had help like getting up because you got to the pouring rain through this forest up there to see the gorillas. But still the fact that she even like wanted to do that is just amazing. And so I see that. I'm like, okay, well, I feel like you're not old as long as you are still enjoying life and doing thing. And if your health is good enough to be doing things, I don't know. Age, it's just a number.

Right on, man. I just, I mean, people will talk about this. This is generational difference here of people in my generation, they, they say that we're grandmas because we don't socialize as much, we don't go out as much.

We.

And a part of it is this kind of health mindset. I think we're a lot more health conscious, which I think is good because we want to live older and you know, we know a lot more Sometimes that's taken to the extreme though. And you know, with all of the pseudoscience and all of the misinformation of juice cleanses and that sort of stuff.

Right. But.

But also it is just a difference. And also it's a financial thing too. People say that people don't have enough money to go out to the bars or to go and socialize with friends because we don't have. Things are just too expensive now.

That's lame. Whenever people used to say that, they're like, I can't go out because I don't have any money. I used to go out as a young person and just not drink.

Yeah.

I'm like, go to the bars, drink water. Water is free. And You. I mean, I really did that, like, on a regular basis. Be like, okay, that's fine. I'll just go out. I just want to chit chat with my friends and I'm just not drinking like that is. I mean, that's just stupid. That's just like an excuse, I think.

Yeah, I think so too. And I think that people like, oh, well, I want to, you know, I don't want to go to the bars and I want to do something else. Which is fine. Yes, do something else. But also, if all your friends are doing that thing, then you can't just make the excuse. Oh, I can't go. You know, if the point is to hang out with your friends and just do what your friends are doing. I don't know.

Yeah, I, I remember knowing people like that when I was young, or you had plans and you were gonna go do this thing, and then they would call at the last minute, be like, yeah, I just don't feel like it.

See, that's a.

That's not relevant.

Is that still see that? I think the rhetoric right now is that that is a my generation thing because people are really big on setting boundaries and self care and these things. And there's been all this pushback of being like, okay, well, no, I don't want to go hang out with my friends because I need to take care of myself. When then sometimes it's pushed to such an extreme where you're not showing up for your friends when they need you, and then you don't have the social circle and then you don't have anyone to go hang out with because.

Yeah.

So I don't know. But do you feel like there's been a shift in that or you think.

I think there's always been people that were like that.

Yeah.

Now they just feel empowered to be like that. Yeah.

Yeah.

But I remember people like that, and I just remember that being a thing. Well, I don't really feel like it. Well, you know, a lot of times you don't feel like doing something at the time or like right before. It happens to me all the time I'm like, why did I agree to do this? Every time I run a marathon, I sit there the night before and say, why? Why did I sign up for this? I don't want to do it. I'd rather be doing anything tomorrow than running this thing. But sign up, got no choice. Got to do it. And then you're glad afterwards. But that's a little different than like going out to dinner. But there were always People that were like that, and a lot of them did just bail at the last minute. And I just thought it was, you know. Well, and what happens is that then you just don't invite those people anymore because we had plans. We were going to do this thing. There were only three of us, and now you dropped out, and now there's two of us. And so then a lot of times it just sort of falls apart. And so you're like, I just don't want somebody who's going to bail on me at the last minute.

So you don't think that my generation is just soft?

Well, that is true. But, no, I don't think you're any softer. I don't think you're any softer than any previous generation. I just think you guys let people off the hook, whereas I'd be. I'd just be like, no, you know, you're just being lazy.

Well, you can empathize with them and say, you know, that's okay. You don't have to do this.

But then if you want to be my friend, but if it's a pattern.

Of behavior, then that's different.

Yeah, it's. Yeah. And, you know, there were always those people also who were. They were your. You know, they were friends, they wanted to do stuff with you until they got a boyfriend, and then, boom, disappeared. Yeah.

I mean, that's just, you know, story as old as time.

But. Yeah. And. Well, okay. But one of the difference, though, is that, okay, if the plan was that people were going out that night, whether it was to restaurant or bar or over to somebody's house or whatever, and you didn't want to do that thing. There wasn't any. All you could do was stay home and watch one of the three channels on tv. There was nothing else to do. It's not like you could watch a movie. Well, I guess you could. You could go to Blockbuster and rent a video cassette and bring it home and watch it, but that's a whole big effort, too.

That's so true. I guess there are just a lot more options of entertainment that you can do by yourself. Yeah. What did you do by yourself? Read a book.

You could read a book. That's true. But you'd get pretty bored. And you're like, all right, I'll go over to your house and we'll order pizza and hang out.

I actually didn't even think about that. Yeah.

And also, just like, the. Being able to kind of communicate with people, too. Like, so you're maybe like, texting or messaging people throughout the day, but that you weren't doing that. Like, if I was at the library study or at home studying, like, I wasn't talking to anybody all day. And so at the end of the day, people were going somewhere. Okay, I want to go do something.

How'd you figure out if people were going somewhere?

Well, you called them on the telephone.

And said, hey, you don't just look at their location.

You don't just look at their location. No, no. You just have to. Yeah, call them up. I do think about that. Like, that was a lot more. There was a lot more coordination of plans for all of those things. But, yeah, so that made you kind of you. You. If you wanted to have anything social, like, that's the way you had to do it. So I think that that's part of the thing with your generation is you're kind of socializing all the time, so you don't maybe need it quite as much.

Yeah, kind of.

But you still need that in person interaction.

Nope. Just gonna talk to ChatGPT.

Or people could just listen to our podcast.

There you go. Shameless Plug. Or they could follow you on Tick Tock.

Pull your tick tock, Dr. Davis, PGY30. Be my 106th follower.

Wow. There we go. And you can follow us on Tick Tock Generations md. Same on Instagram. And you can email us ataskgenerations mdmail.com and we'll talk to you guys soon.

Bye. Bye.

Sa.