Eric Miller & Eric Bauer: Artificial Intelligence in EMS (4.18.25)

Good morning. I'm Eric Miller, humbled to be here because you guys are all some of my heroes in ems and.

I live in Colorado Springs. I've been a paramedic for God, 40 years, believe it or not. Also critical care nurse, spent a lot of time and actually where I ended up meeting Eric have been a flight nurse. I retired from that about three years ago, but flew with Med Trans Corporation as a travel flight nurse. So flew all over the country with that then. I'm also a lieutenant colonel in the Air Force Reserves and currently the interim commander out at Whiteman Air Force Base for the Medical Reserve Medical Unit at Whiteman in the middle of Missouri. Last year I did a disaster medicine fellowship with Beth Israel Harvard in Boston. So I came out and did a in residence fellowship and part of that fellowship led to the research paper that just got published in January.

In Pre Hospital Emergency Care that we're here to talk about today.

Eric, interrupt you I just wanted to say talk to Peter, get my contact information. I'm going to be in Colorado Springs in about a month or so. It'd be great to see you when I'm out there.

Okay.

Okay, thanks.

Perfect. I know all the places to go.

Yeah, that's what I want to know.

Great.

All right, well, thank you for being here. One take it.

So what brought us together is the study that I did. I can read the title, it's a long one. But the accuracy of commercial large language models, basically ChatGPT to predict the diagnosis for prehospital patients suitable for ambulance transport decisions. And it's a diagnostic accuracy study. And so before we started recording, we were already talking about this. This data collection was done just over a year ago or right around a year ago. And the changes in AI, even ChatGPT.

Have changed considerably. So it'd be interesting to see what the results would be today based on that. But we at the time there was basically three very popular.

AI models out there. Google had theirs.

Microsoft had bought Chat GPT. And CHAT was really the one that everybody was talking about. And we thought why don't we take some patient care reports and see how.

Seasoned paramedics would would come up with their differential diagnosis versus CHAT GPT. And so that's what we did. Brad Newberry's what is brad's National Medical Education Training center up here in Massachusetts. He was gracious enough to let us use his database. We did a. We got an IRB Institutional Review Board exception through Beth Israel because there was really no identifying patient information in it. The we he Has a database of thousands of patients that his students will, during their rotations, write up the SOAP reports. And so out of those we randomly got a thousand reports. Then we narrowed that down to 104 reports. One of the criteria with this was that they had to have a.

A diagnosis that they were leaving the ER with so that we had some kind of framework to choose the reports. And out of all of those then we just randomly selected them. The two things that we did exclude were psych and mental health and substance abuse. So none of the, all the patients were either medical or trauma and primarily medical.

So out of Those we had 104 cases. We chose five paramedics from across the country, literally coast to coast. There was four males, one female. Average age was 51, average went from 37 to 61. They served a population. Most of them were in the 600,000 range. But this one was more of a smaller community, 8,500. And the largest was San Diego with about 1.4 million. And all of their experience was 15 to 45 years. And the average, I think there was 26 years. So we had seasoned paramedics and then out of that they got to choose. We took the soap, we took the S and the O, the subjective and objective. And with that, then those paramedics, they were broken up, the 104 were broken up and spread across those by paramedics. They got three opportunities to do a differential diagnosis and so they would choose their diagnosis. We put that into a database and then we allowed chat because chat's supposed to be, you know, ultra smart. And we gave chat then 1. Why exclude psych?

I don't know.

Because I, you know what, there was probably a reason at the time, and I may think of it while I'm talking here, but I saw that question pop up.

I, I don't know, I, I don't remember exactly what the reason was. I think it was just to hone it down to.

Amma.

And.

Yeah, I'll have to think on that. Send me an email, I'll give get back to you. So out of the three diagnosis and then the chat, GPT got one diagnosis. We had two EMS educators, of which Eric was one, and we had another EMS educator from Colorado that runs a large paramedic program. And then if they could not find congruence.

Did the ChatGPT diagnosis match one of the three from the paramedics? If they didn't, then we had two board certified errors docs that would decide based on the scenario who was more correct. Was Chat more correct or was the paramedics more correct? And then we had one other board certified ER doc in case there was.

A.

Disagreement between the docs. There wasn't, so we didn't use.

That physician. So as far as the, the results go, ChatGPT and the paramedics, they, they had congruence in 75% of the cases. So 78 out of 104 cases.

They were, they, they had the same diagnosis where there was any kind of disagreement 46% of the time.

Yeah.

26% of the time our docs, the chat was more correct. That was 12.

Six cases they felt correct, so 29. And then there were eight sent where they felt.

Correct.

There was only one chatgpt yaged where it could have caught an issue, tried. I don't have my computer so I couldn't go back and figure out that case was there was one where it had been under and potentially caused problem. But chat would have the patient.

So Eric, I'm not sure, I'm not sure if it's your connection, but is he coming in choppy for other people or just me?

Yeah, it's coming in choppy.

Yeah.

So. So Eric, I would just take, take a, take off your video just so we. Maybe that's what's taking up your bandwidth. I know you're at a hotel. Okay, try that.

Oh.

Oh.

You know what, he'll probably pop right back on in a minute.

There we go. It says I'm on the WI fi. Is this any better?

Yeah, we can hear you now. Yeah. Can you, can you start over with the data summary? Sorry about that.

Yeah, for sure. Sorry about that.

Yeah. So out of the 104 cases.

Chat and the paramedics were in agreement about 75% of the time. So out of 78 cases.

Of the 104 there was agreement.

Then where there wasn't. There was 26 cases where there was disagreement. The ER physicians found that CHAT was more likely the, the.

Correct diagnosis. That would have been 12 of the cases or 46% of the time they found six cases or 23% of the time that the paramedics were likely more correct. And then finally 30% of the cases or eight cases that they were both equally potentially correct based on the subjective and objective criteria. There was only one case where CHAT had under triaged or was felt that they were under triaged. And I was saying that I don't have my computer here. I can't remember what the specific case was, but in most cases we found that chat over triaged the cases. So it would have been a more conservative diagnosis leading to a transport case decision. And so that, that's really all with the results there. There's definitely some.

Limitations to the study. One is small sample size. The, the write ups were student write ups. So this wasn't necessarily seasoned paramedics taken from a large database from paid paramedics that have been doing it for a while. We're limited to only narrative data. So we didn't have vital signs, we didn't have, you know, the 12 lead EKGs in there. We didn't have any labs or any other kind of diagnostic criteria. And there was no fine tuning of chat. So we didn't, we didn't give chat.

Any prompts or programming as it is, as it were to be fine tuned for ems.

And again no vital signs or other data was put in.

So Eric, I have a quick question. I have the paper open in front of me and again I don't think it's listed in here. However, maybe you can remember in the cases that chat was not correct, the 25% of cases where chat was not correct. Do you have a sense of how off it was and what that distinction was? Because ultimately you're trying to figure out a diagnosis, therefore can chat figure out a destination transport decision that that's ultimately what you were trying to figure out or were you trying to figure out was a diagnosis correct? You see what I'm saying?

Well really what really was the diagnosis correct? And then just the inference on that, on whether the, it could help make a transport decision. And I think, and as we were talking before, I think this study would even be different now being a year later.

And if we, if we program things into this and if we provided more data, I think chat gets close to 100% frankly as we integrate, you know, if you're integrating the monitor the vital signs and then I know when I worked for gmr, their, their PCR was able to pull in data from people history. If you start doing that, the metrics on all of this change pretty quickly. And I think it becomes a, it can even become prior to you ever getting there, it's probably with dispatch data going to be able to start leading you down the road. My, my concern though is confirmat things what happens when we're letting AI do all the thinking for us and we're not looking at maybe some nuance.

Right. And I think that that was Dr. Pepe's point. So Paul, did you want to, did you want to talk about that? Paul, I asked Your question about.

What did this thing is being fed without having the patient in front of it and understanding exactly what's happening with the patient.

Yeah, well, I'll just make more of a generic point that for the last 50 years I've recognized that the numbers are always fooling us and you really have to be into it and know what's going on. It's just like when we say let's use the shock index, you know, as a marker for when we should give blood. You need to be looking at the patient and, you know, and understanding all the other parameters that are there. And it's going to take a while. Things could get fine tuned better. Like GPS systems. Today I'm going, they're fantastic. I mean, they almost pinpoint the, the actual minute when I arrived somewhere, despite traffic or whatever else. And yet at the same time, you know, I remember years ago when people were getting lost because they were following the instructions when it wasn't as good as it should have been at that time. Right. So I think it's going to take time for this to happen, but I think always in the back of our heads, you got to have a general.

Let'S say gestalt, like our spart paramedics do, of what's going on here. So that's my only, that was all.

I was trying to get at.

Right, right. And maybe I'll bring in Eric Bauer in here as well to make some comments. But there's this concept of cds, clinical decision support. And so it's not, it's not going to make a diagnosis for you. But as many of us know, you know, when you work in the emergency department today or if you're up on the floors.

There'S, it's, it's, it's kind of shouting at you as you're going through saying, hey, could this patient be septic? Right. And that, that, that is effectively perhaps what this is. This is a clinical decision support. What is the potential diagnosis and then what is the potential transport destination for this patient? So, Eric, any thoughts on that? Bauer? Yeah, I mean, I think, I think.

A few thoughts, you know, so I'll go back to when I evaluated all the data from, from my standpoint. Right. Looking at the ChatGPT versus the paramedic, I was really shocked at how accurate it was and I think something that I may have missed, we were given basically chatgpt and the paramedics were given three differentials. So basically differential one, two and three on what it could be. And even though maybe not every one of them hit number one, I think. And again, I'd have to pull it back up. The data that I, that I evaluated, if my, my recollection is correct, most often or not, it was number two. Very few were number three. But in some way it, it and the paramedics nailed one of the three differentials that it could be based on the information that was given. I think it's also important. And, and one of the areas that I'm studying personally in my, my doctoral pursuits is clinical reasoning. Right. And, and how, how the information that you're able to put into an AI model like that to, to get back the information is only as good as your ability to understand the big picture and what you have. And so, you know, I think that that is also an important piece to, to be able to say, is this something that you could use at the EMT level? Is this something you could obviously use at the more advanced level, but your ability to understand what's, what's in front of you, to actually utilize an AI model I think is important because if you put in incorrect information or your assumptions are incorrect, then your feedback is going to be incorrect.

Yeah, exactly. Yeah, great point. Let me bring in Tanner. Tanner, from the research mind of yours, you're going to be seeing this, and I'm sure you're going to be seeing many more articles about this. What's your thoughts on AI in healthcare with respect to diagnostics?

Yeah, so certainly before I've thought more about this and I think more has been published about the use of AI in aiding dispatchers and doing things like recognizing out of hospital cardiac arrest in dispatching ALS versus BLS resources to calls based on information that's coming in from people who call 911. And so this is kind of an interesting look at, on the other end of the spectrum, once you actually have seen the patient and evaluated them, kind of deciding where to take them. One thing that kind of came to mind as I was reading this paper is, you know, you do use the words under triage a couple times. You speak a couple times about triage. And so this was, was kind of discussed before, but so I was going to ask you, you know, when you, when you say triage, do you mean, you know, allowing this person to remain at home versus transporting them to the hospital, or do you mean transport to a trauma center versus a PCI center? That was one thing that kind of confused me. Looking through the paper, I noticed that in Table 1, looking at the patients where the paramedic was judged as correct versus chatgpt. There was one of the patients was an elderly female with chest pain and the paramedic differential was coronary artery spasm, stable angina, anxiety, whereas ChatGPT went straight to the highest acuity. Instead of that, it's an mi.

Right.

Right.

Now if you're interested in triage and you're interested in. In transport decision, I would say that that's probably a good call on chat gbt's angle. Right. An elderly woman with chest pain. Just based on history and physical assessment. You know, regardless, even though, you know, 90, 95% of the time it's not actually going to be an mi. Right. I'd say that in this circumstance. Right. That patient should always be transported to the hospital for laboratory tests. You need to incorporate ECG findings. Right. Maybe imaging. And so, you know, I guess I just want to get your thoughts about how this can be used for triage, how you, how you want to use this study moving forward in your EMS systems, maybe nationwide.

Yeah, thanks Tanner. That, that's actually a great, great. I think the use of the triage, use of the word triage is more. Your latter point is do we have to go to a cardiac center with this patient versus do can we go. Are we going to drive hour to go to place that's going to be able to do a cath or can we go to the local hospital there maybe that's a way that this could be parsed out that, that way the.

And again.

As I mentioned, most of the time CHAT was going to be the most conservative of the diagnostic criteria. So it's. Yeah, MI for sure would be the more.

I think we, we said it probably over triaged. But again, this is a foundational study. This wasn't supposed to be some breakthrough. Oh my God. You know, this, this is going to solve, you know, this is the study of studies. This is really one of the first studies that was done for EMS and, and AI. So.

Yeah, but that's one thing that.

We did find that overtures the other big thing with AI. I was talking to some engineers from Zoll actually up here in, I'm in Massachusetts right now and they were talking about the legal aspects. So this is what's going to slow down AI.

What happens when AI makes a mistake, who's going to be held legally liable. And I think that will probably slow down some of the implementation of the diagnostic aspects, especially for EMS and where we don't necessarily have all the diagnostic criteria to do that. And then one final point is we think about you know, machine learning and those types of things. But we've been doing this for a while. Cad, Computer Aided Dispatch. But shoot, we've been doing it with EKGs, and, and I, I really looked at. Look at AI right now as the, the EKG interpretation.

Modeling that came in in like the 60s, right. And, and everybody's like, don't rely on the, the 12 lead, you know, the machines diagnostic, because, you know, even now, sometimes it's wrong.

Right, that's.

That's a great analogy. Yeah, that's a great. Yeah. I do want to remind everybody that when you talk about AI, the prompting is a very, very important part of it. You've heard that word a couple of times. But just as an example, I'm going to share my screen real quick here to show you what we're doing in Coral Springs. As many of you know, we rolled out the Siemens Epoch, and we probably have like 60, 70 cases already. And so we take a photo of the, of the printout, not of the screen, and I, I put it into ChatGPT and then it, it texts the medic a diagnosis and then not a doc, but a summary of the results. And I was talking to Charlie Ingram about this the other day that I've had to modify the prompts a few times because it's a venous sample. It was, it was taking the SAO2 and saying, oh, severe hypoxemia. And now the medic is saying, severe hypoxemia. Oh, my God. So I had to tell it, you know, ignore the, the PO2 or the SAO2 and then to the next one. It removed it. Then it was calling anything above, like 0.75 lactate high, and I said, no, no, no, use a value of two for lactate, you know, as the floor. Okay. And now it's. So did you guys change any of the prompts at all, Eric, or do you use. Did you. Did you tell Chad anything beyond zero? Me? The zero. Wow. Okay.

We.

We didn't do anything with chat to give it any kind of prompts or anything like that. And I. And again, back. You know, this is a year ago, I think when we were doing all of this, there was really only three choices of, of AI models. Now, now there's actual medical chat probably is the worst from a medical standpoint. And as I'm saying this, I'm like, I should have probably looked up what it is because everybody's going to be like, well, what is it? I was just at a national.

A federal medical conference, and there was several people talking about Some new medical AI that's out there that's pretty sharp and it's in my notes someplace. I don't have it handy where I'm.

At now, but it could be a bridge. A bridge is a product that people are using. You come into the, into the room in the er it just listening and if you talk about the baseball game, it doesn't add the baseball game into the note. It just creates everything for you. And you know, we are coming to the end. But I just, I want to mention that gems last week at fdic, all of the companies first do eso image trend. They were all there. And basically what they're creating is something where you open up your phone on their app, you press start and then not only will it provide you a narrative that's kind of like the easy part, but it knows that the NEMSIS dictionary and it's going to start filling in all the fields, including physical exam fields, including chief complaint fields, and all of them are NEMSIS compliant. And now the paramedic just has to look and here's the hard part, they have to look and review, which it turns out they're not doing. And they're just saying, okay, so for example, if, if it said, for example chief complaint is chest pain and the word STEMI was in there, but the paramedic didn't think it was a stemi and took him to a non STEMI facility, then without that review, you're going to have a problem with honest and ethical documentation. So I would love to hear anybody else's comments. Is anyone else using AI? Are they considering using it? Are you testing AI? Any thoughts on, on this conversation? Who wants to see a lot of people out here today and I'm wondering, because it's coming, folks, whether you, whether you like it or not, whether you think it's good or not, it's all going to be an option for you to turn on very soon. Eric.

Hey, Peter, let me, let me ask one question and I'll kind of stay quiet. What, what are your thoughts, everybody, about using AI instead of, you know, Computer Aided Dispatch and the cards? Maybe that's the way to use it. Maybe the information is put into AI, a good AI model and that that helps to.

Dispatch tiered response a little bit, a little bit better than, than we already are. Maybe we do less, you know, code three to the scene. You know, maybe that's a way to use it, that's less liability.

There's a company called Cordi C O R T I, I know that they're using it in Seattle, Bellingham, Washington. And effectively it's exactly what you just said. And as soon, as soon as it picks up what it needs to pick up, it does all the rest for you, leaves the call taker, the dispatcher alone to do other important things. And so that, that is currently out there today.

And I don't know if the legacy systems can make that happen in their system currently, but these AI first applications like Cordi are going to do that, I think to a better degree. Anyone on the call who that resonates, who Eric's question resonates with you or that you're using that.

So I don't have any experience using it, unfortunately.

It's interesting though, I think it definitely puts us in danger of garbage in, garbage out from a dispatch perspective, um, as like dispatching is just so difficult, like call taking is so difficult. And I think that there will likely remain a human component to it significantly of some of these kind of rerouting of, of emotional pieces of it that are, that are just very human. Right. Um, I don't know. I think that it could definitely be an assist, um, but it would surprise me if it is more effective, more efficient. But I don't know. I'm super interested in it.

I would agree with you on that. It's more of an assist type of thing. Dr. Lee?

Peter, go ahead. Yeah, go ahead.

I'm sorry, I couldn't raise my hand. I couldn't figure out how to raise my hand. You know, I, I've been, of course, I think personally, you know, I've been cheating with chat GBT on a variety of things for quite a while now. And. But, you know, and I'm, I'm dealing with, you know, and I've been thinking and looking at this for a while how to best use this. And I think everybody's swirling around the same issues that, and saying the same thing. There's a lot coming in this arena. We're going to have more clinical assistance. But at the end of the day, and I think Paul touched on it, you know, at the end of the day there's so many variables. And I say something to my paramedics all the time. They're always asking me to write a protocol for, you know, this or that. Actually what they're asking me for is to make it black or white. And my answer to that is, guys, I cannot write rules that replace judgment. And so the issue for me comes down, let's say that the rule that I write a lot of rules and let's say, or I use ChatGPT or and with Epic and so on and so forth. And so I had a lot of information. At the end of the day, judgment is still going to come into play. There's still going to be variables where people are going to have to make up their mind in real time. You know, they're going to have to take that information and as Tanner said, garbage in, garbage out, and decide what to do with it. So the question for me is with regard to my. And I have old paramedics and new paramedics. I have very smart paramedics. And I don't want to say not so smart, but maybe not at the top of the list. And so the question, I think the challenge is how do we teach? How do we teach good judgment? How do we teach? You know, And I think along with that comes a certain amount of desire, a need for passion for what you're doing, for, you know, a certain amount of OCD about your job, etc. Etc. So I think this is going to add a lot in terms of assisting, but at the end of the day, I think we're still going to have a lot of judgment arenas for at least some time. And so my question is, how do we deal with the judgment, you know, issue in terms of teaching that? And I'm not sure we can. You get my point?

Yeah, it's a great comment. I'm going to throw a question back at you, then I'm going to come to Tanner, which is you, you've done a lot of medical expert, right? You're like, you, you've done that. You've run huge organizations. You, you, you've been around the block when it comes to medical legal. Do, do you, are you risk averse when it comes to having the AI listen to the medic populate an entire chart for the medic to review, review, review, press. Okay, Maybe miss something.

What's going to happen in court when they blow that up like they do on a big, you know, on a big piece of paper and say that doesn't look like your vocabulary, your language. Oh, did you use AI? Yes, I did. Are there risks for the department, for us as medical directors or as paramedics? What are your thoughts on that from a legal perspective?

You know, that's a very good question, Peter. And I will tell you, it's interesting. The short answer is yes, there's risk. There's always risk. You know, we can be sued for being, you know, very ugly and so on and so forth. And but the. But. And it's interesting you ask that because, you know, many times when I'm talking about cases to my medics and I'm saying, you know, we should have done this, we should have done that.

I always make the point that what I'm most interested in is having a good outcome for the patient. So the patient is the top thing on my list. Risk mitigation is the second thing. Even though maybe it's not the second thing. Probably the second thing is patient paramedic safety. The third thing for me is risk mitigation for the tribe and. Or for myself, for that matter. I will say that.

I have no aversion to using this tool, no more than I have using protocols. At the end of the day, I think the liability. Well, the liability will fall on who has the pockets. I happen to have one of the pockets. I'm the medical director. So if we have AI and I'm using it, I will have to sign off on that. So I will tell you, if something goes south, you know, and somebody, it will depend on, you know, is there money available and who should we. And, you know, so on and so forth. So that's always going to be there. I think that really the. The more important thing is coming up, making sure that we're doing something that is going to help the patient first. That's first and foremost, which I know everybody here agrees with. So. But yes, there is risk to it. And if it goes south and you're using AI, I can tell your name will be on the. On the document. And by the way, the last thing is with regard to. What about this? What about that? The attorneys taught me in many of the depositions, they taught me, the answer was it depends.

It always depends.

Yeah, awesome. Great answer. And I think we ought to have you back for just like a masterclass on medical legal. But I'm gonna go Tanner than Charlie Tanner. Go ahead. I know you had your hand up earlier.

Yeah. So I also had a couple thoughts just about using these models and training them. I think, first of all, whenever I'm writing a chart, especially if I've provided interventions to a patient, I'm really trying to tell a good story that is going to convince any reader that my diagnosis was correct, that led me to performing that intervention or that a patient was suffering from some disease process that I'm suspecting. Right. So if I think the patient has chf, I'm going to be sure to mention, you know, rails. I'm going to be sure to mention, you know, history Relevant, you know, aspects of that patient's presentation. And so I think that from that perspective, we're really making things easy for AI with our.

If we're, if we're using EHR data to train them. Right. Or to test them because, you know, we're prioritizing things that we feel are important and fit with the story and perhaps de emphasizing other points of data that may be important in retrospect. So that's one thing. And then, and then also, you know, I think that if we're going to be focusing on diagnosis.

The gold standard that we use to train these models is important. And so, and this is, this is, I think clear in the literature. I think when you look at, you know, for example, the, the outcome data in the ESO data set, oftentimes the, the outcomes that the emergency department puts in the patient's chart is very different from the working diagnosis that was present in the pre hospital setting. There are some studies out there that look at, for example, administration of Lasix in the pre hospital setting and you know, you go to, you look at the patient's records and their, in their imaging in the hospital and it turns out that like you know, 30, 40% of patients who receive Lasix for suspected CHF have pneumonia or have a pulmonary embolism. And so, you know, I think that moving forward, if we're going to be training these models and actually using them, we need to be sure that we are coming to a good consensus on what a gold standard diagnosis is, not just what's written by the paramedic or what is written by the emergency department physician. Right. There's also the role of anchoring bias. The treatments that a paramedic provides might bias the emergency physician on what they're going to write in the chart. So those are just some things moving forward in this field. Those are the things that I think have to be considered perfect.

Great comment, Charlie.

So I'll go off both of those. Tanner's comment first. Super in agreeance with you that the gold standard is going to be very difficult here. Right. Because where we are making those decisions is going to be super variable. Right. Both in like what information we have going into these as well as like the information out. Right. So like in the er, like I could have anything that is real chest pain that I could just call chest pain. Right. And like that's the privilege of the ER documentation systems at times. And so with that, right. Like you can't just pull these like standard like diagnosis codes either.

Right.

Like you can't you can't pull a data set to train this thing and to test this thing. It's going to be definitely a labor of love for people that are doing it. So, super interested to see what, what comes of that. Back to Wayne's point, which I am not creating an AI documentation system, and we have not used any AI documentation systems or approved them in our agencies at this point. If I were to, I would essentially have a requirement or the thing that the AI piece is publishing that says at the bottom, this was published using or assisted using AI. And it is at the liability of the paramedics review.

Right.

Like that. That directly shifts the, the liability person that is doing the writing.

So, so they are doing that. So I, I saw these demos from these companies, Charlie. They're doing exactly that. They're putting at the very bottom, this was AI generated and now it's on Dr. Lee.

Yeah, right.

And they're going to say that it's, it's AI generated thing and it is up to the, the signature at the bottom of the page to review it and agree with it before signing. Right. Like it's the same thing.

Charts, which I'm doing right now, I have to remove them. Right? That's totally it.

Yeah. Okay, so let's go to Lou and then back to Wayne. Lou.

So to your point before, I recently taught about documentation. And if you're unlucky enough to have more than one of your reports pulled by the same attorney and they look identical, it doesn't look like you're writing a report for that patient. So that's a bad thing. To Dr. Lee's point, I always said you can't legislate good judgment. Just doesn't happen. Third point is, just like with ekg, I always say interpret it yourself and see if the machine agrees with you instead of if you agree with the machine. Same with AI. Come up with your working diagnosis, See if the machine agrees with you. If you disagree, fight it out and decide who's right. But don't let the machine color your judgment. And the fourth thing, remember, in medicine, when in doubt, examine the patient. No matter what the machine says, if your patient looks like crap, something's wrong.

Good. Very good. And I know that you, you actually wrote a very nice article. I believe it was in Gems. It's called the Trees in the Forest.

And, and that's exactly the point of it, that if you get lost in the, by the trees, distracted by all the numbers and all the fancy things, sometimes you miss the forest, which your patient looks like Garbage.

Can you. Can you post that in the link while I have the Dr. Lee coming back on? Dr. Lee, go ahead.

Well, I feel like we should be on my boat with some beer in our hand. This conversation can go on for a long time. But regarding Lou, I agree with you. I can't legislate judgment. I guess I continue to struggle sometimes with judgment that fails, for lack of a better term, which sometimes surprises me. And we try to educate and train on that each time and do better. But back to the disclaimer issues, you know, at the bottom of the chart and so on and so forth. And again, I just. It's just my experience, you know, I did do hundreds of cases as an expert and always for the defense, by the way, and I never got asked very often by the plaintiff. But at the same time, you know, you can write all the disclaimers you want. I think it all helps. But at the end of the day, you know, if you have a bad outcome and, you know, a young adult, a parent or whatever, if you have a bad outcome and, you know, it just depends, but at the same time, you. You can. It will always come back to the medical director in one fashion or another, in my opinion.

Well, you all heard, I'm sure. And I saw Dr. Ingram commented on this post from yesterday that a hospital in. In Central Florida was sued successfully to the tune of $45 million, because a patient came to their hospital who had a stemi, their stemi facility, for whatever reason that day was not up and running, and instead of sending the patient four miles down the road, they flighted the guy to the tertiary center and the guy ended up dying. So that was a $45 million lawsuit. So these things are definitely happening. However, when you look what's happening in the ED today.

There'S tens of thousands of doctors every day, probably right now, using these AI models. And I guarantee you, you go look at that, and then I'm reviewing it. You and I. How many times have we reviewed our scribe charts? Yeah, you scan it, right? And then like a year later when you look at it, you're like, oh, boy, that's not really the best thing in the world. So it's happening. It's going to happen pervasively through all of healthcare. Are we going to sit on the sidelines and say it's not 100% good enough, or are we going to use the baseball analogy where if I'm hitting.300, I'm in the hall of fame and maybe healthcare is not a.300 batting average? Type of thing. You probably have to be like maybe like 900, who knows? But it's coming. And I'll guarantee you that all of the leadership and everyone on this call is going to be asked in the next six months, do you want to turn on this AI feature? Do you want to turn on that AI feature? Do you want it to be back office stuff, dispatch stuff, documentation stuff, diagnosis stuff. It's everywhere. So I think perhaps we ought to continue to talk about this.

Throughout the next year here, just ourselves, to understand what's coming, what's going on, and maybe even bring on some vendors. Jacob put in the chat that there's another vendor he's working with. So.

I think the people who have the distribution, meaning the folks who like the epics of the world and the EPCR companies of the world, those are the people who are just going to put it in front of us.

Like a big fat steak and they're going to say, you want to eat? And at some point I think people.

Are going to end up moving to.

It once those models get better. So great stuff. I'm going to give Eric and Eric the last word. Eric Bauer, we'll start with you. Give us the last word. And Eric Miller. Take it away.

Man.

Guys, I don't really have a lot else to say. I think this is obviously, AI is overtaking our world. I think we have to be cautious. I am a little leery about using AI. I think we have to continue to foster our own clinical reasoning and decision making and stay sharp. And I think if we're using these other models, I think we could become really rusty. So I, I'm a little, a little gun shy in that way. I think we, we gotta, we gotta continue to, to do things for ourselves and continue to study and try to just obtain that excellence.

Very sage advice. Very good. And Eric, just before you go, first of all, thank you for doing what you're doing. You're a paramedic in Colorado Springs who went to Boston, did a year, a year additional, and you're publishing papers. I wish we had many more people like you. So thank you for joining and give us your last words, my friend.

I'll turn off my video. I'm going to be honest, I stopped listening when Dr. Lee invited us to his boat to have beer. So I just need to know where the boat is. I'll bring the beer. A couple of points.

The other Dr. Lee, Dr. Leanne Lee, I thought she made an interesting point in the chat. Was that in her ER they're using it for documentation only and not diagnostic. I think that's a low hanging fruit that really is potentially value add. But as it's been mentioned, you really have to go back and read what's done. I've been reading some reports where a female is, is.

Documented as a male, you know, and, and so you really do have to go back and reread. But I think that's a real good use case for that. And then to Tanner's point with the gold standard right on. And I think if you look at the study that we did, again, not groundbreaking, it's just setting the foundation of a place to start in looking at this. And if you look at the 75% of the time chat GPT was getting it and then if you add in where they, they did the paramedics, where chat didn't agree with the paramedics and the ER docs felt that chat was more accurate. Now we're into the 85% range. But isn't that kind of like our patient? 85% of the people coming to the ER, you don't even have to do the test. You kind of know exactly what's wrong with them.

And so chat is probably hitting the average pretty good. It's that 15%, 10, 15% of patients where there's a lot of nuance in things that the information becomes critical. That's where the problem's going to lie. And I think that's where the gold standard programming, the AI, those types of things, putting in more data sets so that it has a more complete picture of the patient, that's where this is going to become much more handy. And then of course we always have to go back to the human factor that we have to not let it become our confirmation bias and lead us down the wrong path. So again, great, thank you. Thank you for finding it interesting enough and allowing me to come and talk about it again. I think that it's not earth shattering research, but it is a foundation to start the conversation, isn't it?

It gave us a lot to think about. And I'll even, I'll leave you all with this, that what's going to end up happening is that you're going to be at home, you're a layperson at home, zero medical knowledge. You have your Apple watch, you have your aura ring, you have all of your labs connected into whatever app of the day, right? And you're now going to have a few symptoms, you're going to step on your scale, that's going to be connected, you're going to put on your blood pressure cuff and then someone's going to say the app is going to say, you know what, this is what we think you have. We're sending over a paramedic in an SUV. They'll be there in 20 minutes. This may decimate primary care, the actual need to even go to see primary care because you're just, it's just going to tell you what you need when you need to do it. So I think this is going to move very quickly in both the late the consumer space but also in the, in the healthcare space, in the professional space. So I think we all ought to keep an eye on what's going on. But thanks for prompting us and giving us a great topic to talk about Eric and Eric and we appreciate you guys coming on. Please join Dr. Pepe's webinar in progress in the chat and thank you guys for coming and I'm going to stop the LinkedIn live.

Thanks Doc.

Thank you. Appreciate it.