EP 107: How AI Turns Clinical Trials into Medical Knowledge

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

What does the advancement of AI mean for medical knowledge and clinical trials and all of those studies that that we read and we use to make important decisions? Something I think about and I'll tell you why. But welcome to Everyday AI. My name is Jordan Wilson. I am your host and this is yours. This is all. This is all for you guys. It's a daily live stream, podcast and free daily newsletter helping everyday people like me and you make sense of what's going on in the world of AI. How we can learn the most important things, cut through the noise and how we can leverage that in our daily lives. That's what every day is Everyday AI is all about. And thank you for joining. If you are joining, don't worry. We are going to be talking with a guest today who has some fantastic expertise and experience in what this means. What does AI mean for the medical, the medical knowledge and clinical trial community? We're going to get to that. But first we got some AI news. So big news yesterday I actually went came out of a semi retirement from LinkedIn writing and wrote a little bit of a rant about AI. But it was all based on Microsoft Copilot because now Microsoft Copilot has a release date. So they just announced a release date for Microsoft Copilot November 1st for all commercial customers. So it's coming to Windows 11. It'll be three $30 per user per month. And I actually had a whole episode of what Microsoft Copilot is. But if you're new, essentially it's bringing all of these different generative AI functionalities to an operating system to your desktop. If you think Chat GPT had a big splash eight months ago, it won't be anything compared to Microsoft Copilot. All right, next piece of News, Google and YouTube also going all in on AI. So YouTube has announced the launch of four new AI powered tools to simplify the content creation process. These tools are AI Insights, Dream Screen, Assistive search for creative music and a loud, which is a dubbing, an AI dubbing feature. So we'll explain all of those a little bit more in the newsletter. But YouTube going really, really all in on AI. Last but not least, are skills students learning in college going to be obsolete indeed. CEO thinks so. So the CEO of the popular, probably one of the most popular job searching websites in the world recently talked about different skills and how generative AI is going to impact those fields. So he said that technology and business operational skills are the highest, at risk of being exposed to AI. That's like every single skill out there.

Right.

And he, but he also said obviously that AI can be used to help people land jobs if used properly. Right. So if you want to learn how to use AI properly, tune in every day. We do this Monday through Friday, every Single Day Live, 7:30am Central Standard Time. Join us live. Speaking of that, we already have a lot of people joining us live. Get your questions in now. Because you showed up here for a reason. You showed up here to learn how AI can turn clinical trials into medical knowledge. And we have a great guest that I'm going to bring on. So please help me in welcoming to the show Lefterus Tepper Akitis, who is a freelance medical writer specializing in systematic reviews. Lefty. Right, that's, that's a little easier. Thank you, thank you for, for joining Everyday AI. I appreciate it.

Thanks for having me, man. And good job in pronouncing my name.

Oh man, I can, you know, it's. At some point I should run a highlight reel of all of the people's names I have messed up and, and that would be in there, but left yours. Thank you for joining us. Just, just tell everyone, just real quick a little bit about what you do. What is a freelance medical writer specializing in systematic reviews? So yeah, just, just give everyone a little, little brief intro to what that means.

Of course, I'm a pharmacist by training. I've also completed clinical pharmacy residency and in theory way back when, I specialize in emergency medicine, but I never actually put that to practice. For the last close to 20 years, I would say I've been doing systematic reviews and what that is another word for it, another term for it is evidence synthesis. We take all the available clinical trials on a given topic, compile them together using validated methods and highly accepted methods overall, and we end up reaching conclusions that are meaningful in some way, one way or the other for the people we actually care for.

Yeah, and you know, we were, we were talking about this a little bit right before the show. I actually have a background kind of in this, which is strange enough, but for seven, I think for seven years I essentially would, would read all of these long scientific studies and I would kind of rewrite them for the everyday person and say, hey, here's these long studies and here's what it means. So at least in your field, how does what you come up with impact kind of the everyday person in terms of the different medical knowledge that we are all receiving.

So there's two major areas that I would focus on. First one is clinical practice guidelines. The entire medical practice today is based on a process where you have people like me running a systematic review. Let's say you have whatever hypertension. Right. I'll run. Well, my team and I are going to run a systematic review, essentially gathering all the available data on all the different medications available for hypertension. As soon as that's finished, this report goes to some people we refer to as KOLs, key opinion leaders. These are the top physicians and the top institutions with the highest level of knowledge and experience. Based on what we write, they come up with recommendations and that's how, when you actually go to a doctor for whatever health issue you might be facing, that's how they know which drug is best for you, how to treat you. Everything is based on these guidelines and guidelines are based on systematic reviews. Now, if I may, a little side note, it's a really time consuming process. The actual systematic review might take up to a year before it's ready and then by the time the Kols get together and vote, because it's actually a voting process that might take another another year or so, essentially we have guidelines that as soon as they're published, they're already two years old. And at the pace that clinical research is actually moving, it could be old news a lot quicker than two years. That's one of the major, major reasons why I'm excited about AI coming in, speeding up this process and that can actually have significant impact on every single physician all over the world getting more up to date knowledge on the latest developments in their field, the newest medications and treatments. So essentially I think the biggest impact will be through quicker and better clinical practice guidelines. Now the second thing I would like to touch on is the approval process of medications, medical devices, in vitro diagnostics, every single thing within healthcare at one point or another will require a systematic review in its approval process. Again, we can increase the quality and decrease the approval time for medications and medical devices and so on. So much quicker access to novel treatments would be another major point. So I think that's the two most significant parts.

Wow. Wow. So a lot, a lot to get to, a lot to unpack. Just as a reminder, thank you for everyone joining us. Dr. Harvey Castro showed up here in the house. He says great to be here. Maria joining us, saying hello. Peter, thank you for joining us. Val saying good morning. Jack saying hello, thank you for joining us. If you have a Question for lefters about this whole process that we're talking about. Please get it in now. That's the great thing. And if, if you are listening on the podcast, don't worry, we always put a link to this conversation. You can come in, join the conversation. Please do. But I. You left us, you said something that really stuck out to me. So in the current medical field, so whether we're talking about know new, new breakthroughs in, in me medication that can really help people, medical devices. So you're saying oftentimes it is a multi year process to get something fully approved and fully on the market.

No AI can change that. There's two things, right? So the guideline thing, the drug's already out, it's approved by the FDA or whatever authority, it's just not incorporated in the guidelines or what is standard clinical practice. Right. The second part I spoke about. Yes. Drugs take way long, a really long time to get approved and AI can certainly help with that as well. Yeah.

So I would think right after hearing that and knowing all of the wide ranging benefits that the everyday person can have from, from not having to wait in some cases so long, you would think that the entire, you know, whether it's the medical community or the medical writing community would be thrilled about this. But that's not the case, right?

Unfortunately, no.

So, so why do you think that there is this, this split opinion? Because I, I hear your enthusiasm, I'm enthused about it. I'm saying, yes, let's get. Why are some in the medical writing community maybe not as excited about integrating generative AI into this process to expedite it?

Yeah, that's really good question. Essentially I think there's two elements to it. First, one is the biggest hype about AI was and is that it's going to take away our jobs. And to be honest, I'm also quite fearful of that. I actually managed to prompt ChatGPT to perform an entire systematic review, start to finish. Got that published a couple months back and we are now working on a second similar project. Now all that these prompts are actually missing is an automation behind them just to get rid of all the copy pasting from one environment to the other. And there you have it. Systematic reviews can be fully automated, meaning you can start with the topic, upload some documents and get the final report sometime later. This is probably a year from now or two years from now. I really can't put a number to it, but yes, the biggest fear I guess within the medical writing community is that hey, why would we use a tool that will ultimately take away our, our jobs. Now the second thing, and this is where it starts to get contradictory because I've actually heard people the same person use both the arguments. The second argument is it's unreliable, it can hallucinate, it can get you in a world of trouble. Well, how's it going to take away my job? If it's on, it's on if it's unreliable, you know what I mean? So ultimately a lot of people are in denial for either of these reasons. And I think the third element that, that should be taken into consideration is we really don't have a. Most people, at least that, that I speak to, don't have a clue as to how to use these tools. I think that's as soon as this becomes more evident, a lot of people are going to get a lot more hands on experience. For example, let's say we have a tool that comes out that is validated, that we know what to expect of it, that we know it doesn't hallucinate and things like that. I think a lot of medical writers are going to start exploring things like that. However, at the time there is a lot of, I guess the words denial. I mean, I watch your show all the time. We're in complete agreement that AI is here to stay. It's not going anywhere. Might as well get acquainted with it sooner rather than, rather than later. So that's my take on it. It's extremely impressive what this thing can do and I think it's just a matter of time. But we are facing at the moment a very large percentage of people in the field who just don't want to bother with it.

So that I am seeing both sides of this, right? Because I'm seeing the side kind of the, the tech enthusiast and the AI aficionado in me wants to push, you know, wants to push for this and to see this improve things and to, you know, bring hopefully more, more help and treatment and resources for people on the back end. But the other part of me understands the potential risk too, right? Because we're not just talking about some writing on a website. You know, this isn't a blog post that we're asking ChatGPT to generate. It is something that has far reaching and very impactful consequences one way or the other. So in your situation, as someone that is generally advocating, you know, for, for, for generative AI, how can you and others in the medical community, you know, maybe even specifically, you know, in the systematic review community, how can you go forward and you know, advocate for this, yet find that balance of, you know, hey, we're going to be doing this in a responsible way that ultimately has the greatest benefit for the most people.

Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too, just like some of the biggest companies in the world do. Go to your everyday AI.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen.

The operative word would be validation for this. So let's take anything really that gets introduced to the medical world or field. In order for anything to get introduced, it has to go through rigorous testing. We have three phase the pre clinical phase where we test a new medication on lab rats. Once that goes well, it goes into the clinical stage. There's three different phases before a drug gets approved and if necessary there's a fourth stage called post marketing surveillance or whatever. This whole process can take up to 20 years. Essentially this is something that medical writers understand. This is something that MDS or anyone in the field understand clinical research. So essentially the way, at least in my view, to go forward is to get these tools that are being made available to us every day and validate them. Run the equivalent of a clinical study. So let's say you have a tool that retrieves articles for you. You feed it a question, does aspirin help with hair loss or whatever the question might be, and it actually produces a bunch of relevant references. Now what I would like to know when using a tool like this is, all right, you gave me 4.5x relevant references, is that it? Or are there another five that demonstrate the opposite thing? Or is there another five that demonstrate the same thing? I need the full picture. Just getting a reference or two doesn't really help me 95% of the time, I need to have the full picture. Someone needs to take these tools, run them by a lot of questions, check out the output and address what I just described, publish those results, and essentially these reports would be, hey, this is the tool. This is its strengths, these are its weaknesses. This is what you can expect of it. This is the way to establish, at least in my view, to establish trust between the medical writing community and these tools that are being developed. I think this is the required next step so that the two worlds can actually come together more peacefully.

Yeah, and just, you know, Lefters brought up something great there that I want to point out to everyone. So even, you know, what, what he's talking about, this, this process for validating tools, that's not just, you know, for people in the, you know, systematic medical review community, that's for anyone, you know. So if you're using a generative AI tool in your workflow, in your business, you should always be validating, Right? I always do a test. If I'm going to be, you know, uploading a PDF and having a conversation with a document, I'll always hide, I'll always hide a little piece of information on page, you know, 150, something that's not relevant and ask. And, you know, so that's one just, you know, very easy way that the average person can validate their processes. So have a great question here from Peter. Peter, thank you for your question. So he's asking, how do you manage the bias that AI can generate in the clinical trial systematic review? Because as we know, large language models specifically can reflect implicit human bias. So how do you think, especially in your field, how can you combat that?

That is an awesome question. And I was actually talking about this both this morning and yesterday. There are areas, or I guess parts of a systematic review where it's just about data. So there's no potential for human or AI generated biases. However, there are steps in a systematic review process that are not 100% objective, which leaves space for subjective opinions, which leave space for biases. Now, the one thing that I will say is that humans have biases just about as much as AI does, the difference being when humans subconsciously run into whatever biases, that's always different because I'm different, Jordan's different, Peter's different. So it's a different set of biases. However, when AI runs into bias, it can be consistent, it can be predictable, and that, to me, it's not really about removing biases, it's about being able to predict and identify the biases. And AI actually helps us with that simply because it's predictable versus you take 100 reports written by 100 different researchers, you're going to get 100 different sets of biases and somehow you got to make sense out of that versus 100 reports written by AI with the same set of biases that you can just simply filter out or create some type of protocol to deal with them. I mean, it's great help. I'm not saying it's bias free, definitely not. But being able to understand where the biases come from, predict them and handle them much easier with AI versus human biases.

Yeah, it's such a good point. Yeah. Because even pre generative AI, there's always going to be bias. Whether we're talking in medical writing and systematic review community or anywhere else, there's always human bias, whether the extent to which it is played out is another story. Right. So thanks for that one. Another great question from Cecilia. Cecilia, thank you for joining almost every single day. So, Cecilia, asking how are these systematic review processes being improved to assure impacts in diverse populations are being considered in clinical trials? That's a fantastic question because I do think that, you know, you are seeing a lot of just, just a lack of diversity in like so many different avenues of generative AI. But, but. LAUGHTERS what's your, what's your take on that? On, on the impact of diverse populations?

It's really not going to be much. Systematic reviews are performed once the clinical trials have been completed, their results published. We describe this as secondary research in a sense that we need the primary material, the clinical trials, to run the process. So yes, I can help in that domain, but not through the systematic review process. That's, by the time we get to do it, the population has already been recruited, the trial has already been executed, completed and things like that. So yes, there are ways AI can help with that, but not involving the systematic review process.

Yeah, yeah, that makes sense. Like in the systematic review process you can only make do essentially with, with what you are given so that, that makes complete sense. So I do, I do have one more question for you and I know we've, we've, we've been all over the place in this conversation and I love it because we've been able to dive in deep and cover a wide range of topics. But I'll ask you this because there's a lot of just buzz, I'd say, in the medical community in general about how generative AI is going to impact different things. So it's as an example, we talked like, hey, your field, medical writing systematic reviews, that's obviously going to be large. But then also when we're talking about medical and health, what about with the direct patient care? So I know I have Dr. Harvey Castro joining us on the show next week, so I'll have to ask him this similar question as well. But where do you think ultimately generative AI is going to have a more profound impact? Do you think it's going to be more kind of in your space, you know, with getting all of this information out, getting these, you know, new treatments, new medicines out faster, or is it going to be when I go and see a doctor in direct care? What's your take on that?

Honestly, we'll have to wait and see. I know what I would like to say. I know what Harvey would like to say, or I think I do. But ultimately there's going to be patience that gain from this, regardless of where the biggest impact really is. And it's really hard to quantify. That's my main issue for avoiding providing, giving you a direct answer, because I do understand what the impact will be with direct patient care, and it's going to be humongous. I do understand what the impact is going to be on medical writing as a whole, not my specific specialization. I can name a couple other areas as well. So quantifying the actual impact might be a little difficult. So hopefully will all be around to witness and experience this. And again, the ultimate goal is better patient care. Whether it's the physician treating a patient in an office or a hospital, or all the background work that's going on so that the physician can actually do their job. We'll have to wait and see.

So one more thing did pop up, of course, Last, last thing. So let's say you know someone else right now who's listening in the medical writing community doing systematic reviews. What's. What's the one takeaway that, that you hope that, you know, maybe some of your peers, colleagues, others in the medical field, even take away from today's conversation? What's that one thing, as you say, hey, you, generative AI is impacting.

The.

Medical and systematic writing reviews. What's the one piece or the one takeaway that you hope people can hear and take with them?

The reason why I'm so excited is I'm always complaining that there's a lot of legwork before we can actually reach some conclusions. Like I said, even according to whatever guidelines we have for performing a systematic review, it's clearly stated, a systematic review should take about 12 months to complete. Now, in the industry and in the pharma industry, we take a lot less time, but we still may be working on a report for a month or two months before we can actually reach the conclusions of, you know, whatever topic we're addressing. AI is going to reduce this time significantly. And what I'm always complaining about is too much legwork, too little interpretation, too little understanding, too little of the fun part of a systematic review, which is essentially to answer a question, right? So the one key takeaway is that in the near future we're going to be doing a lot of, a lot more understanding, a lot more interpreting of data rather than just reporting it and not even realizing what it's about.

Such, such great insights into this. You know, hey, I even said at the top of the show that I used to, you know, do some kind of, you know, a little bit of writing in the medical space and I had zero clue about this. So lectures. Thank you for giving us all this insight and intel to something that ultimately does impact all of us on a day to day basis. Thank you so much for taking time out of your day to share with the everyday AI community. We appreciate you coming on the show.

Thanks for having me, man.

All right, hey, just everyone, as a reminder, we went over a lot, so even if you didn't have your notebook out, if you were out listening while you're walking your dog or on the treadmill, because I get those emails. Thank you all. Like I always love hearing where, where people are listening to the podcast. But don't worry, we're going to have so much of what Lefter has talked about in the daily newsletter. So go to your everydayai.com Sign up for that newsletter. We put it out every single day, taking an even deeper dive from each and every conversation that we have. And make sure also go to the website, click on the episodes up there, click on the AI learning tracks because we've had more than 105 episodes now. So there's so much that you can go in and dive into. So thank you for joining us and we hope to see you back again for another episode of Everyday AI. Thanks y'.

All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit youreverydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.