BoneChat call #16 - Scott Bruder / Patrick Reischling - Clinical strategies and tips

So we are, we're live. And here's to the extra day we're all getting this year. Happy, happy Leap Day. So this should be a treat. A lot of BoneChat members have asked for discussion around clinical studies, whether for regulatory approval or post-market or But it's clinical studies are black magic for most companies and every startup needs data nowadays. And so I couldn't think of anything better than having Scott and Patrick on. I was thinking of the who's who could talk about this. So Pat's got a, he's built a consulting team that's really world-class and helping companies go from concept to commercialization anywhere along the line. But we're just going to talk about clinical today. And he brought in Patrick to lead the discussion, and I'm going to hand the mic over to Scott and Patrick and get out of the way. Welcome, welcome aboard.

Thank you, Tiger. Thanks for the opportunity to participate in this austere group and to see some old friends and to hopefully make some new friends as well. We, Patrick and I, have known each other many, many years, but Patrick has joined the firm, I'm going to say about 2+ years ago, Patrick. I'm just— plus or minus a few months, give me some cred on that. Yeah. Um, and, and we're gonna walk through about 20-something slides, but the idea is for this also to be interactive. We don't want to give a lecture that at the end you have 25 minutes for Q&A. If something's burning, you don't understand, you have a comment, raise your hand either literally or figuratively and Tiger will manage the admin of that so that we can have a dialogue. We don't just want to talk at you for 25 minutes, we want to be engaging. With that, I'm going to introduce a little bit on two slides and then really hand it over to Patrick. Patrick, if you could go to the first slide and don't press the build yet. So there's two slides here. And when I think about translational research, and as Tiger said, from concept through commercialization, and the importance of clinical studies and clinical evidence to get there, there's two triangles here. And the first here starts with what we call translational research if you're in an academic setting, or even in a nonprofit setting, you're first gonna design your therapy. And now we can go to the build, Patrick, this will stop by itself. And so you've got this in vitro validation and you would do a small animal study. You would probably do a large animal study. And then you do your first human trial. You're some professor at a university and you shout from the mountaintops that you have succeeded. Like you're into humans. And that's fabulous progress. But we know from those of us who are in industry that really the way translational research, when you look at it from industry side, flips this model on its head. So let's go to the next slide. And you can see here, you know, we, we call as a foundation, you have to have enough preclinical data to support an IND or an IDE if you're a device, an IND if you're a drug or a biologic. And when you have that data, you have to make sure you have a supply. You have to do your phase 1 study or a pilot. You have to do your phase 2. And then depending on the approach, you might have 2 or 1 phase 3 studies. And FDA approval isn't enough, 'cause then you gotta get reimbursement and then you gotta succeed in the market. And in many cases, you also have to generate additional clinical evidence. Just to support that launch. Of course, additional off-label studies and collateral studies and new studies can be done in a post-market setting, either as small studies or perhaps even large studies to think about claim expansion and going after new indications. So that's sort of a very high perspective of how, at least we think about, where clinical research fits in this continuum of concept to commercialization. And I'm going to hand over to Patrick, who's really going to talk us through a number of the considerations and pitfalls and watchouts and things to consider in terms of creating value. So, uh, Patrick, take it away, and I'm going to get a cup of water because I'm still coming off a cold.

Great.

Thanks, Scott. Hopefully everybody can hear me. And as we said, please feel free to interrupt with questions at any point. You know, we want to make this a discussion, but we'll try to get through this and leave plenty of time for some questions at the end as well. So just a few things we're going to talk about topic-wise. You know, why do I need a clinical study? And I'll dive into what I mean by that question a little bit more because I'm sure you all are here because you feel there is a need for clinical data. But often questions you hear is, how big of a study do I actually really need? I'm going to go over some things on what you should consider when you're designing a clinical study and just a few tips for successful study execution. You know, we could talk for hours about clinical studies, but hopefully this gives you a little bit of a flavor of some of the high-level things that I like to think of when it comes to first considering a clinical study. And that starts with you know, why do I need a clinical study? And when I say that, I don't just mean do I need one or not, but thinking of all the reasons that you need clinical data. And when I very, very first started out in clinical research years ago, I had a mentor that kind of taught me this idea that, you know, translates to a lot of different things beyond clinical research. But this idea of beginning with the end in mind. And so when it comes to clinical studies, I think it's really important to really try to envision when this is all said and done and I've got all the data in hand, what am I going to do with this? And thinking of all the things you're going to do with it beyond just what might be the most immediate need. So, you know, it's important to come up with the why first. Why, why do I need clinical data? And then from that, generate what claims do I want to make out of that clinical data? And I, I kind of put claim in parentheses because there's a regulatory implication of that. But sometimes beyond that, it just means, you know, you know, what do I want to be able to say about my product when this clinical data is all said and done in the end? So the whys, you know, most of these are—

Patrick, sorry, but Elizabeth already has—

Elizabeth, yeah, go for it, Elizabeth.

You're on mute, though.

Yes, thank you. Hi, Elizabeth Hoffines here, writer, editor. And I have interviewed about 2,000 orthopedic surgeons. I just wanted to ask you, you mentioned there are the immediate, perhaps obvious reasons why you would need the clinical data. But there were also some longer-term things that might not be so obvious. Could you give an example of maybe somebody who missed that? Yeah.

Or, uh, absolutely. And, and, and this slide I'll start with, you know, a big, a big miss I see is, is going from the first one here to the second one. You know, you may have a product that you go to the FDA and they tell you you need human clinical trials to get this approved. That's an obvious regulatory need, right? You may have to do a Phase 1, a Phase 2, a Phase 3 study. That's a need to do a clinical research study. But I've seen companies dive headlong into that, having not thought about coding and coverage yet. And what are we going to do as far as how are we going to get this product paid for? I think we're all well aware that in some cases it's actually harder to get things covered by insurance than it is to get them approved by the FDA. And so if you're going to make a big investment into particularly a large pivotal clinical trial, it you should be thinking about what are my coding and coverage needs going to be when I get on the market as well. And that's where I've seen some misses sometimes where, you know, I've seen well-designed randomized clinical trials done. They get to the end, they're working on their FDA approval, and then they're starting to translate it into the market. And they're, they're then suddenly realizing, oh, but we, we need to get a new code for this procedure. Or maybe there is existing codes But, you know, you need to work on getting a coverage policy in place with payers. And you have market access folks come back and say, oh gosh, it would have been nice if we had these data points as well, if we had some quality of life measures, or, you know, did you collect your operative time? You know, you're reducing operative. And so it's important to have the conversations on what may be some important data points to help you in your coding and coverage arguments down the road. Before you even start into it. Because again, if you're going to make that kind of investment—

sure.

And that's one of the biggest areas where I see a miss is, is that, you know, big trials get put underway for a specific use like FDA approval without also thinking about what also can you get out of this trial that can help with coding and coverage. In some cases as well, I've seen where, you know, it's real common in orthopedics to have a 2-year endpoint. To your primary endpoint in a study. But you may have some things that you could do some pretty nice, at least podium presentations, if not papers off of some 1-year data. Is there a way that you can design your study to give yourself some allowance to analyze some of that 1-year data and at least get it on a podium to talk about while you're still waiting for your 2-year endpoint analysis? And sometimes you can't always do that, but in many cases, you can design your stats to give yourself allowance to do that. Might as well plan to do that up front. Um, so that's where kind of thinking about, hey, when I have this in hand, what I want to do with it down the road, rather than just only the regulatory strategy. Uh, thank you.

Yep. A great example would be Barricade with Intrinsic Therapeutics. I mean, they spent $100 million in 10 years and then try to, try to answer the reimbursement question after the fact. Right.

Vinay.

Hey, hey, Tiger. Hi, Patrick. You know, you made a very good point about, like, if you have a 2-year endpoint, but you want to plan a 1-year outcome publication podium presentation. How would you navigate that with respect to the commitments that you would have made with FDA if it's an IDE, or, you know, where you are trying to submit a PMA at some point, and, but then you're trying to publish before the FDA approval or PMA submission. How do you navigate that landscape?

Yeah, that's a great question. That is where it is more challenging because you may have to maintain some blinding through 2 years, for instance. You may not be able to do a comparison between your product and your control group. For instance, at 1 year. And that's where you may run into issues with that. Sometimes, though, you can still do some broader presentations that are still blinded, that are more descriptive statistics in nature, for instance. The most common area I've seen orthopedics is to do things like looking at some MRI results, for instance, where you're not actually breaking into your primary endpoint analysis. But maybe you're doing some imaging results, that kind of a thing. But you're absolutely right. When you have a double-blinded randomized controlled trial with the FDA, you can't just do what you want at 1 year if 2 years is your primary endpoint. You have to plan that out ahead of time, which is why I'm saying it's good to have that discussion with your biostatistician ahead of time to see what, what are you capable of doing without putting your primary endpoint at risk.

Got it.

Thank you. Patrick, here's a question, an anonymous question actually. For a 510 approval type of implant, is it required to do a clinical study?

Yeah, that's a great question. Scott can speak a little bit more to the regulatory side. There are some products that require clinical study with the 510, like de novo products, for instance. Not often the case. Most of the time, what I see with 510 products is, is that you wouldn't need a clinical study to get it approved by the FDA, but you may find yourself upon that approval wishing you had clinical data in order to support coding and coverage or just compete against your competitors, you know, which is kind of the third main point here is market adoption, competitive advantage. I see that a lot with 510 products where, they don't have a clinical study needed to get on the market. But then immediately, you know, I shouldn't say immediately, but within, you know, the first 5, 6 months post-launch, they're getting all this feedback back from the field on how, you know, the VAT committee at the hospital won't purchase their product over the competitors because the competitor has data and they don't. And so oftentimes I see a lot of folks scrambling after the fact trying to gather some clinical data, um, to have a competitive advantage against their competitors. Uh, so it's something again to think about ahead of time. And I would say if you're going down your product development pathway and you are looking at the regulatory landscape and the answer to do I need a clinical study is no from the regulatory perspective, still be having the conversation early in the product development, but Do I want to or need to have some clinical data for coding and coverage or for market adoption or, you know, to stand up against my competitors? Be asking those questions because oftentimes what I see with companies, again, one of those misses I was talking about earlier is they say no to the regulatory requirement and then they just plow ahead into launching the product. And then it isn't until several months after product launch that they realize, Ooh, I wish I would've had a clinical study underway. If you, if you have a $510 product that's in that realm, a great way to handle it is to set up, set up a plan to be able to start a clinical study right at the time of launch. So at least you're getting the data as early as you can instead of starting to start that or trying to think about that process several months after the fact.

Patrick, I'm just gonna, add a little color, sorry, and say that that question elicits my favorite answer, which is, it depends. The question is, does a 510 need clinical evidence? Some of them do, for example, like vertebroplastic cements that are all cleared through a 510. You have to do a clinical trial for that, we know that. Most, I'll say most 510 pre-market approval pathways do not require that, But if you know that your product is going to be cleared through a 510 and you want to have data at the time of launch, then you can undertake an IDE study prior to even applying for your 510. You would undertake the IDE study, uh, and, and collect the data and have that data available at the time of launch. Because we probably all know examples of products that were launched and then it took 18 months to get any clinical evidence. And the management of the company's wondering why their adoption isn't just on the steep part of that hockey stick. So, and those are things that, you know, we're happy to talk to you guys all about. If you, you know, if you have questions and you want strategy, that's stuff that we do.

Go ahead, Patrick.

Thank you. And I'd say, you know, some of those other kind of long-term goal thinkings as well as regulatory approval, you know, we all think about the FDA, But I've also seen instances as well where, you know, let's say a product started in a startup company or even, you know, within a larger US company with the goal of getting it FDA approved. And then after the fact, they thought, gosh, it would be really nice to get CE mark for this product. So that's one of the things to think about as well. If you're going to invest in a clinical trial, are you, or maybe a future strategic partner, if you're looking for a partner or looking for somebody to buy your technology, Could they be interested in getting a CE mark of the product down the road? And if so, is there anything that we need to do different clinical study-wise to help make sure this dataset can be utilized for that as well? So even if you aren't planning to launch outside the— oh, outside the US right now, sometimes it's good to think about, is there a potential market for this outside the US, and could this dataset then become a valuable piece for anyone trying to launch outside the US? So Another thing to think about. And then the last one, you know, obviously in small startup worlds, sometimes you need some clinical data to help you with your investment. You know, I've seen companies that struggled to get enough funding for a pivotal clinical trial, but they're able to get enough funding for a pilot study, a feasibility study, and that was enough then to get some investors on board. So sometimes getting a little bit of clinical data can help you on the investment side as well. All right, moving on from there, you know, kind of the next topic is, you know, how big of a study do I need? And this isn't as simple as a, you know, from a purely, uh, scientific standpoint, you think, well, you pick your primary endpoint, you have a statistician do a power analysis, you know, on it, you know, look at your delta between your, your, your treatment groups you're wanting to do, and boom, you get a sample size. But I think we all know in practical reality, in industry, You have to then look at, you know, well, how long is it going to take to do that? What can I afford to do? And those factors, you know, end up weighing in on it. So before you jump into how big of a study do I need, it's, it's again, really important to understand the why for your study. And then I think equally important, that sometimes is difficult to do early in the product development cycle, but is to start really understanding the business case. And obviously this presentation isn't about how to develop a business case for your product, but the more you know about the market potential of your product, the easier it is to make decisions on big spends like clinical studies. If you don't really have a sense of what, you know, your revenue stream could look like with your product, it's very difficult to know if You know, is a $100,000 study all I can afford? Can I afford a $1.2 million study? You need to have some sense of a business case fleshed out to help you in your decision-making on your clinical study. And then again, when you have that why, you know, what are your end goals? Are you publishing papers? Are you trying to get regulatory approvals? Are you trying to get market adoption? Then you want to think about what do you want to claim? In that. And once you develop what you want to claim, which could be in the regulatory sense as well, that's where then you get to, well, how many patients do I need to prove it, to prove those claims, to be able to say that? And often that kind of feeds back on itself. You may come up with a great idea of something you want to know, want to be able to say, and then you realize that you need 1,000 patients to be able to say that, and you can't afford to do it. Because your business case tells you you're never going to get your money back if you do a 1,000-patient study. And so you kind of go through this cycle again. Well, what do I want to say instead? And along with that, questions like, do you have any prior data for estimating your power? Whether it's your product itself, which oftentimes we don't. Is there similar competitive products that have some data out there Anything that you can pull data-wise to get a sense can help you mitigate your risk. And that's where sometimes it makes sense to start small, to start with a 20-patient trial, a 40-patient trial. It can be very risky to jump straight into a pivotal trial if you have no sense of, of, you know, the effect size of your of your product yet. And so sometimes doing a 20, 40-patient study, even though long-term timeline-wise, that may seem like, well, gosh, I don't have to do that, why would I do that, can help mitigate the risk. It can be difficult sometimes to jump right into the investment of a large pivotal trial if you really don't have much of a sense yet. On how your product's performing. So sometimes small studies are the way to start, just even to help mitigate the risk of that larger investment. Um, you know, funding drives a lot of it. Um, and, you know, how long is it going to take to complete? Kind of back to those $5-10K examples, I've had many times where I've actually sat down with clients and we've ended up helping them decide that it didn't make sense to do a clinical study. If you're in that boat where, say, you know, you've got a bone substitute material that you got 510 approved, 6 months into launch, you're realizing you're really losing against competitors and the field is begging for clinical data and they're saying, we need a clinical study. And then you add that up and you say, okay, well, we can do one, but it's going to take us a year to get it off the ground and get it enrolled, maybe another 6 months later, and then a 2-year primary endpoint. And then we submit it to a journal to try to get the manuscript published. And in the meantime, version 2.0 of your product is due to launch in 2025. It doesn't— may not make sense to do a clinical study at that point then. So knowing how long it's going to take you to get this accomplished again really helps with the business decision. And then as Scott alluded to earlier, you may need more than one study. Particularly if you're dealing with a biologic, for instance, within the FDA, start asking the questions ahead of time, either through, you know, folks like Scott's team and myself, or in discussions with the FDA themselves. You may be required to do multiple pivotal trials in order to get approval of your product. You want to understand the need early, rather than again, way down the road after you've made that investment. And be able to look at that as a global picture, a whole clinical plan in light of your business case for your product. So then from there, getting into designing your clinical study. And again, this is— this could be a whole other topic in itself. But just a couple of key things that are important. One is your endpoints. And with your endpoints, I think, you know, the first thing is who's your audience for your study? If it's the FDA, you want to look for outcome measures that you've seen in the past, you know, folks have used when they're doing an IDE or an IND study, or have those conversations with the FDA early on on what, what are the outcome measures that you need, both from a sense of, of what is the outcome measure, you know, like a WOMAC score, or a KOOS score, or an IKDC score, and what is the time point?

Right.

And again, thinking about all of your audiences as well. This is other areas where I've seen before, you may be able to get FDA approval of a product with a 6-month, you know, WOMAC pain score endpoint. However, you may find out that payers aren't going to pay for your product unless you've got at least 12-month efficacy data. Gosh, you definitely want to know that up front and be prepared to collect 1-year data You know, you'd hate to get on the market with 6-month data just to find out that nobody's going to pay for it because they want you to have 1-year efficacy. Same thing for journals as well. Look and see what journals are kind of requiring. Orthopedic surgery, you know, 2 years is kind of a bare minimum. They like to see 2-year follow-up anytime you're doing any kind of a surgical procedure. So know that you want to make sure that you're doing your studies long enough for all the audiences that may potentially be looking at your data.

But Patrick, do payers tell you what they need on the front end before you do the study?

Very cryptically. It's, it's, it's, you know, it's hard to not be cynical about it, but it's, it's a bit of the Wild West. And I recommend get plugged in with folks. And we've got some folks on the Bruiter team that know the landscape well. Spend the time, get, get some experts in there that know what's going on in the payer landscape, particularly if you're in the private pay landscape. If you're looking at private payers as one of your primary ones, understand what's being covered and what's not and what they're seeing. It's, it's not as cut and dry. And I've done rounds and rounds and rounds with market access folks throughout my career. Trying to understand it. And it's a little bit of looking into a crystal ball. But there are some general principles of, you know, about how many studies do you need, about how big do they need to be, what are they currently covering. And but definitely get plugged in with some experts on that to get that answer. And again, do it before you launch into your big investment of a clinical trial to make sure you're you're going to get your money's worth out of it. Um, got it. You know, along with that, you know, do you need a compar— comparator? If you're going for FDA approval of a product, a new, new drug or biologic, you almost definitely are going to have to have some kind of a comparator. You— they're going to want you to randomize. Um, a lot of times they're going to want you to do a superiority study. Uh, there's some cases where you can get away with a non-inferiority study, understanding that up front. Is really important. And again, who's your audience? Why are you doing the study? It gets back to that why. If one of your main reasons is you want to show that you're better than your competitor, what does your competitor need to do to help you tell that story? Blinding along with that. Do you have to do a blinded trial? Is it required? Again, particularly for regulatory approval, they're going to push you to blinding your studies. But then making sure it's possible and feasible. Do you have imaging needs? And this is a big one to really think about upfront because this is a big cost driver, honestly, in some studies. X-rays, not so much, but MRIs, if you're doing CT scans, if you're going to have an independent radiologist look at the data, those tend to be some of the bigger ticket items in studies. So making sure you're exploring that ahead of time. Safety, from a safety standpoint, for sure with regulatory studies, you're going to have to demonstrate, demonstrate safety. Do you need a data safety monitoring board? Do you need a medical monitor? What, you know, those are some of the things you want to think about, especially as you're trying to flesh out the cost of your program. You don't want to have any surprise costs crop up as you're launching into it and you suddenly find out you need a whole data safety monitoring board that you didn't budget for. So things you want to think about as you go along.

Yeah, go ahead, Kyle, go ahead.

Yeah, can you go to the previous slide?

Yes.

So Patrick, outside of clinical studies where you've, you've gotten some sort of feedback from the FDA and they've said, okay, this is a 510 device, but we do want some sort of clinical study for your predicate for your substantial equivalence. You know, these other, these other categories of audience, physician, payers, you're potentially going to be working a clinical study for the sake of making it a, a sales tool to a large degree. And I'm, I'm curious if you can expand on some of the drivers behind that or, or detailed considerations for scope, complexity. How do you— because I think you see so many different companies, especially smaller startups, where they think if they just throw something out there clinical study to say that this screw that's made of this material is better than that screw. And they, they spend all this time and money to do some sort of clinical study that wasn't required. They then take this piece of information they think is gonna be meaningful to a potential customer and the customer says, well, that, I don't care. Like, what's the difference there? So how, how, how does that play in here? Because I'm, I'm very intrigued by what you brought up around the idea of The decision to look into doing a clinical study should be driven all the way back to the beginning, even if it's not something that's going to be required.

So I hope that was clear.

Right. Yeah. Let me try and— sorry, Patrick. Let me try and— let me try and—

sorry.

Patrick's exactly right, thinking about what the end in mind is. And one thing that we have found to be very useful is the performance of what we call strategic marketing, quantitative strategic marketing surveys. So we can take, we can write a survey that goes after end users and says, here's a product in development, and you give a little blurb about it, and you ask focused questions. Around what evidence, what data do they think is necessary to cause them to switch from using a competitor product to your new product? Or if they're not using a product like yours, that would compel them to start using a product like yours. And you can begin to develop and build sort of levels of evidence that naturally a 300-patient randomized double-blind controlled trial is gonna cost, I'll make it up, $20 million, or a 20-patient case series done by 2 docs at different centers is gonna cost $100,000. And you can begin to understand what you need. You can also run surveys like that with payers or with medical directors at third party, at the Blues, at Aetna, at, you know, UnitedHealthcare. And, you know, at least the way we do it, these are always anonymized for both the product and they're anonymized by the doc so that they're not feeling like, oh well, like if you have a new screw and you're going to run a survey, the problem if you run a survey with docs that you know is they're going to tell you typically, I mean, they're going to tell you what they think you want to hear. And so whether you have us do it or somebody else run a survey to answer your very question, hey, what do you need to start using the product? And maybe it's $25,000 or $35,000 or $45,000. It's, it's money that is extremely well spent.

Yeah, I'm, I'm gonna add a nuance, Scott. So, I was involved in, in surveys with physicians, I don't know, 20 years ago. And the, the problem is their ego gets involved. And so the better question is, what would your peer need to change screws? If you ask the surgeon, they will give, they will lie to you by accident, actually. But if you ask them, what would your guy down the hall or the next hospital need, you'll get the real answer. So, always ask them about their peers.

Yeah, that, that, that's a nice point.

Yeah.

And I think we try to get around that a little bit by—

it's anonymous. Yeah.

Yeah.

But, but the data we get back and you qualify docs into it, you know, have to have, you know, fellowship training in this, a certain number of cases per month. And then it becomes geographic. You can, you can evaluate the data in geographic segments. We do it across the country, but you can cut the data however you want.

Yeah.

And probably one of the biggest, I would say just kind of a caution or thing to watch out for when you, if you do decide to, to dive into a clinical study on a 510 product in particular is to pay, pay really close attention to what you're allowed to claim. Regulatory-wise with the 510 product. We all know one of the downsides of 510 product is often you're more limited in what you can claim about the product. And what you don't want to have end up happening is that you run a clinical study, get a beautiful manuscript done, and then find out that you actually can't distribute it as a company because there's claims made in the manuscript that you aren't allowed to say from a labeling perspective with the FDA. So, uh, just a little caveat that at the beginning, early on in the process, get, get some regulatory experts involved just to help kind of filter your study, filter your output, your manuscript, or your white paper, just to make sure you're going to be allowed to use it. Because I've seen that happen before as well, where, you know, the indication for the study wasn't even on label. And then it's not that it can't be out there in the public domain, but now your sales force can't use it. So something to really think about as you go about that strategy as well is to really think about what you're allowed to claim, and you can be creative with it. You know, you can be strategic with it. It doesn't mean you can't do studies because you have limited claims. It just means you have to be really, really smart about how you go about it. You need to think about what the output is going to be and where it's going to live. So just something to think about there. All right, we'll keep trucking here, but please feel free to jump in with more questions. So data points. Are an important thing to think about. On the one side, one thing to look out for is survey fatigue with patients. It can be easy to say, oh, I want to collect a WOMAC, and I want to collect an IKDC, and I want to collect a KOOS, and I want to collect a Marx Activity Rating Scale, and just see which one shows the best results. You definitely want to make sure that you keep things within what's feasible for the sites to get accomplished in a reasonable amount of time. And that avoids patients getting burnout and, and starting to just scribble down answers. So you want to avoid survey fatigue. And I would say one of the areas that I think sometimes get overlooked that I like to make sure folks think about is the baseline and intraoperative data as an area where you definitely want to pause for a minute and really think through what are the data points you need, again with that end question in mind. If you think that, that one of the advantages of your product is that, you know, it makes a rotator cuff procedure 20 minutes faster than the other guy's rotator cuff procedure with the augment that they're using, you better make sure you're collecting operative date— operative, you know, operative time on your surgical form then. I've seen things like that happen in the past where you get all the way to the end of the study and there's some important questions that the marketing folks would like to have answered. And you just didn't collect the data. So make sure you think through some of that baseline, you know, demographic data, patient characteristic data, and intraoperative data if you're doing a surgical procedure, uh, that is going to help you answer some of those questions down the road. Um, and then the other thing that I see, particularly small companies, uh, sometimes where they can get themselves in trouble a little bit is on version control. Just make sure you keep really tight version control. Of your protocols, your informed consents, all your study documents. You know, put dates and version numbers on things. You know, circulate them internally and get them signed off. Make sure everybody's on the latest version and they've got IRB approval of the correct versions. Sometimes where I've seen people run into problems where, you know, data starts falling apart or things start getting pulled out of the study is when they lost version control. Selection criteria is another one to really pay attention on. And it's, and it's a balance, you know, you want to eliminate confounding factors in, in your studies, right? Things that are going to mess up your ability to see how well your product is doing. But you can really kind of design yourself into a corner here. And I've seen it happen before where you've got a really tight, strictly run study, you know, you've got the perfect patients, outlined in your inclusion/exclusion criteria, but guess what? Those patients don't exist and you can't find them. And so you can't get the enrollment done. So it's a bit of a dance and you want to balance, you know, good solid scientific integrity in, in those eliminating those confounding factors with being able to get the study done. And the other thing to think about in that, particularly if it's a product for FDA approval, is how is that going to impact your product labeling? A lot of the criteria you set on your pivotal studies, a lot of that will translate into your product labeling as well. You know, if you, you know, say that patients with a certain fracture type are excluded from the study, guess what? That's going to show up on your product label of a contraindication, or it very well could, or, you know, it's not going to be included in the indication statement. So you have to really kind of think about your selection criteria with all those those factors. Statistics. We all know statistics are important, but I encourage you, particularly if you're going to do a large study and a big investment, get a good biostatistician to get their eyes on that protocol, to write your stats section for you, to develop your model for how you're going to analyze your primary endpoints. Critical for the— for an FDA approval study. But I would say even for a $510K approval study, get a biostatistician to really help you walk through how you're going to analyze your endpoints. It's, it's so crucial in making sure your study is designed well and that you can get the most out of it. And then lastly, and we kind of highlighted this, and, you know, the surveys that Scott talked about are a great way to do this, make sure you get some clinician input. And when I say clinician input, you know, some end users, some folks that would actually be using your product, and particularly if you can marry that with some folks that also have some clinical study experience, get them to look at the protocol. What you don't want to run into is putting things to do in the study that a clinician says, oh, we can't do that, you know, or we don't do that here. And I've seen that happen before. I've seen, again, as you're trying to make everything perfect, you know, I've seen folks say that they were going to do tests on a patient at 2 weeks that a physician says, I would never do that to that patient that early. I wouldn't do that till 6 weeks. So make sure you've got, you know, a good MD that understands the indication, understands the treatment, that understands clinical research a little bit to make sure that everything you've outlined in your study protocols are practical and that you can actually accomplish it and give you some, some input on it. Even better is if you can get the folks that are actually going to participate to spend some time on it as well.

Hey, going back to the biostatistician, so they can, they they can— you can play what-if with that person, right? And you'd say, with 20 patients, you get this much evidence or power, and with 100 patients, you get— and you can play with cost and time.

Absolutely. And I do that all the time. Obviously, the most pure thing to do is to try to say, hey, our primary endpoint's going to be superiority in WOMAC pain at 1 year. Tell me how many patients I need to do that, you know, with these assumptions. But then there's other times you come back and you say, well, I know you said I need 275 patients, but I can only afford probably about 150 patients. What can we do with that? It's, it's not the purest scientific way to do it, but practically sometimes you can't afford to do that. And then you have to figure out, you have to tweak your questions, your research questions into something you can afford. It's just the reality of the business sometimes. All right, so kind of wrapping up here. I know we're running out of time, but study execution, just a few tips, a couple things that are some things that weigh on me when I think about how to successfully execute the study. You've decided you're going to do it. You've decided you're going to make the investment. How do you make sure that you actually get your money's worth out of this? One of the big things is to ask yourself, do I need a CRO? Contract research organization. A lot of that is dependent on, again, what is your audience? If you're doing an FDA-regulated trial, you have to have a clinical quality system in place to make sure that good clinical practices are followed for the study, that all the FDA regulations are being followed, including the whole data management system and everything. And if your company doesn't have that whole clinical quality system in place themselves, you need a CRO to do that for you. If you're doing a white paper, you may be able to get away with more of a hybrid model. You know, you're doing a journal manuscript and that's it, maybe able to get away with more of a hybrid model. But some things to think about is, again, the clinical quality system. If you don't have one, you want to find a partner that's got one that you can say, we're using theirs for this study. And then also think about project management. Don't, don't, uh, don't, uh, let this go by the wayside. The last thing you want to do is, is spend a whole bunch of money to have a CRO run the study for you and then just hope that they're doing what they're supposed to be doing. Make sure that you've got folks— and I do this for some companies that are utilizing CROs for their studies, where I partner with them just to kind of keep an eye on the CRO and make sure they're doing what you expect a CRO to do. Are Are they meeting their timelines? Are they keeping the data clean like they're supposed to be? Are they bringing in qualified folks to monitor the data for you? Make sure that you've got somebody that has an internal investment that wants to see, you know, your product come to market that can help drive that CRO to do what they're supposed to do.

I'm going to second that because I have, I've had bad experiences with CROs because I mean, they cover the waterfront, they cover every city, which is great. But I, I had a guy in Holland took a train to see the nurse. The nurse was off that day and just came back and charged us $1,000. And, and it's like, well, don't you even care? I mean, no, he doesn't care. He's running 5 studies, you know, 5 different companies. So I, they don't have skin in the game. And I, I think you gotta be careful with, with that. Going in on the front end.

Yeah, absolutely. And I would say as well, particularly in the orthopedic space, you will see a lot of CROs, the bigger CROs out there have a lot of experience in pharmaceuticals and you will, you'll run into very a lot, a lot fewer monitors and clinical research associates that have experience working in orthopedics and with surgeons. And I think we all know that that's a little bit of a different experience. And for a lot of you, these physicians or you're, you know, if you're dealing with an orthopedic surgeon, they may not only be doing the study for you, but they may be a current or and/or future customer. And so having a CRO that treats the investigators in your study like customers, I think is really important. And so making sure you've got somebody that can keep close to that, can make sure that there's still good customer service going on in the midst of making sure the study runs well is really important. I've seen some really frustrated orthopedic surgeons before because a pharma CRO just came in and made them angry and told them they're doing everything wrong and, you know, and told them they didn't have enough oversight of the study. And so making sure that you've got some folks that are sensitive to having a more customer service-centric focus in their monitoring is really important. Selecting clinical sites is a big one, and this one's tough, particularly for startup companies. You know, it's an important question. Do you need KOLs in the study? You don't always. Sometimes it's nice to have one or two, you know, a champion at the podium to talk about your research. You know, evaluating private practice versus academic centers. Academic centers tend to have a whole you know, set of resources for conducting research, but they tend to be expensive. And honestly, they tend to move really slowly. Private practice, you know, sometimes you'll get some guys that are really excited about your product, but they've never done research before and they're trying to get their medical assistant to do their research for them. And you don't want somebody like that on your FDA-regulated trial because they're not going to be happy and you're not going to be happy in the end. So it's very, very important. That anytime you invest in a clinical study that you do it with folks that have research, dedicated research staff on site. I can't impress that enough. Don't let a physician try to convince you they can do a study just having their PA or their MA collect some data on the side. Clinical research is a whole extra effort. And they usually get paid to do it, right? These sites that do this get paid quite a bit of money to collect the clinical data. It's not something that can just be done on the side. And with that, you know, more and more popular are these site management organizations, which, which some of you have probably seen before. These are groups that specifically partner with oftentimes private practice physicians, and they bring in clinical research staff and they have their own SOPs and their regulatory framework. And their systems. And, you know, they will tell an orthopedic surgeon, all you need to do is show up and see your patients. We'll handle everything else. Sometimes those can be very helpful. I know some good ones. I've also had some bad experiences. But sometimes those can be ways to get a high-volume private practice physician going in a trial and contributing significantly to enrollment and yet still have that research infrastructure.

Go ahead, Patrick.

Patrick, do you, do you have an estimate or a fair assumption for how much markup SMOs should be sort of charging over your average sort of site?

That's a great question. It's on a percentage basis. Yeah, I would say on average, I probably see about 25 to 30%. I wouldn't expect them to be higher than your expensive academic centers. Okay.

Okay.

They shouldn't be more expensive than like your HSSs or your Stanfords. You know, the— if they're more expensive than that, they're charging too much money. If they're gonna ask for a lot of money, and what I tend to do anyway when I screen them is I want 'em to show me their success records.

Mm-hmm.

They need to show me, and, and usually the good ones are prepared to do this. They need to show me where, hey, the last study this site was part of, we were the top enrolling site and this is how many patients we enrolled in this period of time. That helps you make the decision on that investment. You know, if, if, if they don't have those numbers, it's a risk to pay premium price for them. But if they can show you, hey, the last study we were the top enrollers, you know, we enrolled 60 patients in 4 months, uh, you can then start doing the math and that's where it gets back to that business case again. If you go, oh, well, that'll get me to my enrollment a month sooner, what's that worth revenue-wise to me? Yeah, yeah, I'll pay a premium price for that.

Okay.

Yeah. Good.

Okay.

This is next.

Hi, guys. Great topic. Great conversations. I do want to impress— I've seen on the, on the inside how challenging it is to get staff to be compliant with the studies. I've seen literally a teaching institution staff, they stopped doing clinical trials and they wanted to start back up. And literally this, the text said that they would quit if they were going to be responsible for doing studies again. And so they had to, they had a lot of work. It was a major uphill battle. It took 6 months. I think they're still working working it out. And so if there's not infrastructure in place, I would not personally risk waiting for that site to get their stuff together. I've just seen just terrible things happen.

Yeah, thank you for that example. It's— I've seen that a lot as well. You know, you get staff that are burned out as it is already just trying to keep up with with keeping the clinic going and seeing patients. And if you try to add clinical research on top of it, particularly an FDA-regulated trial, it's not a good formula. And that's, again, where those site management organizations can sometimes be really good. The only other caution I would have with site management organizations is their goal, their business is to make money conducting research, right? And so they'll reach out to you proactively sometimes and say, hey, I see you've got a study going, you know, we've got some sites, you know, can— would you be interested in signing us up to participate? And, and they will proactively try to get you to sign them onto their studies. In those cases though, take your time to make sure that the actual physician is excited and bought into your technology. It seems silly, but it's— sometimes you end up with the opposite side. It used to be we had problems with all these, you know, physicians that were really excited and want to be part of a study, but they didn't have any research infrastructure. Sometimes with a site management organization, they're really excited about doing the research project, but the physician maybe could care less about your product. So just make sure in those cases that the physicians that they're going to sign up for the study actually understand your technology and buy into it and are excited about it enough where they're going to talk to their patients about it and do the procedures right.

Okay. Hey, Rohit's got a question.

Yeah, please. Far away.

Yeah. Thank you so much for a lot of learnings that I, that we got today. So my question is related to the antibacterial technologies that are applied to orthopedic implants, joints, hips, and knees. We have seen a lot of like literature and Maybe the company's also doing a lot of research on that, but we haven't seen any product in the market. Do you have any experience with what FDA thinks about it? Or your own perspective?

Yeah.

Yeah.

And I, I can let Scott maybe chime in a little bit as well, but I'll just say in general, my experience with it in the past in, in thinking more from actual like coatings that you would put on implants themselves, not applying at the time of care, was more of some of the limitations around the FDA essentially requiring pivotal IDE studies for every single device that the coatings are being put on. And that proves, proves to be a challenge then sometimes, um, just generically. And then in general, I'll just tell you, trauma studies, which is where most of those, the, those, uh, those studies go when it comes to antimicrobial or, you antibacterial coatings are very, very difficult to do. They're very difficult to execute. They're very messy. And it's one of the most difficult types of particularly blinded randomized controlled trials to get done. Subject retention is a nightmare. Complications are a nightmare, as you can imagine. So it's just— it's a very, very difficult clinical research study to get accomplished.

I'll just add before I have to bounce, and I think Patrick has a little time still. As far as these antibiotic studies go, again, I'll say it depends on what you want to say. Because if you want to make statements around, around reducing colonization on your implant, that's a very different outcome than saying you are treating an infection or you are preventing an infection, or you're simply reducing bacterial load. So really depends on what you want to say. And I'll close by saying, you know, these are conversations we love to have. And if you want to reach out to Patrick or I, you know, you can, you can go to our website, but you can just email scott@bruderconsulting.com or patrick@bruderconsulting.com. At bruderconsulting.com, and we'll be happy to schedule some time with you. So I want to— I'm not shutting this down. It's just that I have to bounce. I want to thank Tiger for the opportunity to have this kind of interaction. Love to do it. And I'm going to leave you in Patrick's good hands from here. Okay. Thank you, Tiger.

Thanks, Scott. Yeah.

Yeah.

Mike's got a question.

Yeah. Thanks for your presentation. I'm curious if you're familiar with some of the remote patient monitoring, remote therapeutic monitoring platforms that have become pretty popular, especially now that there's reimbursement for, for such, you know, applications. And there seems to be some, some growth in that area. It seems like it's an interesting tool for collecting data and research. Are you starting to see that with some of the clients that you're working with? And has that been a successful tool based on, on any experience you've had?

Yeah, it's increasingly so. And I would say there's a lot of really cool things being done, both whether it's devices patients wear that are tracking their motion or even just apps on their phone that ping them and say, hey, it's time to collect your pain score. All excellent tools, all really helpful. I think those things are great additional tools to help with bringing in additional data. However, I, I will tell you that from my experience, and even with some of these really cool tools, there's still no substitute for getting the patient physically in the office, you know, with the clinician, sitting them down in a chair and saying, hey, get your forms filled out. I've done a lot of automated stuff. Studies before, you know, where we either sent out emails or pushed through an app forms for, for patients to, to collect. And I'll tell you, you can get a good volume of data in the first, say, 6 weeks, 3 months, even 6 months. You'll get a decent amount of data. But as you start getting out to about a year, and especially 2 years, if you're just relying on kind of a more automated thing like that to happen, the data compliance really starts to plummet.

Yeah, I imagine the follow-up starts to drop.

Yeah, it really drops off. Yeah. And I started my clinical research career in registries, total joint registries, and I've run some registries myself, even on some biologics. And, you know, I'd have call centers where we're calling patients back up, trying to get them to fill things out. We're giving away free iPads, all kinds of incentives. But if you don't get patients back in clinic. So far, I haven't seen an automated system yet that could get more than a 40% follow-up rate once you get out to 1 year and 2 years. Um, and so again, they're nice additional tools. You know, you have a patient, you know, that moves to Florida and they're not— no longer anywhere near there. Yeah, it's great to have a tool where you can say, hey, I'm gonna send you an email, can you go into this online system and enter your data in And that's a great opportunity for another data point. But if you rely on, you know, some of these systems that kind of claim that they can just kind of automate data collection for you, when you get out long term, the data compliance just really drops off. And kind of this next slide, you really want to keep above 85% retention in your primary endpoint. And so you really have to think about that. If your primary endpoint's at 1 year and 2 years you need to plan on those, on getting those patients back into the clinic, sitting down with the physician, because those tools tend to, tend to fall apart once you get out to get out very far from the initial index treatment.

Got it. Vinay's got a question.

Patrick, have you seen use of instead of like patients coming into, into the sites, have you seen the HCPs or the research coordinators go to the patients?

Yeah, that, that's been done as well. It gets expensive, but I definitely have seen particularly site management organizations that'll volunteer to do that. That'll go on site to the patient. As far as making sure you collect primary outcomes and, um, and get kind of some collection of potential adverse events, it's a great way to do it, um, to kind of help with that. Oftentimes though, if a patient has any kind of a suspicion of an adverse event, or if you've got some specific, say, safety endpoints that require the patient to be kind of physically seen by a physician, oftentimes then you still need to bring the physician in, or you have to have like a PA or an NP or an MD make a house call. It just kind of depends on what you're trying to collect data-wise. But yes, I have seen that before. The cost goes up, but it's another good way to increase your compliance rates.

Good. We're kind of hitting the top of the hour. Um, let me, let me ask one question for Patrick. So I think most of the Bone Chatters are— there's a lot of Class II companies, and so they're probably more interested in post-market studies.

Mhm.

And so how would you— I mean, this is real high-level question, obviously, but How would you think about setting up that study and what would it cost and how long would it take? You know, things like that.

Yeah, I don't tell you, I've done a ton of post-market studies and like I said, I've, I've, I've actually designed a number of registries as well. And that's an area that I'd love to help you out in and navigate. You know, it's, it's not uncommon at all to have 510 devices and then you want to collect some post-market research to be able to show your competitive advantage. Getting back to that why is really important. Understanding your, your understanding kind of your business case for the product is really important as well and making those decisions in the overall product development lifecycle. Clinical research studies take time. They just do. It's the nature of it. You know, you want to do 2-year outcomes. Well, you've got 2 years right there, plus the startup, right? And so you really want to be thoughtful through what's feasible to get accomplished, what can you afford to do, what makes sense business-wise to even spend money on, and get that figured out first. And then, you know, so those are some of the easiest studies to then just design and get going. But I would definitely start with looking at your timeframe of, you know, when do you need data by? So then what's even feasible to get accomplished? What can I afford to spend money on? And then go from there. You know, what are your end audiences there? Is it the Surgeons for Adoption? Is it a strategic partner? Is it investors? You know, sometimes strategic partners and investors, a little bit of human feasibility data goes a really long way. And, you know, this whole presentation has been all about clinical research, But I'll tell you, I've done a lot of good stuff with preclinical data. Don't sleep on your preclinical data. Don't be afraid to pull out your large animal studies and get everything that you can out of those. Because sometimes that may be all you've got, or all that makes sense to do anything with. Interesting.

Can you— for post-market, can you combine a clinical outcome study with an economic outcome study, or are they separate?

It depends on the questions you want to answer, but if you're going to do a post-market study, I would make it an economic study too.

Okay.

I just—

I—

it's, it's the way I've always approached clinical research. If I'm going to do a clinical research study, I'm thinking of health economics when I'm designing it. I think you're foolish not to in this day and age. And, and that doesn't mean that you have to do anything super complex, but, you know, put your EQ-5Ds in there or your SF-12 that you can get an FS-60 so you can Do some cost per quality-adjusted life year. Look at what are the main cost drivers for this procedure or technologies, and what data points do you need to collect to do that? You know, what are the big things that could tell a story to a payer? Make sure you've got those endpoints in your study then from the get-go. And yeah, I wouldn't enter into any clinical study unless you are thinking about the health economic picture as well that you're going to get out in the end.

Got it. Good, good. So we're kind of hitting the— we passed the hour. Are there any other questions? I know we could probably go for another hour.

Oh yeah.

Any burning, any burning questions? All right. If that's— nobody's raising their hand, let's call it. We're probably going to do a, a second clinical discussion in the future. I can already see it. So thanks for everybody's time. It's a wrap. And I really appreciate it, Patrick. And tell Scott thanks for your time. It's been great.

Absolutely.

Absolutely.

Our pleasure. And as Scott said, we're here if you need us. We'd love to chat with you about your technologies and how we can maybe help you out, you know, or just, yeah, answer some questions for you.

Yeah, the website is in, in the chat. Bruder Consulting, or you've got their emails verbally now, so you have access. Excellent. All right, thanks, guys. It's a wrap.

Thank you all. Good seeing you again. Good seeing you too. Take care.

Yeah, take care.