BoneChat #30 - Generating post market clinical evidence - Vinay Bhal

We're live with Vinay Bihal. He is the Director of Clinical Programs at Edwards Life Sciences in Southern California, and he was a clinical regulatory leader at Smith Nephew and Zimmer Biomet. And he's been a BoneChat member for about 10 months. He's going to lead a conversation about getting clinical evidence to support or to, from, for your post-market studies. And he's going to start real high level, but bring him your specific questions. Handing the mic over to Vinay.

Thank you, Tyler. Good afternoon. Good evening, everyone. I guess late, late evening, late afternoon. Let me share my screen here. Just confirm when you can see my slide. Okay. Awesome. Okay. So everyone, so this, I focused this talk on post-commercial phase. So collecting the evidence in the post-market phase of a device. And I've also kept it limited to, with keeping in mind the audience here, mostly dealing with 510 Class II Class I devices. Maybe the Class II devices are Class III in EUMDR. So that's a separate topic itself, but I can talk about it if you want to. And I'll talk about the high-level principles, first principles for how to design the study for generating these evidence. And I also use the two words, trial, clinical trials versus a clinical study. So, because. It doesn't have to be a full-fledged clinical trial to collect evidence. There are other ways of capturing clinical data, and that's where I use the word study. So, but I can talk about it and share my specific examples that I had when I was especially working with Smith Nephew. Okay, let's see if I can advance, if the slide will advance.

Oh, okay. Yes. No, we're back. There is second slide.

Good. Second slide. Okay. Just to recollect the point that in one of the previous talks by Scott Bruder and Patrick— I forget the last name— but they talk about the strategy, clinical strategy, and One of the most critical point they mentioned was to keep the end goal in mind when you are designing the study. And that's what I like to reinforce here. And for a 510 or a Class I, Class II devices, when you're generating the evidence, I think you want to think about, since these are commercial products, so they're already on the market, what are the potential uses of the clinical data for you? So one of them is performance monitoring, which is demonstrating or continuing to show the long-term safety and effectiveness of your device. Clinical trials are sometimes very good at identifying issues with the devices or with the design or interactions with the physiological or, you know, systems. So they're a good way of providing the feedback to iterative designing and product maintenance, and then market adoption, which is making sure that the message about your product is going out to the clinical and medical community. And so that they can, they can see your product and consider them for use. And then other several uses like claims, supporting the claims on your devices, as well as getting reimbursement in different markets. And then specifically, if, since most of you, I believe, are in US, so one of the thing is even with a Class II device, a $510K product, you may need, you will most likely need a clinical study in Europe or even China sometimes. China has its own requirements on guidance for study designs and requirements for the you know, follow-up data up to a certain timeframe. So in that case, you may need a study for a product. And then if it could be for expanding the indications for your product, you may have received, you know, approval for one indication, but then as you learn and as you find that maybe there is a— surgeons are using it for, a different purpose or an off-label use. Maybe that can— if you can capture that data, it could be used for indication expansion. When planning for a new clinical trial study, where do you start? I think it's important to touch base on what is the process that you would follow to plan the study before you actually design it. And one of the most important step, or I guess the initial step, is literature review. And you would probably want to review the literature on— if your device is already on the market and you don't have the evidence, then you want to look up the data on equivalent devices or other benchmark devices that are comparable to the product or other alternative therapies. For your intended patient population. And these reviews are typically included in your clinical evaluation reports as well, especially for, you know, for Europe and other countries like UK, which have now separated itself from EU. So they have their own requirements that are defined by MHRA and So looking at those guidance from these regulatory agencies is also pretty important. So some of the other resources for reviewing the data on the devices are like standards and guidance documents. And one example I can think about is like NICE guidelines. I, My work has been primarily in the hip arthroplasty products and sports medicine and some knee. So most of my guidances are, you know, the benchmarks for clinical performance used to come from NICE guidelines or from, you know, apart from the literature. And then the other good resource for looking at your competitor devices, what kind of trial designs they're using, what kind of outcomes they're measuring is clinicaltrials.gov, which I'm pretty sure the audience here is familiar with. Along with that, there are FDA websites, especially the post-approval study website, and post-approval study would be the post-commercial study. Post-market study. And then also the MAR database, where you can look up for potential adverse events that you want to keep in mind, especially when you are designing the trial from the safety perspective. So when you are thinking about safety outcomes, you want to make sure that you are able to address any potential issues or capture those potential issues during the clinical trial, if not already, you know, from the clinical use in the field. And then other resources is label and the IFU of equivalent devices, which have the various indications. So that gives you an idea about which disease conditions you want to target. You obviously have an idea, but then there are additional areas where you can expand. And that's a good source to look at the labels and IFUs for other devices. Okay, so now before you perform a literature search, you have to be a little strategic in how you frame your clinical question for performing that literature search. And this is just a method, method, which is an acronym stands for PICO, which is, you know, you identify your patient population or the problem. So for example, you want your device treats patients with end-stage arthritis of the hip, and that becomes your target population, patient population. And I, which is intervention, so which is for a hip replacement or a resurfacing arthroplasty. And then C stands for comparator, if you have one. You may not always have a comparator device. Maybe you're just doing a single-arm study and collecting evidence only on your device, or you could be comparing it to the standard of care or a non-surgical management. And the last is the outcome, which, you know, which is what you hope to improve or affect through your device or treatment. And then, obviously, the outcome that you're looking at, for example, in this case, for a total hip replacement, you want to relieve the pain and the disability which is associated with the end-stage arthritis of the hip. So your clinical question can be framed using this method. And let's say you— and you can use this to search the databases like PubMed or Embase, and that will give you a broad collection of all the articles that, that you can use, you know, to perform your literature review. Okay, and then, so once you have done your literature review, how would you— how is the literature review used? And this really requires someone who can go through these articles, read them, you know, and identify all the potential clinical hazards, which can be due to the material of the tech, of the device, or the technology of the device, how it operates, and including the manufacturing processes. And these are some of the things that you Again, keep in mind when you're designing the study that what are the potential safety issues that you want to address. Then you identify the types of outcomes which are measured on the, your equivalent or competitor devices. And what you're trying to do is, and what is meaningful to the patients, to the doctors who treat these patients and surgeons who use these devices. And that's what will drive your primary research question. And it'll also help you identify your clinical endpoints, as well as justify the use of the endpoints that you are choosing, both primary and secondary. And then you can use this literature also to validate the criteria that is used to demonstrate equivalence of a device. If that's the route you take, for especially outside U.S. And then along with all these components, you are able to define the— your research question, your objective of this study, and the statistical hypotheses, as well as then determine the sample size. So, these are the— that's why literature review is the first step that I always recommend to start with. So, let's look at some of the study design approaches. The first one that I have here is a randomised controlled trial, which is a gold standard. But these trials, or this trial design is best for establishing the safety and efficacy because it removes a lot of bias. Or avoids the bias. And it is typically used for most of the Class 3 devices. So that's why I'm not going to go over this kind of design. So most common design that is used for 510 or, you know, lower-risk devices is an open-label single-arm study. And you typically will use a historical control. And again, literature review is where— is where helps you with it. Or you can define a performance goal, and performance goals can be defined using guidance like NICE guidelines. Or if you are in spine, then there may be analogous guidelines for the spine or for sports medicine, depending upon whichever area you are in. Or you could also work with the regulatory bodies. Even for these, even though they are not required, these trials are not for regulatory approval, but you can still reach out to the regulatory bodies and see what kind of performance goals make sense. The next is a cohort study, which is a prospective observational study. These are typically conducted via registries like National Joint Registry in the UK. American Joint Registry. There's a— similarly, you have registries in Australia, Sweden, Denmark. These are the countries which usually have big registries, joint orthopedic registries. And then you have the very common that you may be familiar with is retrospective and retroprospective study. And I'll probably share some examples later, but retroprospective study is very useful, especially when your device is used for revision procedures. And the reason is because if your device has been approved without a clinical trial and it's on the market, fairly new on the market for a couple of years, it's not easy to find a revision, patients who, you know, who will be undergoing a revision procedure. So what you do is you wait for a certain timeframe and 'til there is a substantial use of the product. And then you identify your sites or surgeons, you know, who are performing revision procedures. Again, it'll be dependent. You have to use your sales data or the— some kind of commercial sales information to identify where revisions are happening. Of, you know, once the patients have been implanted with the ER device. There you have— You can then expect to enrol patients. Once you have enrolled these patients who have been revised, you can then follow them prospectively. That's one way of accelerating your enrolment into a— For revision procedures. Or revision candidates and then follow them prospectively through the whatever timeframe you want to follow. The next, another consideration for a design is the number of trials that you use. So multicenter trials are useful because they help you evaluate your device or a treatment more efficiently because they have the, you know, you can, meet your enrolment target both in numbers as well as time frame, as per your needs. There are several centres which will have large number of patients, but there could be centres who have few patients. In that case, you just have to use large number of centres. Also, the benefit of using multi-centre or multi-sites is it provides you a basis for generalizing the findings of your clinical trials to the broader population. And that's what's something, especially the regulators are looking for when you are getting them, you know, when you're trying to get your devices approved, they will ask for why does your clinical study or clinical trial population, you know, address the patient population in this region? And so some of those justifications have to be provided. And it's— that's where a multi-site data collection is beneficial. And you can also extend that to global, like you can have multi-site obviously in US, outside US, and that's even better for that diversification of your patient population. So let's look at some of the study design considerations. Obviously, when you start, you have to be very careful with your planning. One thing I have found very useful is to consult the key opinion leaders. So it's very important that you have some physician leaders identified who are using your product. Who have experience with clinical research and who are maybe well-known in their field. Consulting them at the design stage is very critical because you can gather some important information. For example, what is the standard of care? What are the typical follow-up schedules used by, you know, for these patients? What are the outcomes that they typically measure in their standard of care? What this allows you to do is you can tailor your study to match these standard-of-care follow-up timeframes. Even, you know, and if you have to deviate, you can deviate a little bit, but then it makes the data collection easier for the sites because then they're just able to— they're not deviating from their regular day-to-day work. And then you have to consider what you'll be using. As I said, if you are using it for, EU MDR requirements, then you have to think about is your device a single device or does it have multiple components? Because if it has multiple components, then EU MDR requires you to get evidence on each combination of the device components. And that makes it very difficult and challenging, especially when you're—

So Vinay, that's like a bone plate and a screw. You have to— Is that what you mean? Pieces or SI fusion device and a screw or?

Yes. So a bone plate and a screw, you know, because it will always be used with a screw so that goes together. You can always justify that when you're collecting the evidence on the bone plate, your screw is already covered with it, right?

Got it. Got it. But it's something that where there's an optional fixation.

Got it. Yes, yes. So for example, let's say, so when I was talking about components, so let me give an example from hip because that, or knee, that's more, I'm more familiar with them. You have the cup and the head and the femoral stem. And if you have a different, you know, one variant of a head and one variant of the cup and another variant of the, and you have multiple variants of the stem, you have to get evidence on all those combinations. You may have a, you may have a metallic head, or you may have a, a ceramic head. And if you're using it with your cup, then you need to get evidence for both combinations.

No wonder, no wonder everybody's skipping Europe.

That's crazy. Yes. Uh-huh. Yeah. And, and at least that was my experience, and I know they are working towards making these requirements less stringent, but it is an issue. And that's very important when you're determining your sample size. You have your, you know, let me go to the endpoints. You have your primary endpoint, which is driven by your primary research question., but then you have these combination devices or the combination devices that you want to get your evidence on. So you can, you can sample size the study based upon your primary endpoint, but then it may not be sufficient to meet your sample size requirement for the combinations that you are trying to address. So that's where you have to then kind of assess the usage and the volume of those devices in different regions. And then based upon that, you try to justify what would be a statistically meaningful number, a minimum number for that combination. And that's beyond your primary endpoint, meeting the primary endpoint requirement. Okay. And then another is, obviously, if you are trying to use it for reimbursement, then again, talk to the reimbursement agencies, I mean, if Medicare, Medicaid, or it's in Australia, I believe it's TGA who does it because I remember for one of the product and you all may be familiar, it was called Redapt revision system. TGA wanted, like they specified how much clinical evidence we need to collect to support the reimbursement in Australia. I believe, if I remember, they had asked for data on— Because it was a revision product, again, the revision, the sample size is not— That patient population is not easy to enrol. They had asked for 2-year follow-up on 50 patients. How we went about getting that data, we did not do a clinical trial. That's where we used the approach which we call data capture. We, again, based upon our sales, we reached out to the high-volume users of that product and reached out to multiple of them. And we kind of vetted them. What are the follow-up? How long do they follow up their patients in their standard of care? And do they collect any PROMs? Do they collect any outcomes measures. Do they collect any, do they make any radiographic analyses and do they record any of that data? And based upon that, we defined our endpoints and reached out to these multiple surgeons directly and worked with them to collect their EMR records. So they were able to provide us prompts and The limitation to this kind of study is one surgeon may be using, let's say, Harris Hip Score, other may be using HOOS or HOOS-jr. But nevertheless, you know, it was still sufficient as long as you have one score or the other, one functional score or the other for the patient. And you hope that you get sufficient numbers to be able to at least perform some basic statistics, like the descriptive statistics that you can perform on it. That's one of the method that we were very successful with, and that was accepted by the TGA. Once we had the data from the surgeons, we prepared— a document called Clinical Activity Report, which summarize, basically summarize the data just like you would do in a manuscript, but it's a small report. And you describe your methods, you describe your criteria for, you know, you, for selecting the, selecting the, the sites. And the good thing is that because this is real-world data, you don't have to like enroll any patients. You don't have to get any consents from these patients. This is already being collected as part of standard of care by these surgeons. Okay. Then moving on to these other considerations, you have to anticipate and address, as I said, some issues in the trial conduct and analyses, like how the surgeons will use the product. Now, for example, you may have a product which is used for both primary and revision procedures, but are the surgeons using it mainly for primary or revision? So that will decide what is feasible for you to enroll in your trial and what kind of data you can collect. Is there any off-label use? If there is an off-label use, then that's an opportunity for you to expand your indication.

Hey Vinay, are we on the next slide?

I think we are on the slide number—

Postmarket study design.

Postmarket study design considerations.

Okay, okay, got it, got it.

Yep. Okay, and then again, I just— via examples I've shown here what, you know, what kind of endpoints you would typically identify for an orthopedic product. And I have taken hip here again as an example. So, for example, a primary hip or a revision hip arthroplasty, you look at typically your primary endpoint is a 10-year survivorship. And you're looking at success or failure rates, which are, let's say, at 10 years, a 95% success rate or a 5% revision rate. And then again, same thing for revision. You look up into the literature. Once you have a revision, what is the re-revision rate for that device. And that will, using that, you'll be able to define your success criteria or the repair rates or failure rates and success rates and so on. And then also again, you look at the survivorship of individual components and then typical functional outcomes like Harris Hip Score, HOOS, KOOS for knee, WOMAC, or even for ACL repair procedures, And the radiographic assessments is another important outcome. You look for implant positions, any migration that has taken place, ossification or radiolucencies. And then safety assessment where you're collecting the adverse events associated with the device. So that's how you cover your safety and effectiveness data points. And then based— once you have defined your question, you have identified the endpoints, you determine your target population and the sample size. And then you, while you are defining these requirements, it's also good to identify when you would perform the interim analysis of the data. And those would again be guided by your, by the purpose. What is the end goal? Is it a regulatory approval in an outside market? Then by what time would you be able to enroll your patients? By what time you will have a 1-year or 2-year follow-up, which is the primary endpoint.. And then how— by when can you prepare a clinical report for submission to the regulatory body?

Okay.

Moving on to the next. So, here again, for the clinical endpoints, it's important to look at, you know, your primary and secondary endpoints. But, Your clinical trial should not, you know, should try to answer at least one definite question, and which is driven by your primary endpoint. The selection of the endpoint is based upon which— what is the most sensitive, which outcome is most sensitive to the device or the intervention, and what is more important to the patients. For example, again, in a hip or a knee replacement, it's the reduction in the pain or reduction in the amount, you know, dislocation, increase in range of motions, no adverse reactions to the metal ions, or a failure-free survival. And you can use any of those as a primary endpoint. It depends again, what is more important to you and to the patient community. Then the scale of the measurement, again, there are continuous variables, there are categorical variables, and there are some variables which are time-to-event, like time to death or time to recovery or time to return to work. These are, again, some of the endpoints you can consider as secondary analysis. But they do impact the power of the trial and therefore the cost of the trial. And then again, you want to consider what kind of comparisons you want to perform. And I'm sure you're familiar with the non-inferiority and superiority comparisons. Typically, you go with non-inferiority unless you have a device that you are sure that, you know, will perform better than what's available on the market. But you can have superiority as a secondary If you demonstrate non-inferiority, you can still go and try to show superiority if your clinical data does support that.

Hey Vinay, let me pause here and let's open it up for questions. I mean, most of the members are, they have startups, they don't have a lot of budget for clinicals. They've already got 5TK clear device. And they're trying to get some clinical evidence to help them sell or maybe get through VAC committees or even economic data. So let's kind of go to the street-smart way of doing clinical studies. And for— Mark's got a comment. Jump in, Mark.

Yeah, Vinay, I rarely get an opportunity to talk to somebody with your background expertise. Someone with a doctorate. I have a couple of questions. They're kind of on the practical side. How do you look at a clinical study and quickly judge its validity? And is a technique to judge the validity or criteria of a clinical study taught in medical school or residency? Or is it— is that learning mostly experiential? That's my first question. I have another question too.

Okay, while you try to recall, I can attempt the first question. So, you know, I'm sure there are clinical design courses. But I have learned it through experience. And the way to— again, you have to look at the literature review, what has been the common outcomes that have been measured, what has been the common follow-up timeframes.? And then what have they— have they really been successful that the designs that have, that are existing, have they been successful in demonstrating your primary endpoint effectiveness or your device effectiveness and safety over the, you know, over the follow-up period? And that gives you an idea whether a study design is would be sufficient or not, or would be adequate or not. I think to start with, you have to look at— you start with the synopsis even before, you know, you can start with a synopsis. You're not writing a full protocol, but you are defining the purpose of your study. What is the, again, it is, you know, again, to start with, if you are already on the market, your sale or, you know, of the product will guide you where it's being used, how much it's being used, and what is the nature of the usage apart from, you know, if it's different from the indications that your device is intended for. Based upon this information, the clinical trial design is then straightforward in terms of your primary research question. What is important is how do you demonstrate the— demonstrate, you know, how do you answer your research question, and that's through the statistical analysis. Okay.

Can you— is there any way to— is there any way to pick up a clinical study and quickly say, "This is crap"?

I mean, That would have to be a really bad study. I mean, I haven't, it's, I mean, it's, I mean, I haven't seen many bad, I mean, I haven't really seen many bad studies, to be honest.

Okay.

I've seen a lot of bad studies that are because they, it's based on patient selection. So if you get super users that just do great surgery all day long, doesn't affect how it's used in other parts of the country, different patient populations. It's totally biased. And that's what you see a lot. You see the super users in the teaching hospitals with great results. And I think that's a flawed study.

So to that point, I think it's your site selection, which is key here. Yeah. So it's, I mean, site selection is more of a, I guess, an operational aspect, not really the study design.

It's like picking a juror, you know, if you can pick the jury, it tells you what the outcome is going to be.

Yeah.

Right.

But in terms of bad design, I think it would be not being careful in designing. Let's say you are— most of the studies are non-inferiority, so I'll take a non-inferiority as an example. If you have a very wide non-inferiority margin, then that's an indicator that that's not a good design. Because one, it will increase your sample size. You will be enrolling for a longer time. And therefore, it will be taking— you will confound with more patients. You will confound your results. You could possibly confound. So those are the things that you can look at. Like what's the rates that they're using, or like, for example, what's the criteria for success of the study? Again, if someone is going for superiority for a new product, that's an indicator for like, you know, that's not a good strategy.

Okay, I'm going to save my other question and let— pass the gauntlet to Matthias.

Okay, yeah, yeah, Dr. Schürkopf is on next.

Thanks, Tiger Buford. Matthias, sorry, my camera is not working today, so I have to just talk to you. I wanted to make a couple of comments in terms of value. You can come from an academic perspective, and typically every study is is defined by the level of evidence. So, a randomized trial has the highest level, and it goes all the way up to expert opinion. So, that in terms structures kind of the value of these studies a little bit, and payers typically hone in right on the Level 1 studies and say, well, if you don't have Level 1, if you don't have randomized trial data, it's still not proven, and therefore we are not paying for it. So, there is a little bit that trend coming from the payer side.. But in my experience, you know, in really looking at the value of a study, it's a question of what the study answers. And sometimes that answer is very different. If you speak to surgeons, they're looking for a certain type of data. If you talk to patients, they have other interests. If you look for hospitals, you know, same thing there. Everybody, you know, has a different priority in terms of looking at the evidence. So I think overall it's really important to have a good mix of studies, a good mix of data that ultimately create a very comprehensive pool and answers and addresses a lot of the concerns from different targets.

I absolutely agree with that. And one caveat there is if you have, if you are trying to address, let's say, regulatory approval or reimbursement or health economics, then I think you do want to include as many of these endpoints in the study if it's possible. But also, the clinical trial should still be focused on answering your primary question. The others can be secondary or exploratory analyses. It's good to include as many of those outcomes within your clinical trial. And to come back to earlier question from Mark, having too many questions can actually be an indicator of a bad study because then you can confound, you can confound the results. So it's important that the research question is really well-defined. Focused. And sometimes it's beneficial to do a separate study if it's not feasible to do a separate study for health economics just for those reasons.

Okay, Patrick.

Vinay, a lot of times these— if we're talking about Class II products or postmarket studies for Class II— of course the doorbell rings. If we're talking about studies for Class II products or products that are already on the market, you know, they're saleable already. There's maybe some level of reimbursement, even if it's not ideal. But when you, as a manufacturer, as industry, when you try to get these, these studies started, oftentimes there comes requests for like grant money from the institution, from the facility, from whomever. Do you have any tips for the group on how you should think about requests for grant money? Not necessarily, it's impossible to peg a number to a study, that's all hypothetical, but things to look out for, or, you know, things that can make costs sort of end up going kind of in a runaway direction that you maybe wouldn't want to commit to? Or, you know, can you just talk about the sort of, you know, the money part of all of this?

Yeah. So, and I believe you're referring to the, like, investigator-initiated proposals. Is that what—

Yeah, but a lot of times those investigator-initiated ones are like, you know, it's the— the companies are behind them all a lot of times, but they're, you know, they're You know, even if the company doesn't show up on the, you know, as a list of authors.

Correct, correct. Yeah, and yeah, you have to make sure that the research proposal aligns with your business needs, you know, the need for the evidence on your data. In terms of academic institutions, you know, one thing that really increases the cost up is not just the direct cost, but the indirect, the markup. Usually, they have 30 to 40% markup or indirect cost associated with those trials, which you will not have if it's sponsored necessarily, if it's not academic institution. But if it's private sites or private clinics who have the ability to perform the research, then those are the ones that really help to keep those costs down. Apart from that, as far as the study is concerned, the studies are designed in a similar way. You have to make sure that the— the purpose of the study, you know, what are they designing it for? Are they comparing it with other competitors? That's where I think as a new device player, you have to be careful. If it's, as long as it's evaluating, assessing your device and helping generate data on your device, I think it's good. It's a good way. To generate data. Did I answer your question?

It was just that, yeah, I'm most curious on just how to— what things to look out for when requests for, you know, even if you get a study going, you've got a, you know, a surgeon who's a customer, they're interested, you know, the institution is on board, but the people who sort of sometimes make the grant you know, that you sort of work up all this enthusiasm oftentimes with people like surgeons, and then they sort of pass the baton over to the sort of business end of the, you know, the research department. And sometimes their requests can— they can be really, really expensive. And, you know, all of a sudden you go back to the doctor and say, you know, and they're like, well, hey, listen, this is the research department.

It's sort of—

it's not my end of things. So if anybody has any tips on sort of, you know, how to avoid getting taken advantage of or, you know, how to keep the the cost from becoming this sort of runaway expense? That's the kind of things I was just interested in.

I think Matthias has got the answer here. He just raised his hand.

Yeah, I have had plenty of those scenarios, and you're getting super excited, have these conversations with top surgeons, and it's absolutely wonderful. And then the protocol gets handed off into the research administration department.

Yeah.

And the budget just explodes. And I said, wait a minute, this was considered to be a collaboration initially. And said, wow, this is what we are charging. And you get to a, you know, very tricky, into a very tricky situation. So I think in terms of navigating these conversations, the key is to early on look for caveats. You know, if they're proposing a randomized trial, you know it's going to take forever. You know that you're paying for the controls. And it just becomes incredibly expensive. So oftentimes, you know, you have to kind of make the decision, do I really need to be with this top institution, or can I work with a business surgeon who can be supported and can generate maybe not the same type of data, but at least, you know, find answers and evidence that support, you know, your sales activities accordingly. And I think that's really important because if you come you know, from a regulatory perspective, 510, you prove, oh, we are so similar, absolutely, it's the same. And then you turn around, get on the market, and you need to make a switcheroo and say, oh, we are so much better. So that's— it's really a question of, you know, where is your competitiveness, and, and what do you need in order to support that competitive question? And then really, you know, start working with the, with the individuals. And sometimes it's not the KOL, sometimes it's just you know, maybe a younger surgeon who really wants to get published and present and work with them. Or even I have worked with PhDs, right? They developed the whole PhD program out of a device, and that works really well too, and it doesn't cost that much, right?

Yeah.

Okay, great. Yeah, and if it's possible, then I think it's helpful to engage someone with a background in contract negotiations.

Yeah, CROs will have—

yeah.

Yes. They typically, you know, can help with those kinds of negotiations, especially when it goes to the research department. I mean, I have worked only with these big companies, so there is always an in-house, you know, legal contract person who takes over those negotiations. Once we have, you know, have a discussion with the surgeon researcher, And after those basic initial agreements, the contracts guy take over negotiating with the administration at the other end.

Yeah. My experience with the study of active implants was the— I didn't understand this, but the more data you gather, the more patients, the more institutions, the more endpoints, the cost just snowballs. It's, it become, it's exponential. And so the best way to keep your cost down is just to keep it really simple.

Mm-hmm.

You know, pick, pick one endpoint, pick 3 institutions. I mean, just keep it really simple.

And know your target audience too. Like, is this study to get an inch past the FDA? Is this study to prove, make a point to the scientific community? Is this study designed for reimbursement or whatever the cost. And it's very unlikely, certainly in orthopedics at least, you're going to sort of accomplish all of that in one, like, you know, there's not just one paper you can wave in front of everybody, all those different constituents, and they're not all going to be satisfied.

Yeah, true, true. And if you are going for regulatory, it's always beneficial, you know, always talk to the regulatory before design, finalizing the design of this study. But since we are talking about the post-market phase, I mean, it's all up to you. If you are trying to support a reimbursement or a claim on your device, I think you can design the study. And when working with investigator-initiated proposals, you have the, you have the leeway, you know, to actually design the study with them. You don't have to accept the proposal just the way they propose because it's your product. And you can control the size of the trial, the timeframe of the trial. And again, as you said, it's the research question which drives everything, the size, the time, the cost.

Yeah, yeah. Vinay and I were talking earlier and he made a really good point. You just start with the end. You start with the very end. You know what? You have the data in your hand. What does it say? Okay, that's great. And then you just work backwards and you say, well, how many patients will we need to get that data? And what will we compare it to? And what would the follow-ups be? And then, and you backtrack until you have it. Study design that gives you the answer that you want in the future.

Yeah. And keeping the size of the trial, it's— I think, again, that the key point is your definition of the statistical hypothesis. That's the key point. You can have your clinical hypothesis, that is a research question, but then how are you driving your sample size? What is the guideline that you're using? What is the benchmark? That you're using for that primary endpoint or the primary outcome. I think that's— it's very important that you do not constrain yourself. Like, for example, the NICE guideline says that they recommend a THA or a total knee system which has less than or equal to 5% revision rate at 10 years. Now, that's That's a very stringent requirement. So for a new product, you would not want to target that success rate. So now how do you justify? You can justify based upon the existing literature, you know, what are the available rates in literature? And there you will see a variation that at 10 years you have good implants surviving, having a revision rate of 7% or 10%. And that's where you go. Go with that will reduce your sample size and it'll be easier to meet the endpoint. So those are the ways you can control the size of your study.

Yeah, I don't think anybody here is going to look at 10-year studies. I mean, you basically, you want, you really want to show your device is working better than the competition at 6 months or 12 months. Yeah. It's more, more cost effective. Shorter hospital stay and all that stuff.

Yeah.

And it, yeah, if you are not going for a, if your device lifetime, again, oh, I think I missed this point. The device lifetime is an important criteria. If your device lifetime is not 10 years, if it's 2 years or, you know, 6 months or 1 year, then that's what you are aiming for as for the follow-up.

Period. Yeah, interesting. So here, let me ask the question from David K. because he, uh, he slacked in a question he couldn't attend. Um, regarding the identification of KOLs and KOL institutions, is there a way of vetting the ability to actually do research from the KOL?

Yeah, no, that's, that's, that's very critical. I mean, you have to perform, you have to visit the sites and you have to meet the KOLs. You have to look at what research, how much research they have performed in the past, how many studies they are currently doing, what kind of research staff do they have. Do they have a medical assistant helping them or do they have an actual clinical research coordinator? I think those are the things that you that you evaluate before you decide to engage a site. You do go actually and visit the facility, meet with the people, meet with— get their— collect their CVs. You also perform other checks which are mandated by the clinical trial requirements like any financial disclosures or, you know, so all those things you have to vet before selecting a site. So, in short, yes, there is a way to qualify. You make sure that they have— if you have, and again, it depends upon the outcomes. If you have blood biomarkers involved in your— as one of your outcomes, then you want to make sure that they have the way to ship the samples to a lab, or they have a way to analyse the blood sample. Usually, when there is a biomarker or blood sample, things are involved. You have a central lab that you would like to use to make sure there is no bias, there's no variance, and the tests are performed in a consistent manner. These labs are also certified. So I think the largest, you have to make sure that the sites can maintain and can manage the logistics associated with these kinds of tests.

Yeah, that's good criteria. I think we've all met surgeons that they want to do a study for the prestige or whatever, 'cause their buddies are doing a study. But they don't have the infrastructure to really pull it off. Correct.

Yeah. Yeah. And then you also have to make sure that your, who is your, you know, it's good to have the physician leader within the, within the, all your sites. If you identify a physician leader or a site or two, you know, who will act as the leader, then that, you know, they are, they're also your source of confirming, like, you know, which sites are good. Apart from the fact where you are selling your device, right? Where, where are your high-volume users? So that's, that's the first step to identify the sites and then looking at their infrastructure and their training, their, their experience with the past research.

Good stuff. Patrick, are you guys gonna do some post-market work? Are you already doing it?

Uh, we're, yeah, we're looking in, we're looking into it. It's a matter of, there's a lot of factors involved, but we're definitely looking into it.

Yeah. And it's for, it's for selling purposes or, you know, economic analysis or clinical?

No, I wouldn't say as much, I wouldn't say as much economic analysis. I would say sort of, um, one is to take 510 cleared products and and be able to put them in Europe over time. Another is, you know, especially in arthroplasty-type devices where things are measured over, like, you know, super long periods. Yeah, it's, you know, there is this— there feels like there's this need to get some sort of, when I say bare-bones outcome study, you know, just something that starts the clock almost where it doesn't have it doesn't have to be, you know, 10,000 patients and 100 sites and so on, but something with a couple of reputable centers, you know, by couple, you know, 1 or 2 that where you could just, you know, Vinay's nodding his head, you know, he knows what I mean when I say just something that gets just sort of gets the clock going because it matters in the long horizon for arthroplasty devices.

You can just do survivorship, right?

You know, yeah, yeah, that's what I mean when I say bare bones. I mean just like, like, you know, you know, one or two simple data points. Is it in, and like, you know, a, you know, range of motion, you know, something, something pretty simple.

Yeah.

Okay, Matthias is back.

Yeah, I have one comment on, on the site selection there. A lot more institutions are now switching over to collect routine outcomes. We're doing this here in Miami as well. just to demonstrate quality of care. I think this, you know, a big drive coming from CMS, but also the private payers are requesting outcomes. So a lot more people, you know, have these increasingly large databases available, and that creates opportunity for industry. So when you, you know, start talking to your surgeons and you find those that do have these databases, that could be a very interesting licensing collaboration effort to get to data much faster than you would if you start a study today. Then you, you know, competing with patients at a lot of institutions because they're running multiple trials, then you have to follow these patients. But if there's something that you can already leverage, maybe as a first-generation device and you have a second-generation device, and start, you know, extracting, collaborating, and use that data in, at these institutions could be really helpful.

Yeah, that's absolutely, completely agree with that. And that's, that's, that's something that we had found very useful and efficient way of collecting the data when we were trying to fill gaps for EUMDR requirements. We worked with For example, London Health Sciences Centre in Canada is one such institution which collects a lot of data. And so we were just able to work with them and basically asked them, gave them the set of outcomes that we want to collect at different time, you know, follow-up times. And they were able to provide us anywhere up to 2 years to 5 years of follow-up data.

Wow.

So how— explain that, Matthijs and Vinay. How would a startup get access to that data without, you know, 5 years of surgery?

Yeah. So, so, and well, this is on, on obviously on the products which have been on the market for a long time. Right.

And, and benchmark. Yeah.

Yes, a benchmark. Or, or like if you are, if you are a new product, then you will have to wait for a certain timeframe for that data to accumulate, for that usage to increase. And if one of the places where your data is being used does, I'm sure many, many, as Matthijs indicated, many people are, many surgeons are collecting these outcomes internally within their EMR system or maybe other systems internal that they have housed.. But those are the places that you work with. And you just negotiate a contract that you will collect these— this is again, just like any, like a study outcome. This is the primary outcome that you're looking for. This is the secondary outcome, which could be a HOOS or, you know, a PROM or a pain score, you know, something like those. X-ray assessments is something that they maintain. So you can ask for those X-ray assessment outcomes. And you can define the, you can define the data that you need and you can define the follow-up times at which you need those data. And sometimes even if they have, let's say, patients who they saw who have they have seen for anywhere from 3 months to 6 months, you can then ask them to continue following these patients out to 2 years. You already have a pool, already a pool available of these patients, and it becomes a retrospective, a retro-prospective study. Many of these surgeons are happy to perform this kind of—

Interesting.

Data collection analysis. You just have to negotiate a contract with them. I mean, I was pretty successful in working directly with some of the private clinics as well as some of the institutions. Yes, when we, you know, the same thing did happen when we are working with one surgeon, it's a lot more simpler and there are no other costs associated apart from their time and effort to, you know, enter the data in a, maybe in an Excel sheet or in a database, but you could use an Excel sheet actually to collect the data. What we did was we built a template, just making the data entry easier for them and send that data template to them. They fill it, they lock it and send it back to us. And we then provided to send it to our stats department and, you know, they perform their data analysis. So That's, that's, it's, it's all about, I think, reaching out to, to your users and then exploring these opportunities with them.

Interesting. Yeah, Matthias is back.

Yeah, just one more comment on the value of those existing registries and databases. I think the, the big point that they make is that you, they can serve as your control arm, and oftentimes, you know, costs go up and it's difficult to find control patient populations. But when you have an existing data pool, that makes the study design a lot more straightforward. And you can, you know, take these registries and then compare your results to those results. That creates a very compelling study design. And it comes at a fraction of the cost than, you know, running your control patients also out to, let's call it, 2 years in most studies.

Yeah, stuff. All right, I, I hate to stop it now, but we've gone over time. This is a fascinating topic. It's black magic for most, most of us, so it's, uh, but it's so important nowadays. Thanks everybody. We'll, we'll, uh, we'll do a wrap and see you next week with Anthony.

Thanks everyone.

Thanks, Tiger. Thanks everyone.

Thank you. Yeah, good Good stuff.

Thanks a lot, Tiger.

Thank— yep.