Scientist Stories: Krista Fischer, Improving Polygenic Risk Scores

Good afternoon. First of all, I would like to thank the organizers for the invitation to participate in this session on polygenic risk scores. I would like to discuss some aspects of how to improve the polygenic score, but in addition, I would also like to talk about some challenges in estimating the actual effect size of the score and its impact on the risk stratification process. We have been working on this topic with a team of the Estonian Genome Center, especially with my former and current PhD students, Kristi Leil and Merli Mendul. List of 5 points. Is the polygenic score developed properly? Is the optimal score chosen out of the alternatives? Do we have an unbiased estimate of the true effect size of the score? What does the polygenic score add to the best known non-genetic risk prediction model, and how much does it improve the actual risk stratification? There are several aspects of the first question on whether the score has been developed properly. I would just like to highlight one of them. The need for independent cohorts at each stage of the score development. Discovery, GWAS study, polygenic score development that also involves the comparison of the alternatives and validation of the score. Are we absolutely sure that there is not even a minor overlap of the cohorts used in these stages? Example, back in 2015, we computed a polygenic score for coronary artery disease. We were surprised to see that the effect size of the score on incident myocardial infarction in men was so extremely strong, despite of the relatively small sample size and a very small number of cases. However, we did not realize that there was actually a minor sample overlap with the discovery study. Not even the incident cases, but some prevalent cases and controls of the same sample were used. By computing a new score based on the same discovery study in 2018, but now making sure that there is absolutely no overlap, we still saw an effect, but with a much smaller magnitude. So it was good that the initial effect estimates did not get published or used anywhere in practice. However, I think it's quite likely that a similar mistake is possibly made in several published studies. The question on the choice of the optimal score has 4 components: choice of the discovery study, choice of the genetic variants to include in the score, and choice of the methodology to compute the score, 4. And finally, comparison of alternatives. Here you see the estimated effect sizes of 7 different scores for breast cancer. The first 4 scores are based on 4 different discovery studies. 2 studies that each had published the effect sizes of less than 100 SNPs, for 70 and 75 respectively, were available in our cohort, and 2 large studies with complete summary statistics published. UK Biobank, and the Breast Cancer Association Consortium study. Although the effect sizes are all strong and significant, they are somewhat different, and actually the 4 scores did not have a strong correlation with each other. So we used the idea of the meta-GRS, first proposed by Mike Inouye, to compute 3 different combinations of the scores, so-called meta-scores. Out of them, the combination of the 2 scores, the second and the fourth one of the initial list, appeared to be the best one. Here is another example with a phenotype now being type 2 diabetes. First, we computed a conventional polygenic score by clumping the initial SNP list using a p-value threshold to select the top ones and using the regression coefficients from the discovery study as weights of the SNPs in the score. As an alternative, we proposed adding an extra weight for each SNP basically shrinking down the effect sizes estimated with more uncertainty, to correct for so-called winner's curse bias. We saw that this doubly weighted score is clearly better than the conventional one for type 2 diabetes, and by now this finding has been replicated in several cohorts. However, the same idea has not been superior to other alternatives for some other phenotypes. Depending on the trait Either the conventional clumping thresholding idea was the best, or a Bayesian approach such as LDpred, or we have more recently seen that the machine learning ideas may lead to the best score. To summarize, there is no one-size-fits-all methodology that gives the best score for each trait, and comparison of alternatives for each single case is unavoidable.

Now the next question: do we know what the actual effect size of the polygenic risk score is? So, how do we estimate the effect size of a polygenic score on the risk of incident disease? I think nowadays most people use for that purpose the proportional hazards model developed by Sir David Cox from Oxford, UK. Without going into too many details of this methodology, I will just highlight the partial likelihood of this model that has a component for each event that we observed in the dataset, and in each component, the denominator consists of contributions of the individuals who were at risk at this given point. So imagine if you get the disease diagnosis today, Now the question is, who are the people who are at risk of the same disease today? This question, as it appears, is not so simple as it may look, because it depends on the timescale. Conventionally, in clinical studies, people use study time as timescale. So for an individual getting the diagnosis in the 15th month of being in the study, everyone else who has been in the study for at least 15 months before the diagnosis is in the risk set. As an alternative, one could use age as a timescale. So it means that if someone gets a diagnosis while being 71 years old, everyone who has been under follow-up and still without the diagnosis at age 71 is in the risk set. Making different choices for the timescale results in different estimates of the effect size of the polygenic risk score. We have shown that for biobank data, using age as a timescale would lead to unbiased estimates, whereas other options, regardless of whether one would adjust for age or not, would lead to biased estimates. The reason is that the biobank data is left truncated. One is able to join the biobank at given age only if you survive to this age, and not accounting for this fact would lead to biases. Another challenge is coming from the phenomenon called competing risks. Suppose you want to tell a woman at the top quartile of the polygenic risk score what is her breast cancer risk by the age of 80. By estimating the polygenic risk score effect that in the data, you censor the women who died before getting breast cancer, as they have left the cohort before the event of interest. Seems a logical thing to do. So you get the risk estimate of 13%.. But is that the actual risk level for this woman? By doing the proper competing risk analysis, you would actually see that the woman has a probability of 32% to die before the age of 80 and probability of 56% of being alive without breast cancer at the age of 80. So there is an 11.5% probability of getting the breast cancer before the age of 80 for this woman., whereas actually half of the 11.5% would die before the age of 80 after getting the diagnosis. As you see, this is a complex multi-stage modeling problem to get the estimates right. So not so easy to estimate, but also very difficult to communicate.

Let us now move to the question on the practical gain from implementation of the polygenic score. This is an example of a recent paper that has created a lot of discussion. Based on the UK Biobank data, the authors assessed the effect of the polygenic score on risk stratification when added to the so-called pooled cohort equations risk stratification algorithm for acute coronary events. The table from this paper shows that looking at cases and non-cases in different initial risk categories, that the polygenic score improved the classification in only 2 of the 4 categories. So the authors concluded that the gain by polygenic score was really modest. But let's have another look at the same data. We start with more than 350,000 individuals. The standard pooled cohort equations algorithm would stratify about 17,000 of them to high-risk category, with estimated risk level above 7.5%. However, we see that in fact only about 1,100, so 6.3% of them became cases, so had an acute coronary event within 10 years. We also see that more than 5,000 of the actual cases were initially classified to the no high-risk category. When we add polygenic score to the algorithm, some individuals will be reclassified. Namely, about 5,500 individuals, including 250 actual cases, were removed from the high-risk category. But at the same time, also 7,200 individuals were added to the high-risk category, including more than 520 cases of the disease. So, the new high-risk category has now become a bit larger, but also the percentage of real cases in this category has increased from 6.3% to 7.2%. The absolute number of correctly classified cases increased by about 300. Do you really agree with the authors that the improvement is so modest? If we do this exercise for each of the single components of the PCE predictor, would we actually see larger effects? Do we actually expect the polygenic risk score to do miracles by itself? Or would it be just good if we could add an important component to the risk prediction algorithm? To conclude, there are different ways to improve the polygenic scores, with a range of methodologies being available. So far, we have seen that there is no universal best solution, so one would always need to consider the alternatives and compare them. Besides improving the score itself, one should also make sure that the effect of the score is estimated in an unbiased way, avoiding sample overlaps and accounting for left truncation and competing risks. When we move to the actual risk stratification, there is also an important question on the clinically meaningful effect size. It is easy to rush to either too optimistic or to too skeptical conclusions when we haven't actually defined our expectations clearly enough. Thank you for your attention.

Thank you very much, Dr. Fisher, for this really important talk showing us how important it is to optimize methodologies and look and be careful with all kinds of challenges that can be in the analysis. We now move to a presentation by Professor Mike Inouye, and I again want to invite everybody to already put questions on the chat, but we'll answer them at the end when we go to a joint debate. Professor Mike Inouye has a very international CV, growing up in the Seattle area, studied in LA, worked in Melbourne, and is now in the UK. In 2017, Dr. Inouye was recruited to the Baker Institute and the University of Cambridge to set up a lab spanning Australia and the UK, focusing on core areas of system genomics, including polygenic risk scores, integrated analysis of multi-omics data, and development of analytical tools. He's now the Muntz Chair of Cardiovascular Prediction and Prevention at the Baker Institute, an investigator at the Health Data Research UK, Cambridge site, and a Turing Fellow at the Alan Turing Institute in London. Dr. Inouye, the floor is yours.