A Brief Introduction to Bayesian Mathematics | Mathematician Thane Plambeck

So this is a table that shows your odds of dying in the next year by age in the general population. So when you are age 1, a lot of children don't live past their first year because they have some sort of birth defect problem. But if you make it to your second birthday, you have only about a— if you're male, you have about 1 in— 2. 1,762 chance of dying in the next year, okay? And of course, you can see as you go— the older you get, by the time you get to age 90, it's a 1 in 6 shot that you'll live even one more year, okay? So for someone like— I mean, you guys are mostly between 20— I'm 51, so I'm like somewhere here between 1 chance in 409 and 1 in 160 of dying in the next year. But for you guys, the risk is actually quite a bit— I mean, it's much better, right? I mean, if you're male, age 20, it's only 1 in 5,000. So if you decide to not have the surgery, you actually are multiplying your chance of dying by a huge factor. So in some sense, I mean— and I'm not saying this is the right way to view it. I'm not saying that this is the right way to think about it. But I think there is some truth in the idea that you— It does make sense to have the surgery when you're younger because you get a much— you know, you're really reducing your risk of dying a lot by doing it. Whereas here, at my age, it's not really changing it much on a percentage basis. I mean, it's multiplying it by 4 rather than, you know, 20, okay? And it's not that this— and this is not a completely fanciful situation because, you know what? What happens, of course, is that as a person gets older, you do get tests like this. And there's the— it's been in the news even, at least for men, for the prostate— the PSA test. It has— some people test positive when probably they shouldn't even take the test because they get led down a kind of a surgical path that is very unpleasant and then for many people maybe not necessary. OK. Underlying this is this thing called Bayes' rule. Bayes was— he was a mathematician, a nonconformist minister. Most people who worked on math at this time, they usually had something to do with the clergy. And this is the formula. I'm not going to talk about it in any detail, but it's a key idea in statistics. How do you compute— The probability. a probability of something happening if you have a prior distribution, you have this 1 over 10,000 thing plus the information from some test. How do you put those 2 pieces of information together? And he would have been completely unknown had he not— well, he died, actually, and then he had— someone found a paper on this finding in probability, and it was read to the Royal Society after he died, and then he became very famous. He's still famous. They have a conference for him every year in the little town where he lived. Okay, so this is called Bayesian inference. How do you take prior information, some beliefs of probabilities, what's likely to happen, and then a set of observations, and then compute what you think is going to happen? Okay, this is one way of doing it. It's called Bayesian inference. And this is a— now I've got several references if you want to read more about this. This is a good book that came out the first time I gave this talk, had just come out, How Bayes' Rule Cracked the Enigma Code. This is a good book to take a look at. Does anyone know who this Bayesian is? That's a tough one for an international— yes? Maxwell Boltzmann. That is Nate Silver. I should have you stand up, but you look very comfortable. Yes, this is Nate Silver, who in the 2008 US presidential election, he took all the polling data and used that as prior information plus Bayesian methods to predict that Obama was going to win and pick exactly which states would go for Obama, and he got it exactly right. In 2012, he again— no, he missed one in 2008. In 2012, he did it again. More people were watching him then. I mean, I was a huge fan in 2008, but by the time, like the last presidential election, he was on TV doing the same thing. And he again— this time he did get every single state right. And this is a book that he wrote, which is actually quite good if you want to read about this. The Signal and the Noise. And he's a poker player and uses Bayesian techniques to play poker. There's a lot— there's a lot of great stuff in that book. OK, so Bayes' rule is so great that people make it into a neon sign. OK. OK, I want to say something about how this fits— how it fits into How I think it fits into making predictions, generally speaking. So there's this other point of view on statistics, which is— and this is kind of the philosophy of how you do inference, philosophy of how you make predictions, and how you do science. There's a criticism of this whole Bayesian inference thing, and it basically comes down to where did the 1 over 10,000 thing come from? How do you know that? Right? Usually life is a lot more fraught where we— bad things are happening. People are getting cancer. They're dying. You know, all kinds of bad stuff is happening. We're trying to understand why it's happening and we're not satisfied by just saying, well, it's just a random process. You know, gee, it looks like it's about 2 people out of every 1,000 or whatever. In fact, you know, we want to understand what causes things and we don't— we're never given— this kind of golden thing of, you know, oh, it's just a random thing, 1 out of 10,000, right? And this is a valid criticism. And in fact, all of 20th century science, I mean, all the advances in physics, for example, they take this frequentist view where you don't really know much about the prior information. What you do instead is you do experiments. And then from the findings of the experiments, then you try to draw inferences from the experiment. Experiments. So these are 2 different ways of looking at things. So why is Bayes' rule so popular now? Why does it have this huge uptake right now? Well, there are 2 reasons. Number one is big data. We have computers, software, all kinds of things going on where we're collecting a lot more data about everything. And so it turns out to be a pretty good assumption that what you type into Google when you start typing in, you know, N-E-G-R-I-L, Negril, which happens to be a place in Jamaica where I came back from, you know, that you're talking about the place in Jamaica, right? So we have— there's a lot of— and so many algorithms take advantage of large amounts of prior information, like who's gonna click on a Facebook ad? How do I do voice recognition on a cell phone? And these things are different from science, right? They're different from, say, medical investigations, right? We're often— we're interested in how can I get a good answer quickly? How can I use all of what I know about what people have been searching for on Google or whatever to make a good decision? Okay, and here, this is kind of wordy, but it's actually exactly the point. And if you want to know more about this, I find this kind of thing fascinating. You may not, but anyway. So Bayesian statistics— I'm just going to read this, although that's bad form, I know, but anyway. Bayesian statistics is well suited to individual researchers or research groups using all the information you have available and where you want to make quick progress And you tend to be aggressive and optimistic. You just want to get your best guess of what's going on. You know, it's that search, you know, someone's typing something into Google, what do they want, right? Whereas a frequentist, this is science, right, where they are— they have to be more cautious and defensive. They know that if they say, you know, this drug works, that someone else is going to say, well, where's the evidence, right? And so they can't make assumptions about, well, I think it's going to work. You know, they have to drop that on the floor. All you have— all I can do is perform experiments and then offer evidence that it actually works.