Nate Silver on the Supreme Court and Underrated Stats for Finding Good Food | Convos with Tyler

Nate doesn't need much of an introduction. He is a phenomenon in the areas of data, sports, politics, online media, all the growth sectors basically. So when I think of your work, I think of you as dedicated to the idea of numbers and data and wanting to apply that to as many different areas as possible. If you were to say, of all the areas of human life, where can data bring the biggest improvements? What would your answer be?

That's a pretty heavy question for— I should have taken half an hour to think about that, right? You know, look, I think the answers are probably obvious in some sense where health is an area where I've not done a lot of work personally, but I'm sure it's incredibly valuable. Doctors are not known for being terribly analytics-driven. I don't know the culture enough as to know why. In terms of areas that I would like us to focus on at FiveThirtyEight a little bit more than we do now, criminality and criminal justice is an interesting area, in part because you have lots of issues with data. If you want to know how many police officers are killed, how many people are killed by police officers, you don't really know that very well. Education is an area where I suspect you have a lot of data used poorly as well as data used well. Urban planning is something we're fascinated by. We did a big analysis of Uber data that New York City spent like $2 million to conclude, but we concluded on our own in a week or two of work, which is that Uber by and large in New York was was not adding cars to the streets, at least not in Manhattan.

So we're in a law school right now. If we applied a lot more data to the law, what kind of improvement could you imagine we might come up with, just tentatively?

See, I think that might be the last field where you would have a lot of— and I don't say that in a pejorative way at all, but a lot of the advantage of working with datasets and becoming more adept at it is that you get an answer that's at least kind of approximately right, whereas the legal sector, I think, relies more on precision. You want a very precise and possibly wrong answer, which is kind of what you're trying to avoid sometimes when you're doing statistical analysis.

I sometimes wonder, you know, how much data do people want? As part of my prep for these, I went back to your high school yearbook, and I I took a look at the quotation you left. It's from Macbeth. It goes as follows: "Then the liars and swearers are fools, for there are liars and swearers enough to beat the honest men and hang up them." It's a little self-righteous, but you're entitled to that when you're in high school. No, I have no objection to the sentiment. But I've read papers which show when you give a lot of people the chance to view the quality of their hospital or doctor, they're not interested. So as a citizenry, how much data do you think people want, or do you think it's a kind of entertainment where sports betting, politics, a kind of horse race, it's fine, but real data, do people want to see data on how good or how honest they actually are, or is it more like the Macbeth quotation?

I'm like that. My partner got really into 23andMe, I guess, and like wanted all this detail. They don't actually tell you all that much, but I'm like, you know, I don't want to have to stress about a bunch of things that I can't necessarily affect. But I don't know. I mean, the notion of empowering people to make better decisions with their own health is a noble notion. I guess I'm kind of enough of a free marketer that I say, you know, you should give people information whether they'll use it well or not. It's kind of their right. To have it, but I'm not sure I have a firm conclusion about whether it leads to better decisions or not. Again, my impression is that among doctors and hospital administrators, that they're not terribly data-driven either, despite their obviously rigorous work in other respects.

I think of you as a kind of superforecaster, to use Philip Tetlock's term. Do you think you can beat prediction markets?— not all the time, but, you know, a smidgen above average. So if this were a game and we were all investors, at the end of 30, 40 years, you'd have some excess returns, just like Cliff Asness, one of our earlier guests.

I think maybe by a very small amount, but not by enough to make up the variance. But it depends on what market you're talking about. I think that, you know, the markets in politics are not all that liquid and not all that sophisticated necessarily. I know the various sports algorithms that we have at FiveThirtyEight have tended to beat Vegas, not by a lot, but will win 52% of the time and so forth.

That is a lot.

It is kind of a lot. But I guess I put it like— I mean, this is what I spend a lot of my time thinking about, is the dynamics between how markets can be— it's amazingly arrogant in some sense for anyone to think they can beat markets. At the same time, the more kind of worshipful we become of markets, then the less useful they become as well. So a lot of times I have people say, "Well, I know, say, Donald Trump is going to win because he's up to 52%." I think he's lower than that now, but one point he was up to like 55% to win the GOP nomination. At Betfair. That, I think, doesn't really add any value to the conversation, and so I'm kind of more interested as a person and as a researcher and journalist in providing information that then other people can aggregate, as opposed to the other way around, I suppose. But if you talk about how good are political markets, the first question is, How good are markets? And I think the answer to that is pretty good, but when the distribution of error is not very linear, when they're off, they can be off by a lot. Are you the person who knows when they're off? That's harder to do, potentially.

What are the differences between forecasting and futurism, and do you have any predictions for the year 2050? They don't have to be great, they just have to be better than the market, right? 52. We'll take a 52% prediction and go home and celebrate.

I mean, I think, I think I'm mildly pessimistic in some ways, although not—

what's the biggest source of your pessimism?

Um, I don't know. I mean, there's probably some, um, some survivorship bias in the United States and kind of thinking about how, um how our way will persevere forever and ever and ever. We were talking backstage about how you go to Asia, I go to Asia not as often as you, but if you want to feel optimistic about civilization, then go there. But I don't know, some of it is kind of thinking about, frankly, this Donald Trump phenomenon.

Mm-hmm, I've heard of him.

Yeah. But I'm not saying— I mean, it just kind of made me consider that a lot of assumptions a lot of people made about how American politics work are really based on a relatively narrow slice of history, kind of post-World War II through 2000 or so, maybe even briefer, kind of 1980 through 2000. It's not really a lot of history. In many other contexts, you know, there are all types of places around the world where nationalism is a much bigger phenomenon than it is in the United States. You know, race and racism is embedded in a great deal of political turmoil in the United States. In some ways, I kind of wondered after the Great Recession, how come we haven't seen more upheaval, more social upheaval? And maybe we're seeing that a little bit delayed. It's kind of more of a revolution of rising expectations. At the same time, there is such a tendency now to focus on— I know in politics people can focus on a very small number of stories that are not representative of the big picture. And there's a lot of wonderful news in the world in some sense in terms of poverty rates going down globally, income inequality going down, diseases being eradicated. But, you know, I wonder to some extent how much kind of the media culture tends to focus a lens on negative aspects of society and lowers people's happiness level and all this type of stuff.

More optimistically, how about love and sex? Do you think data can improve matching? Should we just follow the algorithms, or do you think that's a perpetual dead end and all the algorithms really do is force you to, you know, choose someone? Give you like a phony reason and get you out of your indecision?

I mean, I think that— I think the market would say that people find online services fairly useful. I mean, maybe it removes some spontaneity. I met my partner at a bar, which kind of almost feels old-fashioned now, really.

But it could be like these pills that are sold the online service, or like a placebo effect, or—

I mean, look, I think there's a lot of over-optimization, and that's kind of a problem across almost any sector you'd want to talk about where data is being used. You're kind of optimizing for a short-term equilibrium, and it's much harder to measure the long term. Before, if you couldn't measure anything at all, then, you know, maybe your heuristics aren't that bad. But if you can kind of say, what's going to make me really happy Tomorrow, in my business, what's going to get my website the most traffic 36 hours from now isn't necessarily the best decision in the long run. You can measure the short run and not the long run, measure some things and not others. That can make you, I think, quite myopic and is a bigger problem than people realize perhaps.

You mentioned Donald Trump a moment ago. I had told quite a few people I didn't think Trump could get very far. It's not obvious that I was right. Paul Krugman said pretty early on that Trump had quite a good chance. So what is it that Paul Krugman saw that I didn't?

Well, I was one of the Trump skeptics too. Let me say, I thought you'd ask a version of this question, but—

But I wasn't allowed to blame you, so I put it on myself. I'm still not sure you were wrong, but there's something you didn't see.

Important, I think, right? I got a little frustrated because a lot of people were saying, "Oh, Trump's instantly going to evaporate in the polls." If you go back and look at what we wrote, we said that could happen, but there are also a lot of candidates, Pat Buchanan and so forth, Ron Paul, Rick Santorum, who will get 20, 25, 30-something percent of the electorate, and they have a high floor, low ceiling.

Tyler Cowen. Right.

You know, that could still wind up being true, but with that said, you know, for one thing, we're dealing with a fairly small sample of relevant elections. People look at— in the primaries going back to 1972, and I think one very basic lesson is that when you have a sample size of, let's say it's roughly 15, there's nothing you can do to make it not a sample size of 15, right? You know, no matter how compelling you can make your rationalization to say, well, but, you know, we have theory as well as empirics here, and still 15 cases is 15 cases. And I think maybe making people more cautious about saying unlikely versus never. Now, the record will show we said unlikely and not never, but still it's a lot of things to think about. But I don't know. I mean, you talk about what superforecasters are supposed to do, and it's— That's you. Yes. Yeah. But you start with priors, and you can say the prior is that candidates like Donald Trump tend not to win the nomination. And so what signs could I find that would violate that assumption? Well, it's not necessarily performing well in early polls. Lots of candidates who are flashes in the pan— I guess it's a little tautological, right? But lots of unusual candidates have done well in early polls. Lots of unusual candidates have won Iowa or New Hampshire, not usually both, but one or the other. It's kind of the ability to consolidate the field after that, become a consensus choice of the party that's been more unusual. So that assumption still might prove to be true. I did think, though, that I and a lot of people overrated the ability of the Republican Party to stop what I think is in some ways a radical insurgency within the GOP.

So the party's weaker than you thought. What other judgments about the world do you feel you or I should revise that we once held? Like Paul Krugman, I think, would say Republicans are more racist than many people believe. It wouldn't be my take in particular, but it's a candidate.

To be honest, that's a little bit of what I wanted to resist. So actually, I think one kind of lazy heuristic that in my thinking about Trump that I use is like, you know, there are exceptions, Paul Krugman, Norman Ornstein, who have been very consistent for a long time, but But I thought the people who were pro-Trump were generally not people whose opinions I would weight as highly. And I think that's lazy and possibly quite dangerous, you know what I mean?

If he chooses to run, does Michael Bloomberg have a chance?

Well, give me a probability. I mean—

Well, if I go to the betting markets this morning, I think I saw 2.8%. Is that too high or too low relative to—

A 2.8 chance of becoming president.

President.

It's probably about right.

It's about right.

I mean, obviously in some ways the climate could be as fertile as ever for some type of third candidate running, but Bloomberg, I don't know. Number one, I'm not sure he differentiates all that well from Clinton, with whom he has a lot in common policy-wise, and Trump, with which he's kind of the same character, but the most basic problem is that in an election between Sanders and Trump, or Clinton and Trump, everything is quite left of center. Trump, when he was thinking about running as an independent in 1999, 2000, had an eccentric platform, but involved single-payer healthcare, a wealth tax. He was anti-immigration even then, but pro-choice. He said explicitly, like, I'm not bound by any party really. I'll probably reconsider my stance if I become the Republican nominee. So to me, the more viable candidate in that case would be like a Mitt Romney-Condi Rice ticket or something like that, right?

Let's move past the esoterica and give the people what they really want to hear. Now let's go back to 1968 in the World Series. The manager of the Detroit Tigers, Mayo Smith, he took Mickey Stanley, pulled him out of center field, and put him in at shortstop so that Jim Northrup could play center, and the recovering Al Kaline, who was a better hitter, future Hall of Famer, could play in right field. No one had ever done this before. Now you're advising at that time. How do you start thinking about that problem? You know what I'm talking about, right?

You're from Detroit. I do know what you're talking about.

Of course you do.

We know a lot more now about the value of defense than we did back in the late '60s, or even than we did 10 years ago. And if anything, defense turns out to be quite a bit more important than, than people would have thought. So it was kind of an ironic way when some of the conventional scout wisdom was confirmed as data got more advanced and more sophisticated. Now, if you want to get really complicated, say the Tigers had Danny McClain, I think Mickey Lolich, they had a strikeout-heavy pitching staff, maybe you worry about defense a little bit less. Tiger Stadium is, you know, a park conducive to low batting averages to begin with, although high home run totals. But it's not obvious to me that it was the best move, but it is interesting that all of a sudden baseball teams and football teams become in general more fiscally correct when they have more on the line. In the World Series, closer usage is a lot better where you bring your best guy in in the 8th inning or the 7th inning. You see in the NFL teams will go for 2 more often in the playoffs, go for it on 4th down more often in the playoffs. Just kind of a hint that like, that when the stakes are low, culture tends to prevail. When the stakes are high and, you know, the outcome of the game is all that matters, then things are different.

So given your view on Mickey Stanley and the Detroit Tigers, who is the underrated candidate in the Republican race this year? Just to impose a kind of consistency on you.

I mean, I don't know.

You've got to go along someone, right? You have $100 to bet.

I think the markets are fairly close to correct right now. But I've been a Rubio optimist for a while on the theory that he is the only candidate who really has appeal to all the various sectors and constituencies within the GOP, which may be a fraying party, but still he has the highest favorability ratings in the party. I think he speaks the language of conservatives without being too extreme. But the big question in the election right now is, where is Trump's ceiling? If you start out at 25% like he did in Iowa, that's one thing. 35% is much closer to a point where he'd be hard to stop. But even now, you see in New Hampshire, even though he won with 35% of the vote, half of Republicans there said they would not want him as their nominee. So, you know, the question is, can the non-Trump candidates organize themselves into one candidate? And then does he stop at 35% or 40% or 45% or 51%? If he stops at 51%, then it kind of doesn't matter. But, you know, I suppose, you know, I think Rubio at— what is he at, like 3 to 1? I think he should be more at like 2 to 1 or something. So not a dramatic mispricing.

And what do you think of the Ted Cruz theory that this election is not about swing voters, that you actually need a somewhat less compromising candidate to bring out the silent conservatives who maybe don't vote, and that there's a lot of them? And that Cruz is more electable than Rubio. You hear this, there's data on this, and what's your opinion of that data?

I mean, you know, talk about priors or kind of Occam's razor. If you look at lots and lots of Senate races over time, and the reason why that's relevant is because number one, senators are easier to measure their ideology because they legislate, and number two, we have a much larger sample size. You can see that there's a price for extremism. Not a price that can't be overcome if we go into a big recession or if Clinton or Bernie has huge problems, but Cruz would probably cost you 3 or 4 points relative to the median generic Republican.

Last I saw, Bernie's share was at something like 17 cents. At that price, do you go long or short?

Probably short, but I think it's also not dramatically mispriced. I mean, I think the thing people miss is that kind of unlike on the GOP side where Trump's at least passed, I think, the first test, like, you know, he has people who are out there willing to vote for him. There was some doubt about that, especially after Iowa where he underperformed his polls. But Sanders, we haven't really seen, can he win states that are not very white, and very liberal. Maybe he can. Nevada seems to be pretty close. I'm just saying we haven't really received that much information that would make you update your priors about Sanders all that much. He also probably has to win by— with a little bit of room to spare. If it's a tie, then Clinton will probably win on the basis of superdelegates. If she loses by a couple points, but it's close enough. So if you're an underdog in a football game and you lose unless you win by more than a field goal, that actually reduces your win probability quite a bit. If you're the favorite already, then maybe it wouldn't, but that's tricky. When underdogs win, they tend to win narrowly. And if Clinton wins some of the races she should have won or she should have lost, because superdelegates turn a narrow Bernie win by a field goal into, "Oh, after further review, you know, we're gonna have an overtime quarter, superdelegates will weigh in instead," then I don't know, I think 10 or 15%, somewhere in that range is probably about right.

We may come back to politics, but let's turn to a nobler endeavor, sports. Now you're a fan of baseball, and I'd like to ask you, of all the different baseball records, which is the one that is most impressive to you, or the most statistical aberration, and try to stay a bit modern. So we both know in 1889, Hoss Radbourne won 59 games. You know, start with like Wilson's, what's it, 36 triples in 1912, that and up through the modern age. What's the most statistically impressive baseball record and why?

I mean, I think the biggest outlier is the number of intentional walks that Barry Bonds drew in, what year was it, 2001, where he had like 161. Intentional walks and the next closest player is 50. There's just no other example I can think of in sports where the record holder has 3 times the sum total of the nearest highest player.

And what do you think of streaks? Do streaks impress you more or less than most people? So DiMaggio's hitting streak, consecutive games played streak, Cal Ripken, number— Johnny Vandermeer, 2 no-hitters in a row. When will that be done again?

I'm sure you've read— this is another area where kind of the simple sabermetric answer might not have been totally correct. But there was a lot of talk for a long time about how the hot hand theory was false and how basically things are random to a first approximation. Now there's more evidence arguing against that. And in fact, it's a classic mistake, but if you have a test that has low power, then you may mistake an ambiguous result for a negative result instead. But, you know, it appears now that there is some streakiness, as you would expect. You would expect that there are some variations in human behavior from day to day. It's kind of amazing that being professional athletes, there's less streakiness than you might think. But still, and now we're getting actually data too that's less noisy, so we now get data, for example, on— for major league hitters, how hard the ball comes off their bat. So we have some unpublished research that a colleague of mine is doing, but it looks like you can maybe predict batting average up or down or on-base average 20 or 30 points from a baseline, which in baseball terms is pretty relevant. If you have a guy hitting in the leadoff spot because he's a .370 OBP hitter, and he's really 340 based on his current condition, then he should maybe be demoted down to the 8th spot in the lineup instead. So, you know, this is true of a lot of things where, you know, the first cut from data is overly simplified. You kind of refine over time to something which is a little bit more nuanced.

What do you think is the strongest piece of data-based evidence we have that sports analytics work. And let me give you an example. As you know, the Houston Rockets are run by Daryl Morey, numbers-intensive guy. He's from MIT, seems to be super smart. And right now the big debate in Houston is which of their two star players they should trade, or maybe both, and they may not even make the playoffs. So how good is the best regression showing data analytics even work in sports?

I mean, the Golden State Warriors might be one of the best examples where—

But examples, so clearly, let's say it increases your variance. So the good examples will look really good. But as a predictor, how hard should we be selling it? What's the average impact, the marginal impact?

Well, you talked a little bit before about how hard it is to beat markets, and there's a little bit of this too in— in sports. Actually, a colleague of mine, Ben Lindbergh, just wrote a book that's not published yet, but I was reading about. So he and Sam Miller, another baseball stat head, were given basically ownership of a minor league team in Sonoma, California for a year. They kind of had carte blanche, but they encountered baseball culture in a very head-on way. In the end, the team did pretty well. It was a very obscure minor league, but But if the baseline is making half your decisions right and you make 55% of your decisions right or 53% right because you're using analytics, that's a pretty big gain at the margin. And yet in a sport as noisy as baseball, it's going to take it a lot of time to show up. Basketball is less noisy. I will grant you, but in general, I mean, I think the Spurs are fairly analytics-friendly and the Rockets and the Warriors. I think basketball is probably the best example.

Let me ask you about a sport which I find totally baffling: soccer. There's not much of a natural time unit. It's why it's hard to squeeze in commercials. Not many points. There are assists, not many. They're not always well-defined. Defense, my goodness, more confusing to this American than cricket. Yet market salaries of soccer players are determined. There's some papers on market salaries. The small amount of data we have seem to predict those salaries very well. So the fact that soccer behaves in normal ways, does this mean A, we can still get it all done with limited data, or does it mean B, having more data doesn't really help us that much and all of sports is a bit like soccer and are ultimately just throwing up our hands?— and saying, my goodness, might as well be cricket.

I mean, so I feel like I'm answering all your questions the same way, which is that like, which is, you know, analysis is far from perfect. It'll make lots of mistakes, right? At the same time, you know, pretty good is hard to beat sometimes. And the kind of the heuristics that develop over time about how to value players at different positions in soccer, you know, they kind of appropriately don't value goalkeepers all that much, which the analytics seems to bear out, for example. But I do think though in soccer that we're at the very early stages and partly like there hasn't really been very much data collected at all. It's not like in the NBA where you had blocks and steals, very crude defensive stats. I mean, literally, and we built a system for ESPN a few years ago where the only data that's been kept in the long term in soccer is goals and bookings, red cards and yellow cards. So it's not like we have assists until recently. It's not like we have tackles. You can maybe get time on the pitch if you parse play-by-play records. We don't have crossing passes. And so, you know, I think there's still a lot of room for upward improvement in soccer.

We live in a global economy with billions of laborers. Why don't more of them learn the knuckleball? Wilbur Wood, Hoyt Wilhelm, they were not athletic. The pay's pretty high. R.A. Dickey won the Cy Young Award in 2012 with a knuckleball, which he taught himself. People put labor into a lot of endeavors, but why so few knuckleballers? Why isn't that a more regularized statistical process? Why so lumpy?

So I have two answers. One answer is that there are diminishing returns on the number of knuckleballers in the league. So when you have a second knuckleball in the National League, when you already have R.A. Dickey, that already discernibly affects the success of both pitchers. But, you know, I think there's a second thing, which is that sports tends to engender conformism. A lot of walks of life do. And, you know, I mean, that's the whole tension again that kind of comes up, is on the one hand, knowing that the market is usually pretty good, but on the other hand that there are powerful biases to conform.

So Trump is like the knuckleballer of politics then?

Yeah, that's not a bad—

He throws the knuckleball at Jeb Bush, Jeb is baffled, maybe now he's finally coming up with a response.

Well, it's the other kind of statistical way to think about it is that like when you have something which is unusual, then there's a bigger error bar around it, right? So you could say, well, I think Trump is on average gonna get less of the vote than someone who's much higher than him in the polls, but, um, but he has a much longer tail on either side, and so therefore that's one reason to, to not be dismissive of him, at least in the early going when everyone was kind of a long shot, at least.

Now, in all of these interviews, in the middle segment, we do a little kind of game. It's called Underrated Versus Overrated. And I name a few things and you tell me if you think they're underrated or overrated, and you're free to pass on any you don't want to have an opinion on. New York City, the Upper East Side, overrated or underrated? Are they really that happy in Seinfeld? And how do they afford that apartment anyway?

I think a little overrated, but like, but look, I mean, New York is actually extremely efficiently price. We did an analysis for New York Magazine a few years ago where we tried to say what's the best neighborhood and the problem is that cost accounts for like an R squared of like 0.93 with our quality index or something. And so it's really hard to improve your lot in New York, at least at the neighborhood level. But I don't know. It's hard to find good hole-in-the-wall places to eat. In the— although there's some ramen shops that are getting better. I like basically judge everything by food.

The shortest avenue in Manhattan, 4th Avenue, it's what, 6 blocks long?

Yeah.

Overrated or underrated?

I think underrated. You probably saw my list a few weeks ago where it's, you know, it's short but it's in a very kind of dynamic section of town.

Even with all the used bookstores closed down?

Length isn't everything.

Length isn't everything, okay. The idea of legalizing drugs, overrated or underrated?

By this crowd, probably rated properly, but I mean, I don't know, you know, I'm enough of a kind of lowercase l libertarian where I think that the government ought to have a stronger reason to intervene in choices that people are making instead of a a lesser reason necessarily. To me, it clearly makes no sense to treat marijuana as being a more serious substance than alcohol, for example. I don't think in my heart of hearts if I were running for office or in the Senate or something that I would vote for a bill to legalize heroin or cocaine, but decriminalizing it perhaps, but I don't know. I mean, I think the kind of consequentialist case for a long time probably underrated and may have gone a little bit too far in the other direction, but again, I would say if it's close, then you give people the choice.

The musical group My Bloody Valentine, they put out an album in 2013 after almost a 20-year hiatus. Few people expected this album. It was called MBV. Overrated or underrated?

The group or the album?

The album. We know the group's underrated, right?

Yeah, the group's one of my favorite bands. I think the album was properly rated.

Singapore, overrated or underrated?

Underrated except by you. But I saw exactly why you would like it, right? It has great food, it's like a little laboratory experiment and it's a fascinating— we were talking before about how Singapore is a city that would hypothesize that if you have a few constraints that might seem slightly strange, then maybe having a few strange constraints is helpful. I mean, when you talk about kind of the strange things— my sister lived in Germany for a while and she's like, well, yeah, if you're If you're there in the shop and they're gonna close at 4, it doesn't matter if you have a huge line of groceries, the store will close down and you have to put your stuff back and sorry, you can come back tomorrow, which kind of seems irrational. But Germany has like weird little quirks and constraints and yet seems to be doing, I don't know, fairly well in certain ways, or Scandinavia or something. So if you kind of give up a little bit of freedom to to have more freedom. There's like a Björk lyric which is, "I thought I could organize freedom, how Scandinavian of me." You know, Singapore feels a little bit that way too.

Now I have a big compound question. It's about a few things. Feel free to touch on it what you want, but one of them is sports, one is fantasy sports, and one is gambling, and they're all interrelated. But what's your take on what do these actually do for us? How socially productive are they? So you're of a mixed mind on drug legalization. I could ask a comparable question about gambling, either legalization or liberalization, but what's the net social externality on this mix of sports fantasy, sports gambling? What's your view?

I mean, I know it's— I'm biased. I make my living as a— you know, so One of the things about the regular fantasy football league that you're in with your friends is that you get a lot out of it. You get to hang out with people you might not see very often, especially as you get older into your 30s or 40s or whatnot. You get to watch a lot of games with more of a rooting interest. The thing about daily fantasy sports is that a lot of that is really taken away. It's kind of very much like a brute force approach to watching sports. I kind of did this for a few weeks and then got bored with it, but basically I had like a a program that would randomly generate high-scoring lineups, and so you, you know, then you kind of scrape that data and you load up 200 lineups at a time, and it kind of took all the joy out of it. That's not quite what you're asking, really, right? Right.

I think, what if people just watch sports? Like, I watch sports If it's a good game, I enjoy it. I don't feel I need to gamble on it. I don't want rotisserie. I don't want— I want to read analytics. I want to read you or your people at 538 on the game, which is great, and then watch the game and I'm done. And if they're good players, I'm happy. What am I missing, if anything?

Well, for one thing, gambling and fantasy sports are a good way to teach people applied analytics. I'm not being jokey. I think this has probably a measurable benefit to society. But look, I mean, it's a case where unlike drug legalization, where there are not a lot of countries where drugs apart from marijuana, even there fairly rarely, worldwide people are much more relaxed about gambling and it's normalized. You can go to the betting shop, any Ladbrokes anywhere in the United Kingdom and place a bet, and it doesn't seem to ruin society. Maybe you have in low-paying leagues or in tennis the occasional betting scandal, which is not great, but I think it's a way for a substantial number of people to enjoy sports and kind of develop critical thinking skills. Again, I say if it's close, let people let people do it, and I feel that way about gambling. But in that case, you do have examples of many, many Westernized countries where betting on sports is legal, and there seems not to be at least grave societal harm.

So you run the website FiveThirtyEight. It's your vision. You founded it. You developed it. You took it to ESPN. Over those years, what would you say is the most important thing you've learned about managing?

Basically, there are 3 strategies, 3 fundamental strategies of management when you have a disagreement with something that one of your employees is doing. One of which is you can give up, right? You can say, well, I'm not going to pick this battle to fight because there's a consequence to lowering this person's morale, or I'm tired. I have other issues. So you can capitulate. Number 2, you can fiat. You can say, well, sorry, but I'm ultimately the one who signs your checks, or my boss signs their checks. But this is the line of authority, and we're not going to publish that article. I'll explain my mind later on. And number 3 is you can try and persuade instead, which sounds perfect, except persuasion is really, really time-consuming. Figuring out which ones of those 3 tactics to use and in what ratio is, I think, important. I actually found, though, overall that there's a little bit more value in micromanagement than I thought. Not about everything, but strategically saying, "I'm going to spend a lot of time going into detail on this one." art. I guess it's kind of just kind of mentoring, I guess, is a way to put it.

So which sports coach or manager are you most like? Vince Lombardi? Gregg Popovich? Who do you draw inspiration from? And do you think about it in these terms?

I'm not arrogant enough to compare myself to Popovich. But I'm like, I'm laissez-faire, but like, but when I weigh in on something, I'll weigh in pretty directly, right? I think you kind of have to, you do have to basically have to pick your battles. A little bit, and you have to hire really well. But it's a culture of creatives and a culture of journalists, and journalists are strange and wonderful people, and data journalists are still journalists too. But you have to kind of trust people to make their own decision. I mean, a big thing too is kind of figuring out which one of my the other managers and editors on the staff. You know, what's my agreement ratio with them? It's incredibly valuable to have someone who, without your intervention, agrees with you 80% of the time, and the 20% of the time that they don't agree with you, that they're right as often as not, right? If it goes to 95%, then they're a sycophant, and it's probably bad. You know, if it goes to 60%, well, then you might as well do the work yourself. So kind of figure out the people who will listen to you but also challenge you at the right times.

You mentioned food before. Let's take a data-intensive approach to food. You're trying to find a good place to eat. What is the underrated statistic about a restaurant that you will consult or advocate others consult in this endeavor?

So this is a fairly basic one, but I'd rather look— if you're looking at Yelp or TripAdvisor, the number of reviews is a better signal than the average star rating, especially in reviews relative to how long a place has been open. We've done some work on this too where when you're drawing from a more diverse segment of people— there's some theory I want to invent about how every book on Amazon in the long run gravitates toward having 4 stars, right? So a lot of 9/11 Conspiracy books are rated pretty well on Amazon because like only the conspiracists bother to read them. You know, whereas Othello or Macbeth or something everyone reads. A lot of kids have to read it for homework when they don't want to necessarily, so they'll leave a bad review there potentially. But I think that problem is more acute than people might realize when it comes to restaurants where a place is Notorious will draw in people who might not like that cuisine as much. People also, you know, when I go and I used to kind of do more Yelping and stuff like that, if I go to like a mom-and-pop place in a small town somewhere and it's not very good, there's like almost no way that I'm going to leave a negative review for that place. I don't want to hurt anyone's feelings. I don't want to, you know, and there's actually studies showing that Yelp reviews can, like a 1-star Yelp review can cost like thousands of dollars in business for a restaurant that has under 50 reviews or something like that.

So in New York City, is there better food on the avenues or the streets?

I've read you on this. I think the streets, but New York is weird because they're like really, there are kind of 3 New Yorks from a culinary perspective, right? There's kind of rich Michelin-starred New York. There's kind of hip SoHo and Williamsburg, New York. And there's ethnic New York, for lack of a better term. And making sure that you have kind of a mental list of places from all three types of those. And there's some rules that work well in one of those lanes that don't work well in the others necessarily. So in the very high-end restaurants of New York, it's it's like so competitive that I think your rule about, um, order the weirdest thing on the menu—

mm-hmm—

I think there are parts of New York where that probably isn't true because it's so hypercompetitive that, um, that the menu couldn't afford to lead people astray. And so sometimes the thing that the menu is very clearly pointing you toward in New York is the kind of thing that you would want to, to order instead. Um, that might not be true if you go out to to Queens or something like that, where frankly, pound for pound, probably the food is better than Manhattan or Brooklyn. But yeah, I mean, you could write a whole book, and maybe I will, especially with Trump winning the election or something. Maybe I'll write a whole book about heuristics for eating food in New York.

I said before, and I'll tell the whole crowd, one of my dreams is that someday you write the quantitative history of New York City. And this would be one of my favorite books. Question about the weather, weather reporting. There's some evidence that there's what is called wet bias, so storms are over-forecast. Why is that, and is this even true?

So it's true the further downstream you go. So the local meteorologists here in Virginia or in Washington on TV— They want to get higher ratings.

They're trying to scare us.

They're trying to scare you.

It's like they want the Iraq War, so to speak, so people turn on CNN.

I mean, the irony, though, is that the data the government produces is very well calibrated and doesn't really have a wet bias. There are a few individual models for winter weather that do. But I don't know. It's been kind of interesting in my shoes, going from someone who was a total outsider to someone who who has more reputational risk. To a first approximation, I think it might make someone a worse forecaster potentially. By the way, another thing about the Trump thing I've been thinking about is— Right. So my kind of early view that Trump had a very low chance, not zero, but very low, of winning the nomination was not based on any formal model per se. And I wonder what if I had even like a fairly bad model instead. The good thing about building a Syscall model is that it commits you to rules, right? So instead of just kind of saying, well, well, early polls aren't very predictive and your prior is that Trump probably won't win, therefore probably not. Well, it kind of pins you down and says, well, Okay, early polls aren't predictive, but at one point did they become more predictive? When Trump went from being at 25% in the polls to 35% after Paris and San Bernardino, how significant is that? To have an answer that is set up by an algorithm you designed ahead of time, I think is actually maybe more helpful than people would think. Think, right? So I guess kind of a long way of saying is that, you know, I'm not sure that I'm any better than the average pundit unless I have a model, and the disciplining effect of a model, doing your thinking in advance and setting up rules of evidence, I think is probably quite important.

I have a question about the economics and sociology of sports. This has puzzled me for a while. You may have thought about this, but I'm struck by the relatively small number of professional athletes who have come out as being gay? In Hollywood, it's a lot of people. Even in Washington, which is a very conservative town, I wouldn't say it's a lot of people, but it happens in a quiet kind of way. In sports, why is there so little? And if we applied some kind of economic or statistical model, in which sports would you expect to see the new breakthroughs coming when they come?

Well, I mean, I'm sure there are a lot of athletes in the closet. I don't assume that it's 4% or 5% or whatever the population average is. I assume it's a fair amount lower than that. But I don't know. I mean, I think people forget about how much the economics change when you're talking about people who are in the 0.001%. Of something, right? Where the fact that until fairly recently, until maybe a few years ago, and in many parts of the country obviously still now, until fairly recently growing up gay is something that was, if not traumatic, at least required a lot of bandwidth, right? It requires a lot of energy. And because, you know, the fact that for example there's data from Freakonomics about how Hockey players who are born in January, right, just because they start a little bit earlier than their peer group, that's a very powerful effect versus being born in November or December instead. So if something that minor can have that profound an effect— I don't know if there's like twice as many NHL players from January as December— then something as important to your identity as being gay in a society that until recently didn't accept it you know, that's a competitive disadvantage. Maybe there are also correlations in what kind of skills and traits people have. I don't know, but we'll see. I mean, I guess the prediction if that theory is true is that as it's become more normalized and now people who are growing up in middle school and in high school where being gay is not as much of a disadvantage, then you'd expect from that generation there to be substantially more gay athletes.

And in which sport will that happen first? What's the implied prediction? Um, so we see a bit of it in women's tennis, right?

Women's tennis.

Individual sports maybe over team sports? Yes? No? You would—

yeah, you would think that in, in tennis and golf you might see it first. Um, you know, the NBA, where talent is so manifest and one player can make so much difference, right? I mean, LeBron James could come out as gay tomorrow, and I think it would not hurt his ability to get a max, max-plus contract at all.

But it could hurt endorsements.

It could hurt endorsements. I mean, he is kind of a high default, right? But, but I don't know, I think it's no longer about kind of the marketing side of it so much as the fact that that just kind of I know, sports is still a very conformist culture. So the reason why I might say the NBA is I think it's a little bit more individualistic as a culture, and guys are free to express themselves more. Listen to baseball players talk, they're boring as hell, right? Kevin Durant or something, these guys are smart and they're interesting.

Kareem Abdul-Jabbar, right?

Kareem Abdul-Jabbar. And so I would think basketball might be a sport where you'd see it relatively soon.

Let me ask you a general question about forecasting. And I worry about this in the context of finance. So I see a lot of money managers. So there's Ray Dalio at Bridgewater. He saw one basic point about real interest rates, made billions off of that over a great run. Now it's not obvious he and his team do any better than anyone else. Peter Lynch, he had fantastic insights into consumer products. Use stuff, see how you like it, buy that stock. He believed that in an age when consumer product stocks were taking off. Warren Buffett, a certain kind of value investing, worked great for a while, no big success, a lot of big failures in recent times. Is it possible, like, the so-called true model is always shifting and there's a kind of selection bias where different forecasters are elevated— and they have their run for 3, 5, however many years, and then the true model shifts and what they're good at isn't valued, and we sort of turn them over, replace them with other forecasters. As, like, our best forecaster, do you worry about this?

Oh, sure, yeah. I mean, even if you are skeptical about the efficiency of markets, if you have a great gig and you're kind of picking up $100 bills off the ground, then Boy, if you can extend that by 3 or 5 years without adapting and evolving, I mean, that's on the extreme high end, I think. 3 or 5 years is a kind of very long and fortunate run. But that's part of why, even though now we're very immersed in the election cycle, but it's part of why I wanted to make sure that FiveThirtyEight was not just an election site. We're going to blow an election sooner or later. We might blow this one. And to be doing a whole diverse array of things, both kind of intellectually and commercially, I think is important. I mean, the next question that follows is, "Are there people who have the skills to find the next underweighted opportunity?" And maybe, you know, that's trickier. But yeah, I think a lot of people are— have one or two really good insights.— and if you're very lucky, then that can take you a long way.

Here's a related worry. It's clear in the data stock market volatility is correlated with itself over time. So if you have some volatile days, you're likely to get more. That's pretty clear. So that's another way of saying those returns for a while are hard to forecast and stay hard. So this year, politically, it's already a big surprise to me, to a lot of people. Could it be the case we're entering a new era where political volatility is higher and basically all forecasters will just do much worse than they've been doing?

It's possible. I mean, again, I kind of go back and saying what people take to be the equilibrium baseline condition may actually have been an outlier instead, right? That you had this relatively stable kind of long boom politics and economics from the '50s through the '90s or the early 2000s, thereabouts, and that could potentially reverse itself. Again, to looking at examples outside of the United States, I think, is instructive. You know, maybe I'm kind of more of a believer in American exceptionalism than I thought, but you see constituencies that are Trumpian in different parts of Europe and have been extant for for a long time. So, you know, maybe America just got really lucky for 50 years.

Nassim Taleb has a hypothesis that in some ways the world is getting weirder. So there's the example of plane crashes. Planes used to crash a lot for pretty normal reasons. Well, the engine would fall apart. Obviously we invested more resources in making planes safer. I just read in the Wall Street Journal last year There were actually zero deaths from jetliner crashes other than terror attacks. So we have strange events like the Germanwings pilot flying into the Alps. Malaysian Air disappears. No one knows why. So the events people talk about— we're left only with the weird ones. So do you think we're headed toward a future where we're only going to be talking about weird, very hard to forecast events precisely because we get good at avoiding a lot of problems and mistakes?

No, for sure, right? Where, I mean, there's some stupid metaphor I use in, in the book where one of the problems with comparing how shortstops play, for example, is that you always kind of evaluate players who are on the edge of their range. Can they make this spectacular diving catch? And to a first approximation, you know, everyone is equally good at the edge of their range, right? But the question is, how much territory do they cover in between, the non-spectacular plays that we can miss potentially. And it's probably more true, you know, one reason why I like when we forecast sports, right, is you have a chance to build up your sample size. A perfectly routine, you know, Wizards versus Cavaliers game where we have the Cavs favored at home and they win, you know, that counts. You get hundreds and hundreds of those over the course of a season, whereas whereas in politics you're kind of more drawn to the spectacular and the weird events. And, you know, a lot of models are, are good heuristics when conditions are fairly normal, and they don't deal all that well with the edge cases because they're poorly designed or because you have non-linearity or because they have small sample sizes or whatever else. But, you know, How well do models deal with the weird cases versus other types of heuristics? I'm not sure. Maybe the advantage is more in the, in the kind of baseline cases instead.

Other than skill with data, what are the personal qualities of good predictors?

Um, I think you have to have a certain mistrust of conventional wisdom. I mean, that's a tricky thing, right? That on the one hand We know that I'm not that smart, that this room is way, way, way smarter than me and a market is way, way, way smarter than me. At the same time, people are social beings and they kind of behave in herds sometimes. This is easier in politics than almost any other field because the political press corps literally is kind of a herd. I mean, it's like the perfect example of it where you have a few hundred journalists who travel around who are all reading one another on Twitter, who are all talking to one another. And so, you know, it's like not like 500 really smart people, it's like 1 or 2 really smart people and 489 followers instead. But I don't know, we get ourselves in a little bit of trouble, I think, at FiveThirtyEight at times because we are fairly, fairly combative. And for a long time I kind of thought, well, it's kind of part of my personality and the kind of more happy warrior data side's part of it too. And I think that they're actually kind of sides of the same coin. You know, when you read the New York Times or the Post, not basic factual statements where they say today Donald Trump was in Arizona, but when there's a piece of analysis that isn't necessarily obvious, to say, boy, there might be a 40% chance that that's basically wrong right? You know, that leaves you in a weird place kind of, and that's kind of— but to believe that is, I think, the source of a lot of the healthy skepticism that we have and also some of our failings sometimes.

Now let me get to the question that maybe the crowd most wants to hear. Who will be the next president of the United Arab Emirates? This is a trick question because it's a hereditary monarchy. But here's my background question. Intelligence agencies and scholars did very poorly forecasting the Arab Spring and did very poorly forecasting ISIS. So you're put on the case, someone from Washington, McLean, wherever, they call you in, they say, what variables should we be looking at to understand the Middle East that we're underweighting right now? I know it's a tough question, but who Who will be the next president of United Arab Emirates? Will there be a next president? How do you think about what's happening there? Black swans or a regularized process?

The one, we all have certain compromises. I don't know that much about international politics, even enough in a fun setting like this to speculate all that much. I flew via Emirates Airlines, it's like my extent of my knowledge about The UAE, pretty much.

Not this election cycle, but 4 more years out, this nation, what's your best pick for who will be elected president?

Who will be president in 2020?

Correct.

I mean, the boring pick is probably Hillary Clinton still.

Okay, and number 2, next best pick?

Um, I think it's Close between Donald Trump and Marco Rubio. Although I think Trump might be a one-termer, if that.

Who's the most likely next vice president?

Uh, John Kasich maybe seems like tailor-made for the vice presidential—

Even if you think Hillary is more likely to win, he may be the single individual most likely to be the next vice president. Is that the right way to frame it?

I think that might be. I mean, Hillary has a very long list to pick from, a lot of tactical objectives that she would want to fulfill. I think it's probably a shorter list for the GOP.

Can we apply data analysis to figure out the next Supreme Court pick? Again, not to know who it will be, but to get that 52% edge up on what other people are thinking.

Potentially. I mean, there are some kind of fledgling attempts at at Supreme Court analytics, although this is also a case where we're kind of in a sample size of zero, where you have a nominee who's very unlikely to be confirmed, but there are still high political stakes, you know. My uninformed guess would be maybe Srinivasan, who was confirmed 97 to nothing, I would tend to think that— I don't know. My hunch, and this is just a hunch, is that The theory is that either Obama nominates someone with kind of unimpeachable credentials and makes Republicans look very unreasonable, or he makes a pick that kind of trolls Republicans and plays to the Democratic base. I'm more of a believer in the former as kind of Obama's mode of doing things. I think he'd kind of push things either way and have someone who— might just be at the risk of pissing off the liberal base, but the Republicans have to look ridiculous opposing, as opposed to the other way around.

Now my last question before we get to the crowd. As you said before, we have a lot of same interests: food, travel, sports. Not sure politics counts as one of mine, but in a broad sense, politics. You've taken a lot of trips, some for work, some vacation. If you apply data analysis to those trips, what do you learn about what makes for a good trip and what can you do or what can we all do to have better trips?

I mean, I just love travel so much. I mean, so I had an unintended experiment where I went to Hawaii for, I guess, 2 Christmases ago. And for some reason, I like sat on my phone and my phone didn't work. We were flying through Portland for some reason. We were flying New York to Kansas City to Portland to Honolulu. Don't ask why. But like, and the day I was in Portland, I was like panicked. We like drove way out to this like strip mall on the edge of town. And like, they're like, you have to wait in line 2 hours for us to replace your phone. And so I didn't have a phone in Hawaii. And it was like the most amazing thing, pretty much.

So you've repeated that experience each subsequent trip? No.

But no, I was in Thailand, by contrast, this Christmas, and like, I had to build the goddamn primary election model, and so, yeah, your enjoyment goes down a lot. Like, a little bit of work, you know, working 20% of the time, I think, reduces your enjoyment by 70%.

Here's how we're going to do questions. We have two mics, one on each side. We'll run two queues. I will alternate. These are questions for Nate to speak to. They are not statements. If you start making a speech or a statement, I will cut you off. Even though we do not have Kareem Abdul-Jabbar here, this will suffice. So please just ask a question, and it's fine to introduce yourself if you wish, and then Nate will respond. I will start over here. First question, please.

Hi, my name is Caleb. We talked a little bit earlier about superforecasters, and I was wondering if you've ever considered incorporating the work of superforecasters into FiveThirtyEight. You mean literally the guys who wrote the book, or? Or like, you know, getting a market of superforecasters to help you make your models better. I guess I find crowdsourcing sort of boring. As a journalist, I find it boring, right? Even though if you're like in a business setting, that's exactly what you should do. But, you know, we actually are doing a little bit with the Oscars this year. We kind of found 8 different people who created different models and we're seeing how well they do. Of course, 6 awards is not anywhere near a sufficient sample size to deal with something like that. But I don't know. I think I'm very process-driven as a person. And for me, a lot of the joy is in thinking through the process of it. So reading a book like this is really useful because they talk about a process I think is pretty great. But to actually publish the projections in some sense is kind of— kind of almost beside the point in the sense that you're still probably dealing with sample sizes that are too small to really tell you all that much. When you start to do that, then I think it kind of takes focus away from thinking about kind of process and heuristics for forecasting. So it's kind of an unsatisfying answer, I guess.

We've printed out a lot of Nate's columns. Many of them are here, so there's plenty you can ask about. Next question.

Hi, Michael Willey. Thanks for coming. If it winds up being Clinton versus Trump, is that the first time where we've had two candidates with the highest unfavorables going against each other? Yeah, I would think so. I mean, there have been a few candidates. Romney was basically break-even when he was nominated, as Obama was. But Clinton is negative 10, and Trump is negative— Excuse me, 25 or something. You know, there probably will be some reversion to the mean in both cases. Remember, one reason why— and I'd say Trump's a fairly heavy underdog if he wins the nomination, but it's a conditional probability. Conditional on having won the GOP nomination, Trump will have had to display some staying power, some acumen, because sooner or later he will have to get beyond 35% to win 50% or so, and probably will have done something to improve his image with people who are not in his kind of core constituency. But yeah, I mean, it would be pretty unprecedented, certainly. I wonder if you have to kind of adjust— in baseball, you have to adjust stats for the era, right? Where if you're in the kind of home run or steroid era, then 50 home runs doesn't matter as much. Like maybe now Obama with a 48% approval rating is at like a 56% park-adjusted approval rating. I'm not sure, maybe.

Next question.

Hi, my name's Tom. Earlier Tyler Cowen brought up the question, how much data do people want?

I was—

a two-part question. Does the amount of data that people want, is that influenced by the way data is presented? And the second part would be, what advice would you give as far as presenting data or visualizing it? Well, visualizing it might be some of the advice, right? People seem to learn a lot better from visualization. I think one thing I think a lot about as a journalist is preferring simple models to more complicated models. There are other virtues of simple models. People can also take it too far. But as a journalist, for example, to have something I can say, this is a benchmark and I understand what it's doing. And I can explain what it's doing. And I can also understand what the limitations of it might be. And so I kind of know which direction to lean relative to that baseline, I think, is more useful than a place where you just kind of say, well, We fed some data into a random number generator or magic machine and here's what it spit out, right? So for example, like I highly prefer— this is going off on a tangent— but I prefer regression-based modeling to machine learning where you can't really explain anything. To me, the whole value is in the explanation. But I do think kind of likewise people when you kind of explain and say, hey, we probably have the same interest here in mind to say this is actually, is actually pretty simple once you start peeling away the BS. Like, to me, that approach works a lot better in the long run than the approach of saying arguments from authority. Well, this is rigorous and empirical and objective, so therefore believe the numbers. I think kind of explaining people why it's actually not all that complicated and why you're making very defensible assumptions, how that leads you to an answer that might surprise them.

Next question.

Frank Mannheim, School of Policy. Could you put some numbers on the criteria that the median voter would use in the United States to elect major politicians like president? For example, emotions, personal acquaintance, rational concepts, information, and so on. So the kind of classic political science answer is that people are deeply concerned about the economy, and that the economy might make up 50% or so of what people vote about. And there's room to dispute that. There's kind of esoteric critiques of maybe these models are overfit. But leaving that aside for now, I don't know. I mean, I guess one of the reasons why why I was initially skeptical about Trump is that America has a history of not nominating candidates, and electing candidates rather, who are blatantly unfit for office.

I have a softball follow-up question to that. Where in the state of Virginia, to the best of my knowledge, you're the only person to have calculated correctly What is the chance, if you are a voter in the state of Virginia, that your vote will sway a presidential election?

Oh, it was pretty high, right? I think it was like— oh, as an individual voter?

Individual voter. What's the chance that your vote in the state of Virginia will matter? If you don't remember, I do, but it's from your paper.

I mean, it's like 10% divided by 4 million or something?

It's 1 out of 10 million, the highest of any state. So if you're gonna vote anywhere, vote here. Next question.

Hi, Tyler O'Neill, a reporter with PJ Media. My question is, you mentioned how difficult it is, the weakness of empirical models when predicting presidential elections. Is it possible to look at congressional elections, you know, House of Representatives races, and draw more information and modeling from those.

Yeah, I mean, I think we would say that even though it's less kind of sexy to predict Senate races or congressional races, that having larger quasi-independent samples is— that would be the better test ultimately. Although even there, the error— so we saw in the Senate races last year how, how the polls were off on average by 3 or 4 points. Which is pretty bad. It's happened before, but the problem is that all those errors were in the same direction. So Republicans won a lot of races around the country that they were underdogs in. Not huge underdogs, because Virginia was almost a major upset. But yeah, I mean, you know, it's a much kind of purer form of the exercise to do data mining on congressional elections and weighing polls versus fundamentals. And whatever else.

Next question.

Hi, my name's Harold Walbert. You mentioned some things like limited data, limited observations, non-linearity, and things of that nature that make traditional tools like statistics and econometrics difficult. What are your thoughts on more computationally intensive methods like agent-based modeling for dealing with these things, like you mentioned herd behavior, that make that make some of these analyses more difficult? I mean, agent-based modeling is interesting. And there is some— if you can kind of simulate the underlying mechanisms— this is how weather forecasting works, by the way, is that weather forecasting is not particularly statistically driven as we think of it. They actually are kind of creating a physical model of the atmosphere, which they're resolving mathematically. So if you have reason to know exactly kind of how certain people would behave and how they behave as a system, then agent-based modeling could give you insights you couldn't get from a regression analysis. On the other hand, if you're somewhat wrong about those assumptions, then things could go kind of very haywire in a hurry. You know, when I'm building models myself now, I spend a lot more time kind of thinking about about the edge cases, right? Say, let's put some really weird inputs in here that are on the edge of plausible and see how the model responds to those. You know, maybe you have a function that's approximately linear but can't be at the edge case. It would say that, you know, you know, for example, if you have a model saying that Hillary Clinton will get 106% of the vote in Washington, D.C., or something against Trump, right? You know, I used to think, well, who really cares? She's gonna win D.C. anyway. She may, right? You can vote twice in some parts of D.C. But now I've kind of— that bothers me more, right? So I'm trying to kind of think more about the correct functional form of a model that would apply when the going gets weird because When the going gets weird is when things are interesting anyway.

We have 4 minutes left. Next question.

Hi Richard, I'm an intern at the House of Representatives. Do you believe Facebook and Twitter, where people create their own news feed, has led to possibly confirmation bias and has led to people choosing more extreme views of political ideology such as socialism, nationalism, Marxism?

Perhaps. I would also say that the kind of traditional two-dimensional political spectrum is kind of a strange and contrived thing too. I mean, it's the result of a very messy process of coalition building between parties. So, you know, I mentioned reasons to be pessimistic earlier. A reason to be optimistic as a fan of democracy is that you are seeing voice given to quirkier ideologies that are no less intellectually coherent than the kind of Democratic versus Republican acts that we have in the United States. But you know, I kind of believe in the notion of a filter bubble where people kind of surround themselves where they're getting like information and not confronting themselves with unpleasant facts necessarily. You saw that a lot during the 2012 election where the polling was, I guess, a lot more straightforward than it is this time around, and people still were kind of cherry-picking data to tell themselves that Romney might win. You saw Democrats doing the reverse, by the way, in the 2014 midterms, more or less. But yeah, I mean, I, you know, um, as someone who's a critic of media, I think the way people consume media is important and has probably fairly large effects on our politics.

Last question.

Hi, Mike Blyly. I'm a law student here, and I get my coverage of the election exclusively from FiveThirtyEight. And I do that largely because of the unbiased nature, except for Harry's unabashed love for Chris Christie. But, you know, I noticed that Specifically in your debate coverage, one of the things that you all always mention is that the mainstream media's portrayal of the debate matters more than anything else. When they say that someone wins, that coverage carries. And then at the end of those pieces, you and your staff put together grades for how the candidates did. And you may see where I'm going with this. You strike me as someone who would rather predict rather than influence. But do you see yourself playing into this zeitgeist where you could carry some weight in, say, this election?

I mean, that's why the primaries, although they're fun, are a little tricky. I think in the general election, people are fairly sensible and retreat to their corners, but the primaries are so momentum-driven, then it's a little bit weird. I'm sure people do read what we say and so forth. It's kind of not the type of influence that I want. At the same time, You know, the fact is that all news coverage is influential, and I think I would say at the very least we promise some self-awareness, that we're aware that the way the events are covered by the press can affect voters' views. Sometimes the press can be surprised that it doesn't go the way they expect, but you can have these big feedback loops, and I'm surprised how difficult it is. I think one big edge we have— I'm glad that you read us, right? But I think one big edge we have over, say, the New York Times or something is that we can talk about the media as a political actor. Now we are the media too, and so I'm kind of aware of the circularity of that. Frankly, I think one reason why during the primaries sometimes the conservative sites are more interesting to me than liberal sites, that they also start out being more suspicious of the media. Sometimes in ways that I think are wrong, like about the polls in 2012, but I think having that skepticism and seeing the media as a political actor instead of a kind of benevolent umpire is to a first approximation the right way to do things, and that's reflected in our coverage.— I guess sometimes at the risk of being a little bit hypocritical potentially. But we do try and be very transparent about what we think is a fact, what's an opinion, what's an analysis, kind of what is a provocation. One reason why I like your blog is that you have a lot of provocations, right? And they're clearly— sometimes you put the "Tyler Cowen" label on it. But it's clear what they are, right? It's clear that they're provocations meant to incite discussion and debate. And so we'll have a few of those too at times. But kind of speaking in the first person, I think, is important in breaking from the kind of voice of God where a storm cloud gathered on New Hampshire today and the voters decided that— speaking as a subjective individual trying to understand what the objective world is like is a lot of what we're all about. It's not for everyone, but I think that should be reflected at least in the tone and approach of our coverage, even where we wind up getting things wrong in the end.

Here's Nate's book, read Nate's site. Nate, thank you for a great chat.

Thank you.