Michael KaminskyMarketing Science | Causal Inference | Incrementality

Simulating data teaches statistics better than memorizing formulas

Name: Simulating data teaches statistics better than memorizing formulas
Uploaded: 2025-11-17T13:00:09+00:00
Duration: 1 min 49 s
Channel: Michael Kaminsky
Description: In this conversational YouTube segment, Michael Kaminsky recounts his unconventional journey into Bayesian statistics, describing how his reliance on…

17 Nov 2025 · 2 min

Watch on YouTube

1:49

In this conversational YouTube segment, Michael Kaminsky recounts his unconventional journey into Bayesian statistics, describing how his reliance on frequentist regressions in Stata and SAS left him uneasy about opaque, "black‑box" methods. To demystify these techniques, he began generating synthetic data, applying statistical procedures, and observing outcomes, using code rather than symbolic math to build intuition. This hands‑on simulation practice naturally led him to Bayesian approaches, which often start with a data‑generating model. He argues that simulation is a powerful way to test whether methods behave as expected, to explore their limits, and to learn statistics more deeply. The co‑speaker concurs, noting that modern computing lets practitioners replace much of the traditional analytical work with simulation‑based inference—a cornerstone of what he calls "computer‑age statistics." They emphasize that while formal mathematical exposition remains important for scholarly communication, simulation can provide uncertainty estimates and insights when analytic solutions are intractable, though practitioners should be aware of situations where simulation may fall short.

Chapters

I'll share a little story. I came to Bayesian statistics through a weird circuitous route. I started out doing frequentist regressions in Stata and read the Stata and the SAS manual really deeply to try to figure out what magical incantations to apply to some dataset to get something out on the other side. At some point, I got really worried about my lack of understanding of what was happening in the black box. My way to deal with that was to start simulating data, applying the methods and seeing what came out on the other side to understand what was happening, not via the math and the symbol manipulation, which I'm not particularly good at, but I am quite good at writing code and generating data. Once I started doing that more and more and using the simulation to understand the statistics, the transition to like Bayesian methods, which often rely on first specifying a data generating process, felt natural. Any sort of statistics that you're doing, this simulation method works really well, especially in more black box methods that you might be applying, simulating the data, see what happens. Do you get the results you expect on the other side? Is a powerful way to learn more about the statistics that you're doing and test the methods. What situations does it work versus where doesn't it work? And that's a really, like, I think, important thing to know when you're trying to use these methods in practice.
Absolutely. With computers, a lot of math is actually not that important. Of course, like, you know, you want to write an academic paper about how a model works, you better understand the math, but At the end of the day, these simulation-based techniques can substitute for that entirely, right? This has been the backbone of what people can call computer age statistics, which sounds old school, but there's a book with that title. It's a pretty interesting read, but just in general, the whole idea that even if we can't do them, almost all of Bayesian statistics is based on the idea that there's a lot of math we can't do when we want to compute these uncertainty estimates. A lot of times we couldn't do it. If you sat down with the math, you couldn't do it. But a computer can simulate it and you can figure out things based on those simulations that will give you the answer that you need.