Richie CottonSenior Data Evangelist, DataCamp · Ask me about data and AI upskilling

Python Tutorial: Statistical Thinking in Python I (Part 3)

Name: Python Tutorial: Statistical Thinking in Python I (Part 3)
Uploaded: 2017-03-07T00:01:43+00:00
Duration: 1 min 50 s
Channel: Richie Cotton
Description: In this segment of the tutorial, the instructor uses an empirical cumulative distribution function (ECDF) of 50 flower petal‑length measurements to…

7 Mar 2017 · 2 min

Watch on YouTube

1:50

In this segment of the tutorial, the instructor uses an empirical cumulative distribution function (ECDF) of 50 flower petal‑length measurements to illustrate core ideas of statistical inference. He shows how the sample mean can be computed and marked on the ECDF, then explains that this single statistic does not reveal the true mean of the entire population of flowers. By invoking probabilistic reasoning, he describes the uncertainty surrounding the population mean and the expected variation of sample means if the measurement process were repeated many times. The speaker emphasizes that probability provides a language to express how future samples are likely to behave—specifically, that future sample means will tend to cluster near the observed mean rather than being arbitrarily far away. This probabilistic viewpoint is presented as the foundation of statistical inference, converting raw data into statements about what we might expect in repeated experiments. The segment concludes with a promise of exercises to practice these ideas and a preview of learning to “speak” the probabilistic language.

Chapters

Imagine you measured the petal lengths of 50 flowers of a certain species. Here is the ECDF from those measurements. From what you have learned, you can compute the mean of those 50 measurements and I'll annotate it on the ECDF with a vertical line. That is useful. But there are millions of these flowers on the planet. Can you tell me the mean petal length of all of the flowers of that species? If I measure another 50 flowers, I get a similar but quantitatively different set of measurements. Can you tell me what value I would get for the mean petal length if I measured yet another 50 flowers? We just don't have the language to do that without probability. Probabilistic reasoning allows us to describe uncertainty. Though you can't tell me exactly what the mean of the next 50 petal lengths you measure will be, you could say that it is more probable to be close to what you got in the first 50 measurements than it is to be much greater. We can go ahead and repeat the measurements over and over again. We see from the vertical lines that we expect the mean to be somewhere between 4.5 and 5 cm. This is what probabilistic thinking is all about. Given a set of data, you describe probabilistically what you might expect if those data were acquired again and again and again and again. This is the heart of statistical inference. It is the process by which we go from measured data to probabilistic conclusions about what we might expect if we collected the same data. Again, your data speak in the language of probability. Let's do a few exercises exploring these ideas, and then we'll come back and learn how to start speaking this probabilistic language.