Learn how to run a permutation test in R to check if soda preference differs between coasts. The instructor shuffles drink labels randomly, calculates preference differences for each shuffle, and compares the observed difference of -0.09 to the null distribution to see if it's extreme.
Chapters
The idea behind statistical inference is to understand samples from a hypothetical population where the null hypothesis is true. For example, from east and west coasts where COLA preference is the same. As a way of summarizing each of the null samples, we calculate one statistic from each sample. Here the statistic is the difference in the proportion of west coast people who prefer COLA as compared with the proportion of east coast people who prefer cola where each of the sample proportions is denoted P hat. The difference in P hats changes with each sample. First it's 0, then it's minus 1/3 and it will keep changing. We can build a distribution of differences in proportions assuming the null hypothesis that there is no link between location and soda preference is true. That is the null samples consist of randomly shuffled soda variables so that the samples don't have any dependency between location and soda preference. The original sample proportions are P hat east of 0.82 and p hat west of 0.73. A difference of minus 0.09. The first shuffle of the drink variable gives the exact same summaries as the observed data. The second shuffle, on the other hand, gives 27 people on the east coast who prefer COLA as compared with 20 on the West coast who prefer COLA. The difference in sample proportions for the second shuffle of the data is minus 0.02, which is less extreme than the original data. Note that both the original data, the red line, and the first two shuffled differences in proportions can be plotted together. The next few shuffles give differences in proportions centered around 0. Note that the fifth difference is minus 0.16, which is farther from 0 than the original data. That is, the fifth shuffle gives more evidence of a difference in soda preference than the original data does. And we know that the fifth shuffle was created by randomly permuting the labels, so a difference of minus point one six is plausible under the null hypothesis. Generally, the null differences are between negative 0.2 and positive 0.2, and about a third of the differences are as or more extreme than the observed difference of -0.09. Now that we have seen a visual representation of the null distribution, let's see how a null sample can be generated in R using the mutate and sample functions. The vector of soda preferences is mixed up or permuted such that whether someone is on the east or west coast can't possibly be causing any difference in proportions. However, due to inherent natural variability, there's also no expectation that soda preferences are exactly the same for any sample. After grouping by the location variable, summarize calculates the proportion of each coast that prefers cola. Note that drink COLA produces a vector of trues and falses, which r then coerces to ones and zeros when the mean function is applied. Since A1 represents an individual who prefers COLA, the average of these ones and zeros represent the proportion of individuals who prefer cola. Summarize is used a second time to find the difference in proportion of COLA preference across the two coastal groups. The diff function is applied across the two coastal groups because the data have been summarized by location. Notice that the output gives a permuted difference of minus 0.02 as compared to the observed difference of minus 0.09. However, the permuted difference of minus 0.02 represents only one instance of the variability of soda preference under the null model. To get a sense of the degree of variability under the null model, it's necessary to permute the drink variable many times. By repeating the permuting and difference calculations five times, the permuted differences are seen to be sometimes positive, sometimes negative, sometimes close to zero, sometimes far from zero. However, five times isn't quite enough to capture all of the variability in null differences. By repeating the permutation process 100 times, the null differences are seen to range from approximately negative 0.3 to positive 0.3. Although the majority of the differences are between negative 0.1 and positive 0.1, the observed data difference of negative 0.09 doesn't seem too extreme compared to this collection of null differences. Okay, now it's your turn to practice what you've learned.
