R Tutorial : Experimental Design in R

Hi, my name is Kaylin Medeiros and I'm a data scientist who works in the technology and health industries. I'm here to teach you about experimental design in R. An experiment starts with a question. The experiment involves collecting data with the question in mind and will include analyzing the data to seek an answer. In this course, we'll focus on asking good questions in statistical language, formulating clear hypotheses, design the data collection process, and the analysis of collected data. The three high level steps of an experiment are planning, design and analysis. For planning, you start with your hypothesis, your question, or even a series of questions. What are you hoping to answer? What is the population of interest? Those to whom it applies? What will your dependent variable be? The outcome which hopefully can be measured. To answer the question, what are your independent or explanatory variables? The variables you think may explain the dependent variable design questions may naturally follow from planning questions. Choosing a design might entail knowing you want to study different variables and the possible interaction effects of those variables. So you choose a factorial design. Then, if your dependent variable will be a yes, no answer, you know you're going to be dealing with some kind of logistic regression when you get to analysis. We're using open data in this course, so we don't know the original experimental design, but that's okay. The data we'll use throughout the course has been cleaned and altered by me as if it was collected as part of our experiments. Here's an example timeline for an experiment I designed at a past job. There's no standard for timelines. However, the three key components of an experiment include randomization, replication and blocking. All of these are done to assess variability across our study population, meaning we're looking to explain the variation in an outcome by the different explanatory variables. In order to keep bias low, we need to randomize, create a replicable experiment and sometimes block. Say we want to test who can score the most free throws 5 minutes. We select as one group the high school basketball team and for the other, an English class. This is an incredibly biased experiment right out of the gate. Any conclusions we draw will be based on our poor selection of groups and lack of randomization. Randomization is a key tenet of any experiment. Randomization helps ensure that variability in outcome due to outside factors that we that we're not studying in an experiment are evenly distributed among treatment groups. One example of randomization is double blind medical trials where neither the patient nor the researcher knows if the patient is receiving treatment or not. The patient is randomized by a third party into one of the two groups. Replication is the idea that we need to repeat our experiment in order to assess what variability looks like. We certainly can't analyze data if we test a drug's efficacy on only one patient. Maybe it worked for them, but how do we know if it will also work for other people? Our experiment must be designed to be easily replicable to get a wide range of outcomes. Blocking is a technique to help control variability. A classic example is to test the effects of a drug on male and female patients blocking by sex to account for treatment variability, in this case known differences in the drug reactions of male and female bodies. Now let's do some example exercises to demonstrate all three of these key experimental design concepts.