One observation proves nothing. A scientist's breakthrough fell apart when labs couldn't reproduce it, then discovered a graduate student had faked the data to advance his career. Replication and honest data are what separate real research from dead ends.
We want to look at replication and sample size. Your sample size needs to be sufficient. I can't just do this once, that baseball player who ate chicken. I've got to do it on many baseball players and eating chicken to see if it's chicken that's causing something. You want to repeat this again, multiple people. Can it be replicated? Means can someone else look at the situation that I set up and do it again? There's an old story about a scientist who who had come up with, I think it was nuclear fusion, and published a paper and said, "I can do this," and it was a huge breakthrough. Put all the conditions in the paper. Other people tried to replicate it 'cause it was a huge breakthrough and they couldn't replicate it. And he would do it again in his facility and be able to do it. And eventually he went out and tried to do it at other people's facilities. He had scheduled a visit to 3 different places and he couldn't do it at those 3 different places. So he called back to his lab, say, "I'm coming back tomorrow. There's something weird happening. We gotta figure this out because this needs to be able to be replicated." And it turns out that one of his senior graduate students was actually manipulating the results because he thought it was gonna help him get a better job. So it needs to be able to be replicated to be a valid study. Drop the word help in the comments below and I'll send you the link to my free dissertation help community.
