The Most Common Mistakes with Qualitative Research
A guide to common qualitative research pitfalls: start recruiting before IRB approval, separate research questions from interview questions, use a fixed sample size instead of chasing saturation, record and transcribe everything, and support each finding with at least two participant quotes.
Chapters
Good to go. Welcome, everyone. It's the 14th of January, and I hope you're making a lot of progress on your dissertation today. I want to talk about some questions we get about qualitative research and challenges that may arise. And there's a number of them I've written down. So if I glance off a little bit to remind myself that's just what's happening. One of the things, and this is a fairly common one, is difficulty recruiting participants, finding participants, figuring out who's going to be part of your study. Yeah, I mean, this is really important because you're typically only dealing with 10, 12, 8 something people. You want to find people that are valuable for your research. The dilemma is you have irb, reb, maybe in Canada or other places. You have institutional requirements that say you may not go out and find participants until you have approval, which means your proposal has been completed and your committee has approved it, and then the institution has said, yes, the research is not going to hurt anybody. Go ahead and start recruiting participants, which means you're well along the way. You really, really, really should be thinking about recruiting participants from the beginning and do I have access to people? And is it going to be feasible? And is there a backup place to find people? Because participants are the, you know, the guts of your research. You need them, so you want to be able to know that they're out there. You can't ask people ahead, but you can say, I'm going to be doing research and you think there are people who are interested so that people are prepared to then be officially asked. But do this from the beginning instead of waiting until you get the approval to then finally start to go find people, because then there's a lot more pressure on you and you feel a lot more stressed. How do I conduct interviews to bring about or elicit meaningful data? Well, when you design your interview questions, and there's a difference between research questions and interview questions, your research questions are your overarching guides of what you're looking to answer. Your interview questions are what you're going to ask participants in each interview. You want to be designing your interview questions so that the answers that will come about are going to allow you to get to your research questions, which means you don't just decide the day you're going to do this, the day of an interview, that, here are my research questions. You decide this well in advance and your committee should be asking for them before they approve your proposal. And then you test them on friends, family, and see what kind of answers they come up with. In fact, your committee may even want you to pretest them with participants that are similar to what you're going to be getting for your study. Because we want to know ahead of time instead of after the fact that I'm going to get the kind of information that I need. And a lot of these things that we're going to be talking about is you're going to notice that. Yeah, think about this early. Oh, do that early in the process because we don't want surprises later on down the line when it can mean going back and redoing things. So take a little time to set it up better at the beginning. And I didn't say this so far today, but if you're here and have questions, speak up. I'm ready to take them now. Data saturation is something that's confusing for people. You say, see people saying, I'm going to interview 10 people or data saturation. Well, first of all, I'd rephrase that, because if you say, I'm going to interview 10 people or data saturation, your committee can take an interpretation. Okay, you interviewed 10, but you don't have data saturation. So keep going. If you say, I'm going to interview 10 people or data saturation, whichever comes first, then you can stop at 10, even if you haven't had it. Now, what is data saturation? It's not a science. It's not, oh, if I reach here, it's automatic data saturation. It's, am I hearing the same answers over and over again? Have I gotten to, you know, two people in a row or two out of the last three have given me the same answers? I always suggest, go one more, and if you still hear the same answers, you most likely have data saturation. Now, if you've told your committee, hey, I'm going to interview 12 people or data saturation, whichever has come first, and the first two people you interview give you similar answers, they're probably not going to accept that. But if you get to seven or eight and you got similar answers all along, it's going to be much more likely that they're going to say, okay, you've made it. Go ahead and start analyzing. Yeah, Maybe you'll get new data. Correct. And that's a really good question. Which kind of goes to what I was saying. If you got the first two people and they got same answers, the committee's not going to be satisfied because you said you were going to do 12, and there's 10 more opportunities for different answers. But if you've got an eight or nine and everything is being similar. There's the odds on getting a lot of new information in the last one or two or whatever participants is much diminished. So there is a reasonability test here, but it's basically again, not a science, but it's. Yeah, I keep hearing the same thing over and over again and I'm close to the number I was going to do anyways. Therefore we'll stop there. Does that help? One way you can be sure is to eliminate the concept of data saturation. I'm going to interview 10 people, period, in my qualitative study, or I'm going to have 12 people, whatever the number is, that's what I'm going to do. And then the concept of data saturation doesn't come into play and you don't have to be thinking, oh, I might have missed something. I did what my committee approved me to do. I'm ready to move forward. Does that help? Well, data saturation never applies if you don't include it. Okay. It's typically not something part of a quantitative study. Data saturation is used in qualitative studies where you keep hearing the same answers over and over again. But if you're concerned about missing data, you don't even introduce data saturation. It. So are you taking a qualitative approach? And you know, look, if you're concerned at all about missing data, then you don't need to include data saturation as a parameter and then you never have to worry about. Okay, yeah, yeah. Now, one of the other areas, it comes in various forms, is, hey, I've interviewed these people. I want to make sure I'm not putting any bias in and, or if I'm correctly, you know, representing what they said. Well, if you are recording their interviews and then transcribing them, the opportunity for bias diminishes as opposed to just taking quick hand notes and trying to fill in. From there, it can be easier to institute some minor biases. But another technique that's used is what a lot of people call member checking. That means, hey, I recorded this, I transcribed it, I'm going to send it back to each participant to see if this fairly represents what they said and allow them to make any corrections. So this way you're making sure that the data is as clean as possible before any analysis happens. Anytime we have people, though, doing the analysis, we can all come with our own biases and we might get slightly different results coming out of it. Just know that if you are as clean as possible, trying to keep as much bias out. That should satisfy your committee that you're doing the research without bringing in bias or other things that can impact the results. You know, finally, and I've heard this one before is look, I'm having difficulty presenting what I found. Chapter four is this is what I found. Here are the themes that came about, the codes and the themes. One of the things I see people do is, and it's variations of this is here's a theme one and they say what the theme is and then they go on to theme two. But they don't provide support for theme one. It's very common for people to participant one or however they've pseudonym named the people indicated this. Participant two indicated this. You typically want to have at least two participants quotes to support a theme. And it gets a little strange if you keep seeing Participant 3 in every single one of the themes. You know, dig deeper, try to find out Participant 7 has a similar quote or something. So you're not just relying on one person and that's going to help you present your findings better. And you always go back to chapter three. What did I say I was going to do then do what you said you were going to do in chapter four. Qualitative research, great form of research if it matches your research questions and your problem and purpose and it's the appropriate way to go forward. Great. Just think about things ahead of time so that when you get there your data is good, it's clean and you're ready to go. I'm open to more questions. Well, again, I look forward to interacting with any and all of you. I'll be back here tomorrow. I hope you have a wonderful and beautiful day and keep making progress on your dissertation. Bye Bye.
