Dissertation Data Collection: A Full Guide

In order to do a dissertation or research really of any kind, you have to collect data. That does not mean always that you're collecting numbers. It could be that you're collecting people's thoughts or words. And sometimes the collecting of data is things like content analysis or analysis of things that have been written by others, whether it be novels or newspaper publications about politics, things like that. But somewhere, somehow, you've got to come up with data. Because in research, research, we're basically creating new knowledge. And since we're creating new knowledge, it means we have to take something, synthesize it, analyze it, however we're doing it in order to get those results out there. So when we're talking about data collection, it depends upon your method that you're going to be using and your method just to go before data collection is going to be based on the problem you're researching. But ultimately there are two in my mind methods, broad methods, and a third, which is a combination. So there's quantitative research, and that means looking at numbers. How did the stock market perform? How did people's blood pressure change? How did education test scores change? We're looking at numbers and analyzing those. Then there's qualitative, which is getting to people's perceptions, people's lived experience. Typically, those are going to be interviews of people. And then the. The third method becomes known as mixed methods, and that's using both a quantitative approach and a qualitative approach. No one approach is better than the other. No one approach is easier. I will say sometimes mixed methods can take longer because you're doing two different studies in effect, but you really want to find out the method that is going to allow you to answer your questions, your research questions. So if you're in your research questions, you're looking for statistical significance, you're doing a quantitative study. In your research methods, if you're looking for themes, you're doing a qualitative study. So where do we collect data? How do we get data? Well, again, first, what type of research am I doing? Quantitative, Qualitative. And I want to think about this before I do my study, before I get my proposal into my committee. I want to be thinking, do I have access to data? Because I've been seeing too many people over the years who move all the way through their first three chapters and then haven't thought about how they're going to get data or where they're going to get data from. And that's really, really important because now you have to scramble and figure it all out. If you Figure it out ahead of time. You're much better off is you don't want to get to the point of having three chapters written and then find out, oh, I can't get the data. I got to go back to my committee. I've got to go back to the irb. So that's really important. So where am I going to get my data from? Do I know that it's already there? Do I know people to contact? Am I going to go through Facebook groups or industry groups? Or do I have a contact at a company that's going to get me the data? Do I know people I'm going to use in my study? And this can be a little bit tricky, especially if you're working for a company or you're an employer and you're using subordinates. Then your committee might say, well, you have power over them and they may not be answering honestly. So you want to be aware of things like that. I'm going to start with qualitative research right now. If I'm in qualitative research, I need people. So let's say I've found my people. Either it's through a Facebook group, it's through contacts, I know I've sampled these people. I may do snowball sampling where it says, I found one person and I asked them to find me another person, and then they find me another person. But ultimately it's going to break down to an interview. How am I going to get their permission? Most universities are going to want to have a permission slip from them or endemic, acknowledging that it's okay to participate. Am I going to interview everybody under the same condition? If I don't, I've interviewed four in person and three on Zoom and two using Microsoft Teams. Your committee starts to wonder about the legitimacy or the consistency of collecting your data. So you'll want to try to collect data under the same conditions as everyone else. I'm going to record these interviews, most likely might take some notes, too, but it's hard to take complete notes while you're there. So I'm going to record them. How am I going to record this information? Am I going to use my phone? Am I going to have a recorder? Am I going to have software do the recordings of my computer on where I am? If I'm going to use my phone or my computer, am I going to have access to the Internet or is there a cell phone reception there? These are things to check out ahead of time. And it's always good to have a backup just in case something goes Wrong in the middle. You're 80% of the way through with an interview and then you lose it or you lose the ability to complete it. Having a manual recorder or something is a good thing to have as a backup. Now I've collected my data, I'm going to have to transcribe it, turn it into a document that can be analyzed. So if I'm going to transcribe things, there's a few ways to do that. I can do it the old fashioned way and do it myself, and listen and write it down, and go back and listen and write it down, and go back and listen and write it down. Saves you money perhaps, but it can take hours to do this yourself. Just be aware of that. Or I hire somebody and they have a system where they listen to it, write it down, listen to it, write it down. It's going to take them a long time too, and it's going to cost you usually a decent amount of money to have them do that. And you're still probably going to go back through and check on them. Or today with technology there are things like Otter O T T E R A I or Reverend and others that will take your document your recordings and turn them into a transcript. They're not perfect, but in my experience they've been over 90% and you're still gonna have to go back through and check on them. So now you have a transcript. What a number of schools and committees like is now you want to send it back to the people you've interviewed and do what some people call member checking, which is, hey, here's what I recorded. Did we get this accurate? Now one of the tricks you want to use here is, here's what we recorded. Do you want to check on it? And if I don't hear from you, buy and give them a date, maybe a week later, we'll just assume it's, you're happy with it. If you don't give them that, they may send something back six months later and then, and if there's a change, then you've got some issues. But if you put a time limit on it, that allows you to move forward at a decent pace. So you've got your document, you've got everything transcribed. Now time to analyze. Well, many people use Braun and Clark's six steps. Familiarize yourself with the data, generate code, generate themes, review the themes, name the themes and start writing things up. You can do this, you can do it manually, which gives you a great familiarity with things, but it also takes a bunch of time you can use software and there's a bunch of them that are out there in no order. Atlas, ti, Quirkos, nvivo, Deduce, maxqda. That's a number of them. They all work a little bit, bit differently. I will say. There's hundreds of videos online with YouTube on how to do this analysis with them. The best one to use is the one your school provides you for free. And you most likely have taken a methodology or research course in which they've asked you to use that software. So think about that. And I'd be remiss. Otter and rev that do the transcription thing. Sometimes they charge for this, sometimes it's free, depending on how much you're putting in there. But gosh, Otter, last time I checked was like $20 for a month for unlimited usage, something like that. So you can get everything transcribed for $20. Something along that line. Okay. There's also something we've come across recently called qualitative IO and they will do the analysis for you of developing themes. No matter if somebody else does it for you though, you have to be ready to answer questions on it. If your committee asks you get into your defense and your committee say, well, how did you develop themes here? Or what themes came about or what process did you use? You can't say, oh, qualitative I o did it. I don't know. It's your work you have to be ready for. That's a quick run through of qualitative data collection and analysis. Before I run into quantitative, I guess I'll pause for a moment to see if there's any questions. So you're going to have. That's a really good question. You're going to have research questions, but those research questions are going to be supported by interview questions. And your interview questions should be such that they're open ended. You don't want to ask. Do you agree that Steve Tippins is the best looking dissertation chair you've ever seen? The answer to that is either yes or no. We all know it's yes, but it's either yes or no that you can't get much more information if you ask questions that have yes or no answers. So you want to have open ended questions. You also want to be able to ask further questions. What do you mean by that? Are you explaining it that this is what you're asking so you can ask open, open ended questions followed with follow ups that allow you to get more clarity and more data. Setting up a schedule. Sometimes it's a little outside of your control. In that you want to get as many, you know, your data collected. I need 12 responses, let's say, and I've budgeted that, you know, Mondays and Thursdays are the best days for me. And you get people responding saying, hey, I can do it on Wednesdays. So you may have to be very flexible in your scheduling. But if you're not available at 2 in the morning ever, just you can't do it at 2 in the morning. So you find other people. You want to set up a schedule where you are as fresh as possible. Meaning I don't want to schedule nine interviews in a row because by the eighth and ninth person you may not be able to ask those probing questions. And those probing questions come about from having open ended questions and a good knowledge of what you're looking for. If you're going deeper and deeper and deeper with someone, you better make sure you've gone deeper and deeper with almost everybody or everybody on those topics as well. Because you don't want just one person defining everything for you. So setting up a schedule that works for you and planning ahead on that schedule is really important. So when you're preparing your proposal and you get to chapter three, your methodology or the typical chapter three, you set that up where you're going to talk about every single step you're going to use and every tool you're going to use. So you're going to say and I'm going to transcribe this using otter AI or REV or whatever it is and then I'm going to use NVIVO or maxqda or Querc, whatever to analyze the theme. So you put it as part of your methodology. So your committee knows everything you're going to do and every tool that you're going to use. Your committee is going to approve your research questions as well as your interview questions. When you have gotten them to answer your interview questions and fully answer them and you've got good interview questions, that's when your interview is complete. If you think you're going to interview people and 10 minutes, you're probably wrong. You have to think about an interview being probably 40 minutes or so, 10 minute interviews, your committee is going to be suspect that you're collecting enough data. I also wouldn't think about having a four hour interview because it's going to be really hard to get people to commit four hours setting things up. So you're going to get everything you need starts with your research questions. I got these research questions or research question and now I've got my interview questions. And before I even go out, will my interview questions provide me the information I need, the answers I need to be able to answer those research questions. If they won't, then you need more or new interview questions. So it's being prepared ahead of time allows you to then complete and answer your research questions, because that's the purpose of your research, is to get to those research. Research questions. Now, quantitative research numbers, and I've seen too many people say, well, I can't, I hate numbers, I hate math, whatever. I'm not going to do quantitative research. You know, neither path, as I've said earlier, is easier. They all have, you know, work you're going to do. But quantitative research has a research question that says, is there a significant, to what degree? Is there a significant relationship? Things like that. So now I have to determine. I need to find data to do that. Secondary data. What that means is if I'm looking at, for example, stock market prices, someone has collected stock market prices, someone has collected student scores on SATs, someone has collected data like this, and there are sources of this data, and that's what's known as secondary data. If you're going for secondary data to answer your questions, first, is it available? Okay. Second, it makes your IRB approval a lot faster because there's no people involved and the IRB's purpose is to protect people so they will give many times exempted or expedited reviews. When you're using secondary data. If I'm using secondary data, stock market, SATs, GDP, whatever, where there's sources, government organizations have lots of data. We want to make sure it's collected on the same basis. By that, I mean, hey, let's look at reading scores in second graders. Already there. Great. And I want to look at it over 20 years. Well, did they collect it all 20 years, or did they change the method of how they collected the data? Or in four of the years did they do second and third graders? I need that data to be consistent. So. So those are things we want to be looking at in the collection of secondary data. If I'm looking at primary data, it's typically going to be something like a survey. I'm going to send this out electronically, I'm going to send it out manually. Going to give people paper and they're going to fill it in on the paper. However that's going to be done, I have to get it to them. Most committees, if you're going to give a survey, they want what's known as a validated instrument, meaning an instrument that others have used before and the data collected from it is going to be data that they think is good data. So I've got an instrument that's going to collect that or I've got to create a survey and do pretests with other people and validate that myself, which is more work. But I found a survey or surveys that I'm going to have people complete now, just like in qualitative research, I got to go find people. You may again, industry sources, Facebook groups. You may have natural ways in there. There are services that will help you collect data. SurveyMonkey can, you know, they'll create a survey. You can use that their system to create a survey and collect all the data data. But they will also, for a fee, go find people for you. They are a little less expensive than the one I'm going to talk about. But you're doing. They also aren't as guaranteed don't provide as many guarantees for finding people. Qualtrex is the other one and they may charge you several thousand dollars. But if they have the data, they will get you the exact people to respond and the number that you need. Now the number that you need need comes down to what's known as a G power test. There's a number of calculations that go into it, but that will come up with based on what you're looking for and the population. Here's the number of responses you need and it might be 142. So you need to go collect 142 data points and then you're able to do your research or complete your research and your analysis. So I've got my people, either secondary data or people to survey. I send this information out to them and they respond. I have found that if you're dealing with people, if they haven't responded in maybe four days, they may not respond. You may have to send things out again or find other groups of people. That's why again, prepare early, know about access early. As an aside, if you're dealing with the military and collecting data, even though they've told you they're going to do it, commands change and it's difficult at times to get them to provide data. They even have said they're going to provide. So you have to think about that in your timing. I collect all my numbers now I have to look at that data. Does it look right? Hey, I got my 142 responses, but nine people didn't answer all the questions. So that means I need nine more surveys. I need 142 complete surveys. So you usually shoot for more. Did they answer cognitively? Did they think about their answers? You're going to find. Now I've had a number of clients who've had issues putting something out on Facebook. And they'll get 700 people respond, of which maybe 18 are real people responding and providing, you know, saying yeah, I want to do it. So I've gotten all my data. Is it good data? Well, that's why we do scatter plots and see if things are normally distributed. So we look at the data to see does, is it reasonable data? We want to find out the distribution of the data because the next step in the process is the analysis. And the analysis means doing some sort of statistical test. And doing statistical tests. One of the great questions that they're going to ask in your defense is oh, did you meet the underlying assumptions for this regression analysis? Meaning was the data normally distributed? Was it? And you know, every statistical test has underlying assumptions. So you want to make sure that the data you have meets the underlying assumptions or can be transformed into meeting the underlying assumptions. Now by that I mean there are a number of test regression, for example, that one of the underlying assumptions is the data is normally distributed. But maybe your data isn't normally distributed, but if you take the log of that data, it's normally distributed, then, well, then you're going to do a logistic regression test. So it's understanding what you have or if you're violating the underlying assumptions, then you find tests that are more powerful. Parametric tests are really powerful, but if their underlying assumptions aren't met, then we go to non parametric tests which are more powerful for their types of assumptions. So this is all part of our analysis of data. When we have it now I've got my data already, I understand what tests I'm going to use now. I plug it in, I analyze the data. Many people use software to do this. You can do it by hand, but it's a lot of work. I can use spss, Statistical Package for Social Sciences, which has been around forever and most schools will support that. So you've got it available. If it's not available, there are other packages as well, but this is probably the most popular one you're going to run into. But there's also jasp, J A S P, which is an acronym for Jeffrey's Always Smart Program, something like that, or R or jamovi. These are packages that are out there to help you. Again, what is your school support? What have you already used? Now whether you're doing a qualitative test or qualitative study or a quantitative study. I always suggest that and pay close attention to what I'm saying. You make up data not to use in your study ahead of time, make up a response to a survey and two of them, let's say, and let's find some secondary data that's made up or interviews that have been made up and plug them into the software you're going to use not for any results but so you can figure it out how to oh, what do I do next? What step is followed, what happens here? And then you, you can make mistakes when it's free, when you're not trying to analyze real data, but if you're doing it with made up data, it doesn't matter, you know, if you make a mistake, you learn from that so that when you have the real data, you plug it in and you're ready to run. And then mixed methods, you're going to do both quantitative and qualitative. Let's talk about first of all the nervousness. Doing this ahead of time with people who aren't part of your study will allow you to become much more comfortable with asking questions and to be able to probe into what they're saying and keeping an open mind and listening to what they're saying, not trying to, you know, making sure this is recorded so that you can have the time to listen to what they're saying and really be able to ask questions from there. You can always, if you're doing member checking, go back for clarification. But it's best to do it during your interview. So don't develop the confidence ahead of time that's going to help you when you're in an interview and you have those well thought out interview questions because you don't want to get into interview and not having thought out your interview questions and then having to be make them up or continue them on their own or somebody comes along five interviews in and says oh well that's an interesting, I should have asked this question, let's do this. Well now I got to get back to everybody else. So doing some testing ahead of time is really going to help you get the data you need and be more confident in the data collection.