Speaker 1: Hey guys, welcome to Grad Coach TV, where we demystify and simplify the oftentimes confusing world of academic research. My name's David, and today I'm chatting to one of our trusted coaches, Alexandra, about five common mistakes students make during their qualitative research analysis. This discussion is based on one of the many, many articles over at the Grad Coach blog. So, if you'd like to learn more about qualitative research analysis, head over to gradcoach.com forward slash blog. Also, if you're looking for a helping hand with your dissertation, thesis or research project, be sure to check out our one-on-one private coaching service, where we hold your hand throughout the research journey, step-by-step. For more information and to book a free consultation, head over to gradcoach.com. Hey, Alexandra, welcome back to the CoachCast. It's really great to have you back on board.

Speaker 2: Hey, David. always a pleasure to be here and happy to talk with you today. So today we are talking about

Speaker 1: five common mistakes students make about qualitative research analysis, and let us just dive into it. The first one that comes up quite frequently is a lack of alignment between the analysis and the golden thread. Alexandra, what am I getting at with this? Yes, so this idea

Speaker 2: of the golden thread, you will hear it in all walks of research, whether it is quantitative, mixed methods and qualitative so really what you want to do and consider for this golden thread are these three fundamental we'll call them puzzle pieces of the research aims the research objectives and the research questions so these are kind of the foundation of your qualitative research study and so how you consider these and you know what you're trying to do and answer and how you're going to do it will then help you determine what methodology you should choose that would be the most appropriate or suitable to answer those questions and this is not particularly easy because there are several different kinds of qualitative methodologies out there but it can have some some positive outcomes or some negative consequences depending on which methodology you choose to answer those aims objectives and questions of your golden thread so that's

Speaker 1: really helpful alexandra maybe you can give us an example or two of where there's alignment or a

Speaker 2: lack of alignment sure so two of the most common methodologies in qualitative research that we see at grad coach or elsewhere are case studies versus grounded theory and so the first thing to keep in mind with any study is that your the methodology that you choose should be the most suitable one to answer those golden thread notions of the aims objectives research questions not the other way around and so for example with the case study the case study should be used if in your golden thread ideas of the aims objectives and research questions you already have some sort of working knowledge of a group or an event and so you're using this case study methodology because it will appropriately answer those foundational aspects of the golden thread on the other hand let's say your research aims or objectives or questions are about something that you really have limited knowledge about or there's scarce research out there and you're wanting to kind of build up a framework or a theory in that case using a methodology like grounded theory would be more suitable. So you can see there with those two examples of case study versus grounded theory, these two methodology should be applied to answer different golden thread foundational aspects.

Speaker 1: That is really helpful, Alexandra. And I know it can seem a little bit overwhelming to think about getting this alignment right. In cases like this, do not necessarily just rely on your own judgment. It can be really helpful to get a friend or someone from your cohort just to take a look through and read of what you are working on. They will be able to help you identify where there is a lack of alignment. For instance, if you ask them to sort of give you the elevator pitch back of what you set out to do, and it is not lining up with your thinking, then maybe it is a good point to sort of identify where those lack of alignments are, and use that to help you sort of address that. But try and do this earlier rather than later. It's definitely going to make your life easier. So our second mistake is making use of a transcription program software without checking the transcripts. Alexandra, why is this such a problem? Yes. So first of all, you know, there

Speaker 2: are programs, an increasing number out there that are cost effective, mostly free, and for the most part accurate things like zoom transcription software otter ai atlas ti and these certainly have a lot of benefits for convenience sake and cost effectiveness however um that's not to say that these programs are perfect because with a lot of ai and other kinds of automated software it does lose that human element that can miss some of the more nuanced or minute pieces of information that are important. So for instance, in my own dissertation research, I had about 100 participants who all verbally reacted to a stimulus. And half of my participants were doing this in English and the other half in French. And each of these were about 30 minutes long, each participant 30 minutes now with qualitative research you know you have to have something to analyze and it's difficult to do that directly from the audio files so what you have to do is transcribe these from audio to text and so i was going through and i was doing these manually myself from about participant 80 i was beyond exhausted and so i decided to use one of these outside services or programs to kind of expedite this, kind of help me. And of course it was convenient. However, when I got the transcripts back, I noticed as I was going through the first few of them, some errors to content, to spelling, different words were showing up where other words had been said actually in the audio files. And as I was going along through the rest of them, I noticed that pretty much all 20 or so of these outside transcribed files had errors. So I ended up having to go back myself regardless and going through them again and fixing them. So this is all to kind of say that even though these programs can be very convenient and cost effective, there are some drawbacks. most of that has to do with kind of content, the words that they miss, spelling, punctuation, grammar, et cetera, et cetera. And you'll oftentimes definitely actually still have to go in and check these for quality and accuracy. This is why it's very important to kind of think about, even though these programs might be convenient, they're never going to replace kind of that human element of being able to really read and understand what's going on, make sure that it matches what was said in the audio files. And so one of the things that you can do if it's not yourself, you should check it yourself, but even go beyond that and ask someone else to check these transcripts for accuracy. Because either if you've used an outside service or program, or if you've done all the transcriptions by yourself, sometimes we miss things. Having someone else, an outside person, an actual person look at these and kind of make sure that they're accurate will not only help you catch potential errors, but in doing so, it kind of promotes the credibility of the transcripts because they're accurate, they're clear, they're actually what was said in the audio files and so sometimes what might be happen if you don't do this having that like human element it can diminish the credibility of the rest of your transcripts if they are accurate because the reader or your marker might say well this one was not accurate so maybe there's some flaws in the other ones as well but beyond that I mean other than the marking your transcriptions this is your this is really your raw data in qualitative analysis and so if you have errors or missing information in your transcripts that were there in the audio files this makes the coding and analysis flawed this puts things in misalignment and as such there's kind of a domino effect of repercussions that can happen if these things aren't transcribed

Speaker 1: accurately. I think that in the same way that in quantitative research your actual data is key to your analysis, it is the same for qualitative. So we really want to make sure we are doing due diligence to assess the quality of the work. That is not to say you cannot use services to help out. It will depend on your type of research as well. For instance, from a business perspective, you might be less interested in the specific nuance of how someone presented an idea compared to a language study. So in cases like that, there is a bit of a cost benefit to consider, but regardless of whether you are using a service or not, getting a second run through of it can be super helpful. And there are a range of services out there that you can use, both in terms of software or human run services. If you are interested in it, we even do it here at Grad Coach. So do take a look for the link down below. So our third mistake that frequently comes up is not specifying what type of coding you are doing in advance of actually jumping into the analysis. Alexandra, why do we need to be aware of what coding type we are using so early in the process?

Speaker 2: This goes back to the idea of making sure that all steps of your research align with the previous one and are justifiable in terms of it makes sense. There's a reason why you're doing what you're doing in the order that you're doing it. And coding is no exception to this. So the reason why coding is so important in qualitative research is that qualitative research is inherently kind of subjective. There is this inherent human interpretation that can happen. And so one of the reasons why it is so important to do coding appropriately is to kind of add the systematicity and the academic rigor to your research that is inherently not there. And so to kind of ensure this increased objectivity of something that is inherently subjective, doing this coding, you need to consider which kind of coding will be the most appropriate to answer your research goals that you've outlined prior, going back to that notion of the golden thread. And coding inherently kind of falls into two camps. There is inductive coding and deductive coding. So on the one hand, inductive coding is an approach where you are going into your data analysis and you are kind of, you're letting the themes and the codes emerge from the data. You don't have any preconceived notions, no existing ideas of what to expect. You're really letting the data, whether it comes from interviews or focus groups, you're letting the data from those transcripts emerge into these codes. And this is best for studies such as grounded theory approaches where you don't really have any idea of what to expect or anticipate. And you're really kind of trying to explore what is out there. You're letting these codes emerge directly from the data. On the other hand, deductive coding is another coding approach where you are actually, you have some ideas about what is out there, what you're looking for, what you hope your final findings to be. And for this coding approach, it's top down where prior to even collect the data, the interviews, focus groups what have you you have developed an initial set of codes into a code book whether you've put this in say Microsoft Excel or Microsoft Word or Google Sheets etc and you have kind of looked through the existing literature on your research topic and seen what what are the potential codes out there what are the themes you're looking for And then once you have collected your data and transcribed it, you're assigning pieces of that data to those codes that you've already created in advance. And you are not looking for new codes to emerge like you did in inductive. So all codes should go into something from your codebook.

Speaker 1: I think deductive coding is most commonly used where you have a theoretical framework that you're working within or a field that is really, really well researched. There, you're not going to be starting something new. Similarly, it's also become really popular to use a mixed approach of inductive and deductive. This is primarily starting deductively with a codebook and using that codebook to lead your coding and then develop further from that with an inductive approach. It is worth noting this is a fairly new way to go about coding, and so it is important that if you are choosing to go this way, that you can justify why it is appropriate and why it is useful relative to that golden thread, those research aims, objectives, and questions. Because you You don't want to be overcomplicating things or stepping too far out of your comfort zone just because it's novel. Rather, make sure it is what you need to do, where you need to do it.

Speaker 2: That's great advice, because sometimes as graduate students, we have this urge to do something novel or do it a different way. And that should not be your motivation or your justification to do something. So even though this this kind of new way is developing and coming and becoming increasingly popular, that doesn't mean that it's right for your study. So how you know it's right for your study is going back to that notion of the golden thread. And this idea extends even beyond inductive and deductive coding, because those are kind of your your starting idea of how you're going to code. Beyond that, there are additional specific approaches that you will use for your initial or your first set of coding versus your second set of coding. As an aside here, you should absolutely do more than one round of coding. Again, this will increase the systematicity, the rigor, and kind of the credibility, so to speak, of your data analysis. and so there are many different specific coding approaches but some of the the most common ones we'll name here are starting with your open coding and so for this one this kind of approach it's very loose it's very tentative as indicated from its name it's open and so this is more suitable when you're starting out other common approaches are things like in vivo coding and so with in vivo coding, this is actually using the participants own words in your analysis, not putting your interpretation of what they said or suggesting what they meant, but actually letting the participants own words do the talking, so to speak. And so this is typically most suitable to things where you're really interested in the perspectives or points of view or experiences of your participants and then the last one we'll mention but there are still plenty more is structural coding and so we use structural coding specifically well not specifically but commonly in cases where you say have conducted an interview or focus group discussion and you want to use those questions that you posed in the interview or the focus group kind of as headings all of the codes that go under one specific column for instance should be related to one specific question that was asked in the data collection and so this is really best if you are kind of looking for specific answers or codes or themes in response to one of your interview questions so or focus group questions so again there are still plenty more out there but these are some of the more common coding approaches.

Speaker 1: That's really helpful, Alexandra. And it can feel a little overwhelming that there are so many options to choose from. Don't worry, there are a ton of resources out there. Definitely take a look at any of your methodological textbooks from a qualitative perspective. You can take a look at methodology papers that have been published, YouTube tutorials, blog posts, you name it, it's out there. We even have some videos and some content about coding as well on the Grad Coach blog. Links to that will be down in the description below. But importantly, when you are considering these coding decisions, it is important to realize again what you are using them for. So look for that alignment, make sure it is on track, and then it will flow much smoother going forward as well. So our fourth common mistake is students downplay the importance of organization during both coding and analysis. How important is organization, Alexandra? It is so important. The reason why

Speaker 2: this is so important is that oftentimes we kind of assume that qualitative research and qualitative data cannot be structured. Of course, it's not as black and white or objective as quantitative research. And so what you need to do as a qualitative research is to kind of apply a framework that yourself that will promote this kind of objectivity, systematicity. And part of this relies on organization. And organization is important not only for the coding, but also the analysis. So part of the difficulty, but the importance of organizing is that sometimes the codes that you end up with after you've transcribed and done your, let's say, initial round of coding, you can end up with very high numbers of codes. For instance, I've seen some where it's upwards of 1000 codes. And so this number is very overwhelming, very large. and some of the ways to tackle this large amount of codes is one to make sure that you're organizing all of your codes in a spreadsheet of sorts whether it's excel or google sheets having them all in one place will then further facilitate you doing additional rounds of coding which we recommended previously and in doing so having these additional rounds of coding on your codes that are organized in one place, it will help you kind of whittle down these codes to the point where you have the codes that you need. There's none that are kind of superfluous or repeated, but it's very important to keep these organized in one place and to go through multiple rounds of coding. And this will make your life a whole lot easier and make sure that you have only the

Speaker 1: codes that you need and can justify. I think that's super helpful. It's also worth emphasizing that coding and organization it's a back and forth you're going to be moving from one to the next and back again and that's a good thing to do it enriches your analysis but it also allows your organization to inform your coding and your coding to inform your organizational structure and through that iterative process you're really going to develop the analysis so don't think I've coded it once, I'm done and dusted. Sorry to say it's a multiple approach. In terms of organization helping analysis, Alexandra, why is it also important to keep a track in that Google document

Speaker 2: or sheet of all your codes? Yeah, so this goes back to that notion we've repeated several times of the golden thread. So if you think of dominoes, for instance, you need to have your dominoes set up in such a way that if you knock one down, the rest go down. We can think of that, our qualitative research in such a way. And so if in the coding stage, everything has aligned with that golden thread and we move on to the analysis, the analysis will be further aligned with the coding, the transcription, the data collection, going back to the research questions, aims and objectives. And so having our codes organized in a sheet will then allow us to start to analyze our codes in a way that we can see themes and patterns emerging that are aligned with the codes, which will then add this rigor and systematicity of your study by having analysis that you know is based on very organized, solid foundations of your coding and your transcription. And so through this analysis, if we have our analysis organized, we can keep track of our patterns, our themes, and then going beyond that, actually, when we get to the point where we're writing our findings chapter, we have this set organization that will then kind of allow us to know how we're going to present these results because everything has been organized and justified up to that point.

Speaker 1: I think that's really helpful. It's also worth noting that having your codebook organized can be really helpful in sort of preventing you from getting stuck with your analysis or feeling like you're unsure of how to code because, you know, things are feeling uncertain. If you have an Excel sheet that you've developed before you start your coding process, you have it organized by the different rounds and you start bringing it from a large number of codes to the specific codes you are going to be using, that organization really helps make that process move forward. And it can be kind of cathartic to really work through that process, get it from a hundred transcripts of 30 minutes each down to some key findings. So our fifth and final mistake that we're covering today is not considering your researcher influence on your analysis. Alexandra, how do we affect our analysis and why is this something that we need to even think about?

Speaker 2: Yeah, so this kind of just goes back to the innate nature of qualitative research. It relies a lot on interpretation. It is subjective. It's not inherently black and white, such as quantitative research. And so the ways that this is kind of mitigated is through things like positionality and reflexivity. So these two concepts are becoming much more prominent and required in qualitative dissertations and theses. And so what these essentially mean is that you have your positionality, which are the underlying kind of beliefs, judgments, opinions, perceptions, all of those things that kind of make you you, the human elements. And so the way that you think about things might be different than the way someone else thinks about them. And so why we need to state our positionality in qualitative research is that it can impact our interpretation of the data, which then impacts the findings. And so, for example, in an example study where someone is exploring the perceptions of the tech industry of men versus women, a researcher who kind of identifies as a feminist versus one who identifies as more conservative or traditional, they might have underlying beliefs or assumptions about gender when it comes to the workplace or just in general. and so acknowledging that that you have these kind of underlying preferences or perspectives what have you it's important to acknowledge that because like i said it can have consequences for your analysis and your findings taking this a step further typically now we also have to to talk about our reflexivity in qualitative research and so essentially what this refers to is how our positionality affects our kind of interpretation so whereas positionality has to do more with the underlying assumptions reflexivity is taking those underlying assumptions and acknowledging how they might actually impact our interpretation and our findings and so the reason oftentimes why these are required now in qualitative studies is that this idea of you know validity and reliability we don't really use those in qualitative research we use more of these ideas of trustworthiness and that connects to our positionality and our reflexivity this reflexivity how it can impact you know it can impact the coding of your data the themes that you pull from the coding how you interpret it how you present it so in my example of the researcher who has more feminist underlying beliefs versus more traditional conservatives even if they're exploring the same phenomenon they can have vastly different interpretations and so acknowledging your positionality and indicating with your reflexivity how it might impact those steps of the research analysis can lend more credibility and more kind of trustworthiness to your your

Speaker 1: findings and ultimately your study. So that's really helpful to think about these aspects because we do need to consider how our positionality and our reflexivity might affect how we proceed with our analysis. There are potential opportunities for bias and if we're engaging in these behaviors we are able to a mitigate them during the analysis and in cases where you cannot mitigate it you can at least acknowledge it so other researchers can interpret that going forward but bias goes a little bit beyond just your positionality and reflexivity so Alexandra what other biases can come up because of research effect yes this idea of bias so going

Speaker 2: further beyond positionality and reflexivity it can be very easy to have biased interpretations and there are a few ways this can manifest so for instance spending too much time presenting the the findings from one particular participant in your study and neglecting those of the others and so one reason why this might happen is either you as the researcher totally agree personally with their perspective or even totally disagree and you want to to present that in um in some for some sort of reason um so it's very important to kind of mitigate that bias by presenting a balanced approach of all participants on the other hand there's also things like spending a lot of time presenting on one particular theme that emerged from your qualitative analysis and you know, kind of avoiding or neglecting the other ones. So this can happen where you found a theme that emerged from your analysis that was particularly interesting to you, whether it was novel, whether it confirmed what you thought, or even aligned with your personal beliefs. It's very important to make sure that you are giving enough attention to all the different themes that have emerged. And a third common bias that we see is that sometimes it can be easy to make claims or assumptions such as this means that or people should do this. So for instance, in my example of the tech industry and gender norms, making claims in your writing such as women in the tech industry felt that, or the way that the women in the tech industry talked means that, or the tech industry should do that. So making those kinds of grand sweeping claims that your qualitative findings mean some sort of big, big thing. We really have to try to avoid that in qualitative writing, despite it being tempting, especially if it aligns with our personal perspective. So, those are some common biases we see.

Speaker 1: I think that is super helpful to think through, particularly because biases are inherent to us. So, it is important to take that step back, to think about how you might interpret, interact with things, and then engage with that. One way to really go back to this is take a look at the data. We do not want to be making statements or assumptions that do not have support in the data. that is just gonna undermine your argument and your position as the researcher. So wherever possible, if you don't have data to support it, maybe consider not including it. If you do have data to support it, maybe just confirm with a second opinion, your supervisor or someone else, just to make sure that there's not bias coming in. But I think the most important part here is to think about the fact that we do have biases. And so as long as we're considering this, we're doing our due diligence as researchers.

Speaker 2: Yeah, and so one of the ways that you can also make sure that you are kind of following what you said you were going to do from the get-go is not to step out of your codes and your themes that you've established. The reason why this might be tempting to do, again, is going back to that fact that maybe you found something super interesting to you and you want to present it. What I would caution you towards is making sure that any findings that you're presenting fit or align with what your objectives, aims, and research questions were. Another reason why this might happen is because the dissertation or the thesis is such a long process, sometimes we can kind of get away from our original intent of our study. And so presenting these things that are outside of our codes or our themes, we think we can get away with but in reality this kind of minimizes the the rigor of of your findings and so even though you might find something very interesting like you said David be really careful make sure that you're still kind of staying within your codes within your themes and following that golden thread that you've been establishing throughout yeah you've

Speaker 1: probably heard it so much today but golden thread is key we want to make sure that we're maintaining alignment with our research. It is only going to improve the impact. So Alexandra, thank you so much for joining us today. It has been really great. There are some great insights here and thank you again for joining us on the CoachCasts. Always a pleasure, David. Thanks so much for

Speaker 2: having me and letting me kind of chat about these qualitative foibles.

Speaker 1: Alright, so that pretty much wraps up this episode of Grad Coach TV. Remember, if you are looking for more information about qualitative research analysis, be sure to check out our blog at gradcoach.com forward slash blog. There you can also get access to our free dissertation and thesis writing mini course that'll give you all the information you need to get started with your research journey. Also, if you're looking for a helping hand with your dissertation, thesis or research project, be sure to check out our one-on-one private coaching service where you can work with one of our friendly coaches, just like Alexandra. For more information and to book a free consultation, head over to gradcoach.com.