Speaker 1: Hello, my name is Evan Ortlieb and welcome to another chat about common qualitative and quantitative data collection tools. Today we're going to get into these as they pertain to mixed methods research. So first let's have a look at different types of qualitative data. Of course, when we think about qualitative, oftentimes it comes in a narrative, non-numerical form and providing insight into experiences, behaviors, interests, attitudes, and beliefs that teachers, students, or other participants may have. Sometimes they seek out to answer questions that start off with how or why, for example, that could include interviews, focus groups, and even open-ended surveys, for example, questionnaires even. So one example of a qualitative research question could be, how do students engage in online learning activities? Another, why do students choose online learning over traditional face-to-face programs? When we think about the quantitative, on the other hand, that is in the form of numerical or ratio scale data, and that can be quantified and analyzed through the study of statistics. Here is where we draw connections between a number of factors. For example, we could say, what is the relationship between academic achievement and the number of hours spent studying? Or we could say, what are the most influential factors that predict college success? That is looking at a number of factors and even the relationship between one or more factors compared to one another. When we think about these different kinds of methods for quantitative and qualitative, sometimes creating a table to show the differences and nuances is incredibly helpful. So we think about the types of data collection tools. Sometimes they're more standardized versus semi-structured, and oftentimes that aligns itself to a more quantitative or qualitative base, as well as whether they are closed-ended in a quantitative format, such as a Likert scale, 1 to 5, 1 to 4, or they could be more open-ended in terms of trying to come up with more of an emergent theme, per se. The data formats, again, are numeric versus narrative primarily, and then the means in which we can analyze this data set often differs with respect to statistics or more of a content analysis, and there are different types that we can go into later on. And finally, the results of quantitative versus qualitative methods, one is a more descriptive data set that we're going to look at, and another is more exploratory in form. Again, the common methods that we are going to be looking at when we collect quantitative data, one example could be surveys, right? And surveys can be done from either way, but when we collect data through surveys, we can get an idea of a situation or characteristics among particular cases, and oftentimes using a standardized instrument is incredibly helpful towards validity and reliability issues. Oftentimes those standardized instruments come in a closed-ended format, and that can be administered through a number of different ways. The most common today is through an online survey. Secondly, tests, quizzes, and assessments, all these in terms of quantitative data are oftentimes standardized, hopefully. If not, then we need to do some kind of pilot testing, ensure that we're actually measuring data valid in a valid way, and that's going to assess knowledge, skills, or performance-based assessment. These are typically administered pen and paper, but they're going more and more to an online format or electronic format in nature, and programs can also create their own assessments to ensure that they're specifically targeted to their group or their participants with whom they're studying. When we think about qualitative data, on the other hand, we can look at interviews and observations and even focus groups. Those are typically four or more people in a focus group, and you can do multiple focus groups even, and then different kinds of document reviews, and that can be existing qualitative data sets, it could be anecdotal records, it could be previous information about student work samples, it could be portfolios, all these sorts of things would fall underneath document review. We also have to examine the advantages and disadvantages of each of these types of data collection. First, with respect to surveys, it can be quick and efficient, and we can get a lot of data at once, and the responses are relatively easy and straightforward to analyze. However, the disadvantages is that they might not actually be representative of the sample that we seek, and oftentimes, they lack in-depth information on a specific topic, and also, people are prone to misunderstand or not understand some questions that we may ask. When it comes to quizzes, assessments, and tests, these are standardized versions, and those are advantages, easy, again, to compare, and it's relatively objective in nature, although there is some room for discussion with respect to that, and that really goes into the disadvantage listed here at the bottom right of your screen when we talk about they may be biased or oversimplified in nature. With respect to interviews, this, you can get a pretty good response rate in comparison to surveys because it's typically a direct one-on-one kind of situation, and people are more likely to participate, in fact, if they know you, if they can relate to you on a one-to-one basis. It also allows some flexibility in questions, and you can probe and then go in other places if your questions are emergent in that form. Some of the disadvantages is that it's time-consuming to you as a researcher as well as to your participants, and sometimes it's difficult to analyze and compare data because people are so different, especially when you get them talking to you one-on-one. With focus groups, you can collect multiple perspectives in a short duration of time and oftentimes provide greater insight. However, at the same time, it can be time-consuming and requiring somebody else to act as a facilitator if you can't be there, and so, again, it's more of a time issue, and sometimes being in a group setting, as you know, whether it be a school format or work format, people sometimes are influenced by others in the group. With respect to observations, these can be viewed in real time, which is incredibly helpful. However, sometimes people only remember some aspects of the observation, right? The highlights or the lowlights are the ones that we remember most if we think about our day in general. We observe a lesson in the field, for example. Those are the things that kind of stand out, the things that are more or less normal oftentimes go underneath your radar, even if you're right there. These can also be time-consuming and difficult to observe several things going on at once without the use of multiple cameras or other types of data collection devices. And then with document review, here's where the data already exists, so it's easy to access. You can not necessarily interrupt what's going on right now. You can do this in the comfort of your own home or in the library archives or something like that, but it is time-consuming, and the data is limited to whatever already exists, you won't be creating your own data set. Some things to remember, again, we're trying to combine these by combining elements of qualitative and quantitative research. These data collection tools and strategies are very helpful as standalones, but we're trying to think about ways that we can combine these in unique pairings to be able to suit our research questions and our research agenda. So some things to remember, your evaluation purpose, questions, design, and resources, those are the things that help determine the best data collection method for your particular evaluation. There's also these common methods that we've talked about today, feel free to look back at the video there, and that each method distinctly has advantages and disadvantages. So we really want to consider those when thinking about, is a focus group perfect for this research question? Is it better or different or worse than one-on-one interviews or than surveys to teachers, for example? And finally, that a mixed methods approach to your data collection can provide the most comprehensive perspective to a program evaluation or a study in general. And so that's my perspective, I hope you have gleaned some information from this PowerPoint discussion on data collection methods across quantitative and qualitative fields. I wish you all the best in your research.