Speaker 1: So, I've been working as a research and evaluation expert for a large state government based in Australia, and in my role as an evaluation expert, I create lots of evaluation reports and consult with clients on matters related to research and evaluation. So before I undertake an evaluation project, it's really important for myself to create an overall mind map of the whole evaluation project. So before jumping into any evaluation project, I create a document called an evaluation framework. So in this video, I'm going to tell you what is an evaluation framework and what are the various components of an evaluation framework. So an evaluation framework is more like a mind map of an evaluation project. Now imagine an evaluation project, it's not an easy task. It's a long-term task. Sometimes it takes months and years of time to create and do all of the activities involved into the evaluation. Now at the beginning, if you look into the whole evaluation project, it's really important that you create that structure of that evaluation. So the evaluation framework will probably give you the structure, or in other sense, it's more like a sketch map of the overall evaluation project. So the evaluation framework, although you create at the beginning, it's a live document and it can transition time to time, but it's really important that you have a very succinct and clearly defined summarized format of the whole evaluation. And that is where the evaluation framework comes in. So to begin with in the evaluation framework, the first thing that you need at the beginning is to have a bit of background. So you start with a background about the evaluation project. So what's the overarching management aspiration behind this evaluation project? So you underpin that at the introductory section or the background section of your evaluation framework. Not too long, but in a very summarized format, you clearly underpin what are the main objectives of this evaluation project. The next part is really very crucial. So this is the part where you underpin the question and objective of the evaluation. So what exactly is that you're going to evaluate? You need to very succinctly create an evaluation question. Typically the evaluation question should be expressed in a sentence or two so that it's easy to understand. So anyone, whoever looks into the evaluation question would understand, all right, so this is what is going to be measured through this evaluation. And then it's also a good idea to break down the question into several objectives. Now for creating objectives, you should always keep in mind the SMART principle. So the SMART principle will guide your understanding and that how can you specifically derive some of the core objectives or plan from your main evaluation question. Then it's really important to understand what is the evaluation type. See, evaluation could be a summative or formative types. And within this broad two classification, there are many other types like process evaluation, impact evaluation, economic evaluation. So after consulting with your clients, you have to underpin which type of evaluation project this is going to be. And you need to very clearly underpin that at the beginning in your evaluation framework. So once you know what is the type of your evaluation, that makes it a lot easier to go through the next steps. So the next step in your evaluation framework would be to think about the evaluation design. Now for evaluation design, typically it will follow the same principle as research design. So in research design, there are three different type of designs. One is a non-experimental design, the other one is quasi-experimental, and the third one is experimental design. So evaluation also follows the same three design principles as well. So the non-experimental design is typically explorative in nature, where you will find out everything based on qualitative measure. The quasi-experimental design, whereas, involves both non-experimental design and randomized control trial, where you will have control group and intervention group. And the experimental design is purely based on randomized control trial. So in randomized control trial, you will have two groups. One is the control group, and the other one is the intervention group. And then you'll see the performance of these two groups and weigh between and compare, and then you'll derive the outcome. So sometimes the experimental design is also known as the gold standard of evaluation. So depending on the project, however, you have to determine which type of design would best suit the need of this evaluation, and then you have to underpin your evaluation design. So after working out the evaluation design, it makes it very easy to talk about the methodology of your evaluation, whether you're going to use qualitative or quantitative. For example, if you're using a non-experimental design, so the choice is simple, you'll simply use a qualitative methodology where everything will be interview-based or focus group or observational type, whereas if it's a mixture of quasi-experimental or experimental design, then you will have both qualitative and quantitative methodology. So depending on your evaluation design, then think about which type of methodology you are going to use part of your evaluation project. And your evaluation methodology will then guide your data collection process. So in data collection process, you simply highlight which type of data collection method that you're going to use, whether it's going to be interview, focus group, or surveys or observational techniques, or any other type of data collection method. So you need to just list down those methods that you're going to use to collect data from your intervention group. In the evaluation framework, it's always a good idea to highlight the data analysis technique that you're going to use. For instance, if you're using a qualitative methodology, then your choice is simple, you're going to do a thematic analysis or a summary analysis of the qualitative data. But if you're doing a quantitative methodology where you'll be collecting data through surveys, then you'll probably have to do some statistical analysis. So which type of analysis that you're going to do once you have collected the data, that you should also highlight in your evaluation framework. Typically, the evaluation framework document should be very summarized and short, not exceeding more than three to four pages. But in the evaluation framework, it's always a good idea to have a work plan of the overall evaluation project. So this is more like a work breakdown structure where you list all of the activities and the responsibilities and also the date for each of the specific tasks. So that will then allocate the task according to who is responsible and also the corresponding dates or the deadlines for each of the tasks. So thus, you should highlight the overall summary of the evaluation task, although you have to use a Gantt chart or any other type of project management tool later on. But in your evaluation framework, it will always be useful to have an overall work summary so that your client will understand that what are the various tasks that are involved into this evaluation project. So I hope this gives you a very clear understanding of what evaluation framework is and what are the various components. Now, bear in mind that the evaluation framework is a live document and it needs to be updated time to time. So it's not set in stone. So you create the evaluation framework with the expectation that you'll have to update this. But the evaluation framework will actually help you to consult with your clients. So when you're approached by your clients or the stakeholders, then you have got something to show to them and they will then understand that what are the various aspects that are involved into the evaluation project.