Speaker 1: Hi everyone, this is Ellen Watson, Educational Developer with the Centre for Teaching and Learning and today I'm going to talk about writing an analytic rubric. In an analytic rubric, you'll see a list of criteria and you'll also see a list of the levelling of your criteria and then you'll see descriptions of each criteria. Today I'm going to be writing a rubric based on a reflection that I have students doing in a course about educational technology that I teach in Edmonton and we're going to talk a bit about how I might grade this reflection using an analytic rubric. Now typically what you'll see in an analytic rubric is criteria are written for your assignment. This might include criteria like content, argument, grammar if that's what you're grading. If you were grading a performance, you might grade vocal projection, but whatever goes in those criteria boxes really should come back to the assessment description as well as the learning outcomes that you are going to assess. Now as of late, when I'm writing a new rubric, what I have started grading is instead of criteria I actually put my outcomes slash criteria right in here. So I have two learning outcomes that this reflection is based on and I'm going to take those outcomes and paste those right into my criteria here. I like this method because it forces students to look at what they're supposed to be able to do by the end of the semester and I can clearly show how my assignment connects to my learning outcomes, increasing that course alignment that we talk about so much. The other thing I like to include on assignments, and perhaps this is just me, but I like to make sure that students actually read the assignment and what's expected of them. So I do want them to meet the requirements of any assignment that I include. Not everyone will include this. I like to, just because it's an area that I think is important. But because I'm not going to make this criterion weighted as much as my outcomes above, I am going to move it down and I'm actually going to give this a whole new grading space. The reason that I separated this from my two outcomes is because the course outcomes are what students are actually meant to complete. So I'm expecting students to achieve these outcomes by the end of the course. The requirements of the assignment is something that as an instructor, I would like students to pay attention to. I'm going to attach a small grade to that. This assignment right now could be out of a total of 30 marks. However, I'm going to rewrite that section right in here so that it won't necessarily be a total of 30. Now to look at our descriptions, I typically like to start at the good area. This is what I think the average student will give me to meet this learning outcome. Now in the assignment, I'm asking students to write two outcomes to add to our learning and technology policy framework. Since the outcome that I'm looking at here, outcome to measure my outcomes, is speak to and integrate Alberta Education's learning and technology policy framework into their future courses, I can simply write a statement that says, I expect students to add two outcomes that contribute to the definition that I'm looking at. That's what I'm asking them to do. What are two outcomes that would contribute to that? Now also in here, because I'm asking them to speak to the LTPF. I'm asking them to provide a rationale. Their rationale I am going to separate into two different grades. One is critiquing and discussing and debating the use of information communication technology. The other part is speaking to and integrating Alberta's LTPF. So here I have to write something about their rationale. Well the average student I would expect to tell me why these aspects are needed. You'll also note if you read my assignment here that I do ask them to use literature and course material and discussions to explain their inclusion here. I like to add this into my analytic rubric because a good student in my opinion would use that. Now what I need to do is level this rubric up and level the rubric down. So I have to match what I expected a good student to do and what would an excellent student do to go further and what would a student do that might meet the mark but not quite and the student who's just starting to get it and the student who's really struggling on this. There is a tool that I like to use. That tool is called the AAC Rubric Wordsmith. This is available online and it provides instructors vocabulary to help them level up and level down these rubrics. In my description I ask students to provide one or two outcomes that contribute to the definition so I'm asking them for some meaningful information. If meaningful is my level of proficient or good to level that up I might say students provide one or two insightful outcomes that contribute to the definition. Let's play off this word and I'm going to think a little bit further about that. It's not that I want students to just provide outcomes that contribute to the definition but I want them if an excellent student will not only contribute to that definition but take it further. One way to phrase that is student has provided one or two outcomes that enhance the definition. An excellent student is not only going to contribute to the definition but they're going to enhance. The other next statement I have is rationale speaks to why these aspects are needed. Well, I want students to not only tell me how they why they're needed but I also want them to connect for me if it's an excellent submission. So I do want them to meet that good quality there but I want my excellent students to go a little bit further and contribute show me how they're connected and how they contribute. I want students to use literature and course materials but not only use them but use them to effectively argue their point. Note that when I say effectively argue this is a subjective term and students will need some coaching on what an effective argument is especially if you've never discussed this. So now let's talk about what a student might do in a satisfactory category. I mean in a satisfactory category I'm still going to ask that they should still provide some outcomes but these outcomes may not necessarily contribute they might not further the definition they might be also connected to what's out there. So I could say that the student has provided one or two outcomes that connect to the definition. I can see why they're there and I can see that they're connected but it doesn't necessarily contribute to taking that definition further. Then I would expect a satisfactory student to also at least provide me the rationale but it's probably not going to be as thorough as a student who provides me a good assignment. So the rationale provides some ideas on why about why the proposed outcomes are needed. I would also expect the satisfactory student to try to use some of the citation but they might be inappropriately used. So they use support in appropriately or ineffectively to explain the inclusion. You can see the pattern here so I'm going to I actually paste into the next one and then I look at my level here. So student a poor or a minimal I would probably expect students to provide me only one outcome that might be connected to the definition of a digitally confident teacher but the reader is with left with some questions. I can see that it's connected but not quite clearly. The rationale provides some ideas about why these proposed outcomes are needed but again but the reader needs more connections. So I'm having trouble seeing what the student is saying or how they might be connected and here I would say the rationale uses support inappropriately or ineffectively. I would actually change that to the rationale likely does not use support or incorrectly uses support to explain the inclusion of these proposed outcomes. For a fail any of these statements would not be met. So in my case I would say student outcomes are not provided or are not connected to the definition of a digitally confident teacher a rationale is not provided. Note that I can grade all along here so students might actually meet some statements in the first category and some statements in the second category. This is why I have the grading ranges at the top. This student would likely receive somewhere between an eight or a nine depending on which of these was worth more. So now that we've talked a bit about the leveling here I do want to point out how I came to those numbers at the top and you'll see that they are ranges. I like to have ranges because sometimes a student you know meets the mark but not quite at the level it should be so that discretion is still there and it also avoids giving half marks if they cross different sections. That being said you will notice that the top three categories in my rubric are actually six through ten. I want to remind people not to split these in half so if you make the satisfactory let's say if I went there's five of these so I said excellent is worth five good is worth four satisfactory is three poor or minimal pass is two and a failure is one. What that would mean is that a poor or minimal pass at two is two out of five which actually is not a pass so that's problematic. The rest of it might line up with about where you want but that two is problematic. So here the way that I've set my rubric up I can see that the poor or minimal pass is worth five satisfactory being about sixty percent good is seventy to eighty percent and excellent is ninety to a hundred percent. That being said you don't necessarily need to use numbers you can use these descriptors but I also match my descriptors up to the GPAs on the University of Alberta grading statement that they have on their on their website. Now I wanted to talk a bit about meeting the requirements of the assignment. In order to do that I'm going to make this a little bit smaller just so that we can see that requirements. I do need to level it similar to what I have up here but because I want it to be worth quite a bit less I might not use all of these categories. So instead I'm going to use excellent or good satisfactory failure. This is for a few reasons and then I'm not going to do anything with these two blocks so I'm just going to make them blacked out. This is for a few reasons now I don't need to make this worth a total of ten I can make it worth you know maybe this is my four or five satisfactory is going to be a three and failure being two through zero. So now my total assignment would be worth out of a maximum of twenty five marks with only five of them being dedicated to meeting the requirements of the assignment. This forces students to recognize the weighting in what's important when they're completing their assignment. Do I want you to meet the requirements of the assignment yes but it's more important that you meet the learning outcomes they're worth more. Just to talk about leveling here when I'm talking about meeting the requirements I would probably include something like assignment requirements word count or timeline or length et cetera have been met. Of course I would like that if they're in the excellent or good range. The task has been completed as described. That's going to be reflected in some of the pieces up top as well you don't have they included outcomes et cetera. If I'm going to include something about grammar or spelling I would probably put it in here and I would say something like there are no or very few grammar or spelling errors. Now to level that down in the satisfactory the assignment requirements have almost been met. You know this would be if the student gives me I asked for 500 words or four minutes. If they give me six minutes or three minutes or three minutes you know they're close but it's not quite what I was asking for. If they give me four and a half minutes I'm probably not going to put them in the satisfactory range but this is you know if you ask for 500 words and a student gives you 800 that's a little bit long. Being succinct is also a skill. In the satisfactory range I would still expect that the task has been completed as described. You know maybe some questionable modifications have happened. Maybe some aspects have been missed. You could include that as well. Then for grammar or spelling you know grammar and spelling errors here would be present but not you know the student could work on their spelling or grammar but it's not to the point where they're actually failing this assignment. So then for failure anywhere between two and a zero assignment requirements would not be met. The task has been submitted but not completed as described or task has not been submitted and the grammar and spelling for me in a failure would be noticeable. You know they're to the point where it's difficult for me to read the piece of work or listen to the speech because those errors are so noticeable. I hope that this video has given you some insight on how you might write an analytic rubric for an assignment. Remember it's easiest to start either with the excellent or the good category depending on what you're asking students to do. Typically the good category will meet all of your intended outcomes. Be sure to give students a description that's in a language that they can read and in using words that they're familiar with. Also try to use your scoring to your advantage by giving students an idea of what aspects of the rubric are worth the most and using this design to your benefit. If you have any questions or you want any help creating your own analytic rubric please contact the Centre for Teaching and Learning by emailing us at ctl.ualberta.ca or drop by our website at ctl.ualberta.ca. Thanks see you next time.