technology tip of the month Pointer and Clicker Article
Virgil E. Varvel Jr.
Rubrics


Using Rubrics

Once you've created your rubric, the next step is to use it. As stated before, during the creation process, you should have taken into account both who would be using the rubric and who would see the rubric. Here are some guidelines for putting the rubric to use. They may seem like a lot, but as you begin to use more rubrics, these guidelines will become second nature.

  1. Begin slow. The first time you use a rubric, you should carefully look over the student's work to make sure that you have a good idea of how to interpret your criteria and scoring dimensions. Often, as you go through a rubric for the first time, you change your mind at some point on the scoring divisions on at least one of the criteria. You are then left to go back through what you have already looked at to see if they need corrected.

  2. Try hard to stick to the rubric. As you deviate from the rubric, you enter your own personal biases into the scoring. You also may not deviate in the same way from one student to the next, reducing the reliability of the rubric.

  3. Monitor your thinking as you go through the assignments to insure that you are maintaining consistency.

  4. Train multiple scorers. If you will have more than one person using the same rubric, it is very important that you achieve inter-scorer reliability and consistency. One method of helping to maintain consistency is to first test scorers on a small sample of student work to make sure that they provide the same results and results that are in line with your expectations. An example of such a situation would be when two different instructors teach two different sections of the same class. They may score the same rubric in different ways, making one section's scores appear different than the others in a manner that is not related to student knowledge, instructor ability, or some other variable.

  5. Beware of boredom. As you grade many papers, no matter what method you use, it can become tedious as you reach later papers. It is not uncommon to begin to think of the current paper as no different from the last and thus probably deserving of the same grade. Especially when you have an analytical rubric, you can't judge a final score based on the look and the feel of an assignment. You have to maintain diligence.

  6. Along the same lines, don't skim. I once had an instructor way back in high school who obviously skimmed student work. Several students would add sentences randomly to their work such as "You stink." although often in not such nice language, and the instructor went most of the year without noticing or at least not saying anything. There is more to a paper, and you can extrapolate the context to other assignments, than the first sentence of every paragraph and the overall length.

  7. It is not how a paper looks that you are scoring, but the work inside. While presentation may be a part of the score, don't let an initial impression bias your assessment of the content.

  8. You are not grading the student, but their work. Just because you like a student or s/he always does well in the course, does not mean that the current assignment is as good as the others. From personal experience, I would always try to get the best score as possible on the first assignments and actively participate at the beginning of a course. These first impressions can carry over into a course. Even though the student may begin to slack off, their future scores may not accurately portray this situation if bias affects his/her score.

  9. Avoid empathic scoring unless that is your intent. In other words, unless effort is one of your criteria, you shouldn't grade the final work on it. If something comes easier to one student over another, then an accurate grade may reflect that if the slower students are not able to complete quality work. Unless there is an individualized education plan with accommodations for effort or some other factor, it should usually not come into your scoring. An A for effort is great as one part of an overall grade, but it should not be so great as to raise a letter grade unless there is a clear objective for effort.

  10. Order effects. After viewing 10 good assignments, the next assignment that doesn't measure up may be given a lower grade than it deserves. The corollary would be true when 10 bad assignments are viewed followed by an average one that is given a higher grade than it deserves.

  11. The error of central tendency. It was mentioned before that some rubric developers prefer to use an even numbered scale as opposed to an odd one. The reason is that you may tend to feel that if a work is average, then it deserves a 3 out of 5. The problem is that 5 is not above average, but excellent. When you have a 5 point scale, don't forget to use 4 and 2 when a work is deserving of those intermediate scores.

  12. Possible pitfalls with multiple scorers exist. Use a sample score and training to bring the scorers together.

    • Positive-Negative leniency error. One scorer is harder or easier than the other.

    • Trait error. One scorer is harder or easier on a specific criteria.

    • Length effects. One scorer tends to grade easier as the assignments get longer or the more assignments that have already been viewed.

    • Personality clashes. One scorer may like one student or believe that a given topic is more important. Sometimes the only way to correct for such an error is to add a weighting factor on final grades. Determining what that weighting factor is can be difficult though and require you to go through many samples of that graders work in order to compare it to the others and determine what corrections need to be made.

    • Self-scoring. Some people are harder on themselves than others.

Back to Top

 

ION Home | Pointers and Clickers

Home | Online Courses | Institutes & Presentations | Educational Resources | Initiatives | Institutional Partners | About ION

uiuc uic uis