## Creative grading schemes August 6, 2010

Posted by Noah Snyder in Uncategorized.

This fall I’ll be teaching my first regular college class (I’d only taught sections at Berkeley, though I suppose the summer sophomore tutorial I taught at Harvard might count). It’s on group representation theory, which is my favorite subject, so I’m excited about it. I was just thinking about some possible homework problems, and I got to thinking about creative and unusual grading schemes I’ve seen in previous classes I’d taken, and figured that might make a fun blog discussion topic. (Since this is my first time teaching I won’t be experimenting with any unusual grading this time around, though I think it might be interesting to try one of these in the future.)

Redos:
At the Ross summer math program if you don’t answer a problem satisfactorily then you get a REDO. This means you’re expected to go back and redo the problem and get it right. I’ve never seen this tried in a regular class, but I think it could be a good idea for an “intro to proof writing” class. The point being that in such a class the material itself isn’t super important, and so if you do fewer homework problems total but learn how to do them right that’s a good tradeoff.

When I took group representation theory from Richard Taylor, the exams were graded out of a ridiculous number of points. A 5 question midterm would be out of 600 or so points. At first glance this seems silly (and it certainly would be a bad idea for a class with multiple graders where you want consistency between graders), but it actually works very well. Here’s the point: if someone does something you don’t like no matter how small it is you can take off points! Unclear sentence? Minus 1. Used the wrong terminology? Minus 3 points. This way the grader can effectively communicate relatively small shortcomings in your write-ups, which wouldn’t be possible if you were grading out of a smaller number of points.

Perfection bonus
This idea comes from a class that I didn’t take our first year of grad school with Givental, so perhaps someone who took the class can correct me on the details. The basic idea that was for the final in addition to points for each problem you got, there was a pool of extra points which you got if you never wrote anything false on the exam. But as soon as you wrote something that was wrong you lost those points. This is good training for graduate students who soon won’t have graders telling them when they made a mistake, and it’s a good way to keep people from spewing nonsense in an attempt to get partial credit. If I remember correctly the perfection bonus was quite substantial (I want to say it was worth as much as a full question on an exam where you need a little more than 2 correct solutions.)

What do people think of these ideas? Any other interesting grading schemes you’ve heard of?

1. Suresh - August 6, 2010

I like the perfection bonus. A corollary that I’ve heard of is to assign at least a 20% fraction of the points if you merely write “I don’t know” instead of blathering on

2. Scott Carnahan - August 6, 2010

I’m interested in hearing more details about the redo policy at Ross. What sort of feedback was given with each redo request, and how stringent was the criteria with respect to, e.g., writing style? After I heard you talk about the redo idea in grad school, I mentioned it to a friend who implemented some version of it when teaching some undergraduates. Unfortunately, the students only fixed the specific problems that were explicitly pointed out in each iteration, rather than (say) polishing the presentation in general.

3. allan adams - August 6, 2010

hi noah,

many points: will you be grading, or will there be a separate grader? i’ve done this and it works great when i grade — but when i’ve had graders, they *hated* it, and did a lousy job, so the students hated it too.

i’ll also be experimenting with redo’s in my quantum class this year, though for a slightly different reason: I want to force students to go through the solution sets *before* the night before the final exam. so each week, in addition to the weekly problem set, the students will turn in a corrected version of their previous week’s problem set, allowing them to recoup some fraction of the point previously lost. i’ve never heard of anyone doing this but surely it’s been tried — would love any feedback or advice on such a strategy.

wouldn’t it be nice if there were data on how various strategies panned out?

good luck!

cheers,
allan

4. Noah Snyder - August 6, 2010

So such things at Ross certainly varied from counselor to counselor. I typically met with students to go over their problems and explain what I was unhappy about with the problems that got redos, and also wrote short comments on the sets so that they’d be reminded about the issues when they went back through. I don’t think I very often gave redos over issues like writing style, but if the student was generally unclear in their writing I’d pick some particularly bad example and make them redo that one until it was clearly explained.

It’s true that things don’t necessarily work so well with students who are dedicated to putting in the minimal amount of effort. If people are happy to copy their old solution word for word and then make small changes then certainly there’s less value in the technique.

5. John Palmieri - August 6, 2010

For exams, I write minimal comments on the students’ papers, and then I give them about a week to fix any mistakes; they can earn 1/3 of a point for each point they missed originally. And yes, I do get students who got 49 out of 50 who turn in revisions to boost their score to 49.33… You have to grade twice, but the first time is sort of quick because you don’t have to write much, and the second time is really quick because usually they have figured out how to do it the second time through.

For homework, this isn’t really a grading scheme, but I’ve been using “portfolios” for a while for intermediate to advanced classes: for maybe a half dozen of the homework problems for the term, the students write several drafts. The first draft of every problem gets graded by the grader if I have a grader, or by me if not. The second draft (for only one or two of the problems) gets looked at in peer groups — they pass out copies for their colleagues to take home and evaluate, and then in the next class they spend some time going over their comments. I may let them turn in a draft (of one or two problems) to me for further comments. Then at the end of the quarter they turn in final drafts of everything, with a cover sheet explaining which are their strongest problems. Then I grade the portfolio, focusing on the strongest problems, evaluating both mathematical content and exposition.

6. Tom - August 6, 2010

This is perhaps not so uncommon – I once had a professor who wrote gigantic exams that had a total of 400-600 points available. The goal of the student was to answer 100 points worth of questions (of the student’s choosing) correctly. The questions varied widely in point value. The student could state a and prove a big result for 60 points, or write a definition for 5 points. Many of the questions on the exam were taken directly from assignments.

From my perspective as a student, these exams were beautiful. I knew that every ounce of studying I did would translate directly into a better exam score, as the tests were so big that every last topic would appear on each exam. The downside: We (the students) all knew that it was okay to skip some topics while preparing for the exam, because we could skip 3/4 of the questions on the test.

My impression was that the professor just wanted us to show him that we learned *something*, in some detail. This exam style is probably not well-suited for a professor that wants to see if the students have mastered the basics of all of the topics in the course.

7. Nathan Dunfield - August 6, 2010

I’ve definitely reduced the number of points for each exam problem over the years. A typical hour exam now usually has only 30-40 points. I find this makes it quicker to grade.

I also like having the total number of points be out of something other than 100 — it lets you control how students interpret their grade as they can’t easily apply the usual A = 90, B = 80, correspondence.

8. Allen Knutson - August 6, 2010

I also like having the total number of points be out of something other than 100 — it lets you control how students interpret their grade as they can’t easily apply the usual A = 90, B = 80, correspondence.

That is a _great_ idea.

I was going to mention something I do that I think of as the Caltech system — A = 60, B = 50, approximately. It means that when people say “What do I need to do to get a B in this class?” I can say “Get a 65 on the final” not “Drop it and take it again”. Similarly, “Yeah, you got a 30, but that’s not going to totally ruin you”.

It means that the grades are, even more than usual, determined by the tests and not the HW. It has the good side that people don’t sweat the difference between 90 and 93, believing it’s the difference between B and A. On the other hand, surprisingly many people act as “He can’t possibly mean what he said, that my terrible 55 was a B+” and are depressed by what is just a number.

9. Brittany - August 6, 2010

The redo system is wonderful in a proofs class. This was utilized in my “Intro to Higher Mathematics” class, which was essentially Proof Writing 101. Sometimes the only way to figure out proofs is to write a whole bunch of them. I highly recommend this method for your future classes.

I would probably enjoy having a test out of 600 points. The points you make about writing style and nitpicking would actually be helpful–if annoying. But in reality, I feel that you could still mark those on tests and just not take points off.

10. Chris Godsil - August 6, 2010

I use the redo scheme for my graduate courses. As with all these
things, the devil’s in the details. I hand out a long list of problems
and the students are required to hand in some minimum number
during the course; they can do more (and often do).

I give 5 marks for a correct solution, 4 for something basically correct (where they’d learn nothing be rewriting it); otherwise it’s redo and they earn at most four marks. It’s amazing how it improves the quality of the submitted work.

You need to set up some system of due dates, or at the end of the course they’re still working on the early material.

I do not give a final exam in these courses. I reserve the right to impose one, but this is so students with no time management
skills can earn an acceptable grade.

11. Tim Riley - August 6, 2010

Here’s a measure (not the only one) that is used to compare students taking final exams at Oxford: the sum of the squares of the scores on each question. It has a similar effect to the perfection bonus, but isn’t all-or-nothing.

12. Dagan Karp - August 7, 2010

Here are a couple related grading tips used in some courses at Harvey Mudd; I think they were developed by Michael Orrison for courses in algebra and representation theory.

Redos: problem sets can be redone, due two weeks after the original due date. The grade given is simply the highest grade earned. The original is resubmitted with rewrites.

This only works if you have lots of graders. It seems to really reinforce the importance of understanding HW problems and writing solutions up clearly.

Nonstandard scale: 95 and up is A+, 85-95 is A, 80-85 is A-, etc. If you use only integers when grading, this gives more room for variance, instead of leaving over half the scale for different flavors of FAIL. And it has the positive effect on perception Allen mentioned above.

Greatest component: say the course has 2 “midterms”, homework, and a final exam, and they are weighted equally in the course grade. Instead of assigning each 25%, make each worth 20% with the final fifth being the greatest of the components. So, if a student kicks ass on their HW, it may be worth 40% of the course grade. Similarly for the final.

This helps give the message that it’s never too late, and that HW is important, etc. And of course you can augment for different numbers of components or unequal weight etc.

13. Jan - August 8, 2010

I have some experience with the “redo” policy both from a grader and from a student point of view. You need to be very careful and deadline strict (for student and grader) for that to work. As an example, I took a class where we had:
week 1: exercise class (1)
week 2: exercise class (2) + hand in homework assignment from week 1.
week 3: exercise class (3) + hand in homework assignment from week 2 + get back assignment week 1.
week 4: exercise class (4) + hand in homework assignment from week 3 + get back assignment week 2 + hand in redo from week 1.
week 5: exercise class (5) + hand in homework assignment from week 4 + get back assignment week 3 + hand in redo from week 2 + get back redo from week 1.

So there was a 5 week delay between the exercise class and the final homework grading, if there were no delays anywhere. If you do this for 20 weeks, chaos is sure to ensue without very strict policies.

14. Edvin - August 9, 2010

At my university we’ve been using the redo-grading for a numerical methods course for quite a while. This really forces the students to learn the intimate details of some selected methods, but it is a bit more painful for the lecturers who have to correct the same assignments over and over again. No one is allowed to present themselves for examination until they get all assignments right, so there is a practical limit to how many times a student can redo.

I have also seen this scheme in a quite different course, when I studied molecular modeling at the chemistry department. What I found attractive about this scheme as a student, was that I could apply a different strategy to the topics I was unsure about. In stead of utilizing rhetorical skills to hide my shortcomings (like I might do if I knew that my mistakes would be reflected in the finale grades), I could do the opposite and be particularly elaborate on matters I knew less well in order to get problems pointed out.

15. Lyla - August 9, 2010

I had a first proofs class (Intro to Analysis) where we could do redos for problem sets. Problems were originally graded out of ten points, and you could submit them the next week for a maximum of eight points.

As a student, I really liked it, because it gave me an incentive to go back to problems I had never figured out, or didn’t fully understand, which otherwise fell on the endless list of “things that seem like a good idea but are never going to happen.” It was especially nice in a first proofs class because I think people newly exposed to real mathematics are a little more prone to going into mental shock at some idea or other, and forcing/giving them an opportunity to process it for another week made it a little easier to stay afloat.

One downside was that it made procrastinating more tempting. (And now that I’m on the other side, I can certainly recognise that it could cause an oppressive amount of grading to pile up.)

16. Scott Taylor - August 9, 2010

When I taught an undergraduate topology class, I used a redo system for the first half of the course (point-set) but not for the second half (algebraic topology & classification of surfaces). I had only 7 students in the course, but it worked really well since it leveled the playing field at the beginning of the course between students about to graduate and students straight out of “Intro to Abstract Math”. During the point-set portion of the course, I would assign weekly problem sets which I would comment on and assign a “tentative grade” (if the problem set was good enough to earn one). The students then had to turn in all of their point-set homework one week after we concluded the point-set section. They also had to turn in their original drafts so in regrading I simply had to focus on the errors and poor communication I had pointed out in the original draft.

Not using the redo system in the second half of the semester gave the students the feeling that they had “graduated” and were now capable of meeting my high expectations for correctness and exposition.

In the fall, I will be teaching “Intro to Abstract Math” for the first time and am pondering the best way to use a redo system in that class (for 30 people). In particular, I want to be sure not to provide any incentives for students to slack-off the first time through a problem set. A colleague uses a system whereby students are only allowed to redo a problem set if the original is of sufficiently high quality — and he keeps the standards for reaching that threshold a secret. He has success with that method, but I prefer to be more upfront about what students exactly need to do to meet my expectations. (Not that I have no subjectivity in grading …)

17. Asaf Reich - August 18, 2010

Hi Noah!
Just thought I’d add, like others, that the ‘Intro to Proofs’ class at Brandeis is always graded on the redo system for problem sets (and maybe for tests as well.) The actual grade is based entirely on the revised version.

18. Andrew Stacey - August 24, 2010

Tim Riley beat me to the mention of the Oxford system. It’s worth saying that this was for the final exams, and was used in conjunction with three other measures. One of which was the ordinary sum of all the marks. The other two started by assigning each question a letter grade (α, β, γ) and then one was the number of αs and the other was twice the number of αs plus the number of βs. To get a first (top class degree), you had to be above a certain level on all *four* of the systems.

However, grading an exam is very different to grading homeworks. An exam is a test to see if a student has achieved enough to pass the course. Homeworks are part of the teaching of that course. I’m probably not alone here in thinking that mathematics is best learnt by actually doing it (it’s not a spectator sport!), from which one could deduce that the homeworks are the most important part of the course.

I would say that you’re in danger of going about this the wrong way. You should start with your aims and objectives for the homeworks (with apologies for using the A&O phrase!). What do you intend the students to achieve by doing the homeworks? Then you need to think about the following:

1. How do I encourage my students to actually do the homeworks? This is what, for unmotivated students (ie most of ‘em), schemes like “Homeworks count for X% of the exam” are aimed at. However, I’d prefer “You must do N homeworks to be allowed to take the exam”.

2. How do I encourage my students to think about the homeworks after they’ve done them? This is where “redos” can help. This obviously needs tying in with whatever scheme is used in part 1.

3. How do I make life easy for my graders? Since the graders are involved in giving the feedback, to make it useful feedback, it needs to be easy for them to do (otherwise they’ll not do it well, or at least, there’s no guarantee that they will).

So a grading scheme for homeworks has to do two jobs: encourage the students to do the homeworks, and give feedback to encourage the students to look at them again. These can actually be in conflict at times, so I would recommend trying to have a scheme that has two components so that the two roles can be put into the two components and they won’t conflict.

(time for me to go and teach now, so I don’t have time to develop this further – which is probably just as well as I really only wanted to make the point that the grading policy should *follow* the homework policy, not the other way around)

19. Josh - September 16, 2010

I had a professor at Caltech who did two-dimensional grading. He plotted exam scores versus homework scores, and gave the same grade to clusters.

Effectively a non-linear mapping between exam scores, homework grades, and the final grade.

He also thought that the final shouldn’t be weighted the same as the midterm, but he also thought that weighting it twice as much was too severe, so he took the geometric mean of the two.

Sorry comments are closed for this entry