In a previous shared office a colleague always looked for the EASIEST way to do things. He had a very disruptive and creative approach to simplification, one that really provoked thought. We often discussed marking: specifically how the grunt work could be taken out of it while ensuring we provided clear feedback to students on what could be done to improve future work (see my post here).
With my colleague's constant tinkering and prodding, I too was always considering 'better ways'. In those days we marked using Word rubrics, which - taken together - were (a) vague and (b) dangerous and (c) could lack consistent interpretation. Let's take the 'vague' part first.
Vague is what I see, particularly when we consider a rubric approach. What exactly does the following mean as a chunk for a presentation?
A Grade
Presentation (20 marks). Verbal communication is clear, audible and highly effective. Seamless integration of visual aids/technology/ supporting materials. Ideas/opinions are conveyed fluently that intentionally stimulates critical discussion.
Of course, the B is also listed/specified. And the C. And D. The questions such a chunk as this inspires in me are:
- How much is each element/sentence worth? Are they all worth 6.66 marks each? Or are some worth more?
- Are there some elements which should have been listed, but haven't yet? What about the slide deck? What about pace? Volume? Length?
- Are a quarter of the marks in the last sentence for ideas, for opinions, for stimulating and for critical discussion? Or are there other characteristics for a good presentation? How much discussion can we really create in a presentation...? Should discussion appear instead in an - unspecified - Q&A session?
- Further, why would we then need to list what a B or a C or a D is? Doesn't it become obvious because we have listed the A, that the B, C, D, or E hasn't reached that standard? Aren't we wanting all students to aim for an A? Do we need to list what a D is - usually something pointless like "doesn't reach minimum standards" which surely is rather obvious?
- And why, oh why, did we end up having to write SO MANY similar comments in the comments space? Many are repeated student to student... so surely they should be in the rubric?
OK. So let's consider dangerous. Word is not really an 'adding up' tool. We tend to manually add our marking up, which leads to easy mistakes. We can get Word to add up for us using formulas, but the table addition functions are less than ideal. In my experience, this: often stops working; must be manually refreshed; and others don't 'see' the sums as a formula, so they overtype the totals even when they are working. Why use a tool so obviously unsuited?
After a lot of thought and office conversations, I felt that a master check list dashboard would be ideal. That I could be marking a piece of work, and tick a box to have a particular comment appear on a marking sheet: effectively pick something already written. I was sure that someone out there must have already created something. We could just enter our percentages and go. Well, no. I found nothing over a summer of seeking.
My next thought was that I could contract a computing major who knew Access really well, who could create a pick list for me, and we could go from there: I could have a few master tables, a pick list on a form, and create a customised report, student by student, without having to manually tailor the marking feedback. I could create a comments bank from my own marking to begin our master list of "what was good" and what "needs improvement". For two semesters I sought a student to work on this as their capstone 300 hour project. However, despite having written a clear brief and having good connections to the computer science team, I had no takers. And I didn't know enough MS Access to go solo.
Then the need suddenly became urgent. The issue of consistent interpretation arose. I was leading a team of up to ten research supervisors who were all marking on the same paper. Marking results were initially variable using a standard Word feedback sheet. One person's 'B' comments were another marker's 'A'. I really needed a more consistent method, because the inconsistency was immediately visible between students. They talked to each other about their marking. And I needed it now.
Expediency drove the choice. I decided to have a crack at creating something in Excel, which I knew well. Maybe I could make a huge list of what would have been my Access check list dashboard, and we could tick the level which students achieved. If I set it up with cell protection and the right formula it would avoid addition errors: Excel adds. And if I only listed "A" comments, but with a range of grades where markers only needed to put an "X" into the relevant box in the relevant grade column, it might simplify the process.
So I had a go. I sat down with some experienced markers, and we brainstormed the contents of our rubrics and marking splits. We broke down all the marks for each area into even chunks, and created a comment that could be marked. Because there were so many items that we were could select, additional comments were minimised. I set up the sheets, and we ran some trials. We marked students both ways on a small assessment to check that the Excel marking would work out the same way (it did). We tweaked some statements that didn't quite work. Then we showed the students both the marking sheets, and asked them which one they preferred. The preference for the Excel version was overwhelming. So we rolled it out.
Moving forward five years, and the benefits of using Excel are huge. It speeds up marking; it reduces time in writing comments; it is clearer as a guide to students; there are no addition errors; and it creates consistency across markers.
Interestingly, lecturers who haven't used it think on first look that it is too 'specific'; too confining. However, once they try it, they tend find it makes marking much easier, and faster.
I too find it easier. I tweak these marking sheets each semester, so the wording gets better, the formulae better, the streamlining better. View a presentation sample marking sheet (as per the illustration) here.
Sam
read more "The thorny problem of Word marking sheets"
