Guidelines for Markers
Tim Kovacs. Last updated Dec. 2009.
This was originally written for markers on the Advanced Topics in
Machine Learning unit who were marking either essays or reports on
a data mining challenge. There are different kinds of assignments and
feedback, and this discussion may be less relevant to other kinds.
Summative and Educative Assessment
There are two main objectives in marking coursework
- To obtain a mark. (This is called summative assessment.)
- To help students learn by giving them useful, constructive feedback (called educative
or formative assessment).
The difference is emphasised by thinking of them as assessment of learning
assessment for learning
Students typically really appreciate good educative assessment. As a
very rough rule a credit requires about 10 hours of work. An assignment
worth 1/3 of a 10-credit unit might therefore take about 33 hours. Imagine
you have spent 33 hours on an assignment. You would probably want to
know why you got the mark you did and how you could do better
in the future. This is particularly true if you did your best and could not
easily think of ways to improve your work. In short, getting useful feedback is very important!
Educative assessment is particularly important for advanced work (such
as final year projects) which requires many
skills (e.g. an ability to write well, to judge what is
relevant, critical evaluation), a high level of understanding, and
ideally shows some creativity and a synthesis of material rather than
just repetition of other sources. Note the use of the word
'skills'. Skills, unlike e.g. simple facts, are hard to teach and hard
to learn. The only way to develop better skills is to use them, and to
get feedback on them.
What to look for
Markers should of course read the assignment specification, and also
for written work
which contains a list of things to watch out
for. Here are some additions.
A report should:
- Synthesise existing material: make new connections between existing areas, present new theories, present
a new point of view on existing material (e.g. a new taxonomy of algorithms). This tends to appear in only
the best work, and there is little scope for it in short reports.
- Present only relevant and significant background material.
- Criticise shortcomings in other work or in the field generally.
- Identify directions for future work (either as a continuation of this report or for the field in general).
- Use appropriate citations (e.g. whenever an algorithm or result is mentioned,
or a claim is made).
- Be clear.
- Use appropriate examples.
- Develop intuitions.
- Present material graphically when appropriate.
- Justify claims either with citations, logic, or sufficient empirical evidence.
- Consider statistical significance and computational complexity when appropriate.
- Have clear objectives.
- Have clear conclusions.
- Have a complete and clear structure: introduction, body, conclusions, references.
- Be nicely formatted.
- Longer reports (including dissertations) should also reflect on how the
work went and what was learned by doing it.
A report should not
- Plagiarise either text or figures!
- Present irrelevant or insignificant background material. In particular it should not be a 'brain dump'
of everything you know on the subject. This shows lack of judgement.
- Spend much time on well-known material (e.g. what a decision tree is, or a tree, or a computer...).
- Follow the structure of other sources too closely when there are sensible alternatives. This does not show any judgement.
- Use excessive notation. This complicates things unnecessarily.
- Use excessive quotation. This is lazy and shows a lack of conviction in your
understanding and ability to express things. It usually suggests to the marker that
you don't really understand the material.
- Use low-quality screen shots of equations and figures. This is sloppy and does not
give the marker a good impression of your abilities and efforts.
Provide a mark as a percentage using the online system, but do not
publish it. For reports it is best to read the reports, make notes on
them, and then rank them. Then you can map them to the UK's nonlinear
. For MSc students a pass is 50% and marks of 70% or more are
reserved for excellent work.
You must provide individual comments on each report. You may also
write group feedback in which you cover points which apply to many
Educative feedback should indicate what is good and bad about a report
and ways in which it could be improved. The amount of feedback should
be in proportion to the value of the assignment.
The standard approach is to upload a text file for each student. Here
are some examples of good individual feedback for short reports in plain text. Another
approach is to write comments on the report, scan it and upload it.
Here is an example scanned
report. Finally, you can also return the hardcopies to the
students but we want to keep feedback on the online system for our records,
so only do this if there is little important feedback which appears
only in hardcopy.
Educative assessment is most useful when it is returned quickly, while
the work is fresh in the students' minds. Since the students do a
series of short reports it's important that they get feedback before
doing the next in the series. Therefore we will set a deadline for
each assignment to be marked. Please meet it or let us know as soon as possible if there is a problem.
Some final words for markers
If they disagree with your comments or marks students may reply to
them. (Usually this is in the form of an email from the student which
one of the lecturers forwards.) Often these students have worked
hard and simply want to know whether their work really is correct or
not - they want feedback. It can also be a matter of principle; when
you're right you want the marker to acknowledge it. Often they are
more interested in this than in changing their mark. (In fact it is the
students who care more about whether they are right or wrong than about their
marks who tend to be the best students.)
Remember that marking is an opportunity for the marker to develop
some of the skills needed to produce reports, especially the ability
to evaluate work critically.
If you have any questions on how to mark don't hesitate to ask one of the lecturers.
Some words for students
If you disagree with a marker's opinion you can debate it. Markers
aren't always right, and don't always agree with each other.
Feedback on feedback is useful for markers. Some marking
is done by PhD students who need to learn how to do it, and
even very experienced lecturers have things to learn. So, if you feel
you aren't getting the feedback you need, or if you find the feedback
particularly useful, let us know.
More generally, if there are things you think are good or bad about
a unit or degree, let us know. It's easier to calibrate the pace
of a unit, or the way material is presented, or how useful the
coursework is, with feedback from students.