Wednesday, November 15, 2006

Test administration in online courses

Profgrrrl recently posted about a new device that helps proctor online exams. The device is apparently a combination fingerprint reader, microphone, and webcam - it can verify the student's identity and then record what the student does (both audio and video) during the entire exam.

It's the first good solution I've seen to the two biggest problem of online courses - how do you know the person taking the tests is the person enrolled in the course, and how do you know that they're not using outside resources to help on the exam? That there was no good solution to these problems is why my newly designed online course requires students to take all their exams on campus. I only mildly care if students get help writing discussion board posts or have someone else do their homework (since that will likely come back to haunt them on exams), but I definitely care if they get help while taking their exams (as then someone could pass the course without ever learning the material). In short, this camera is a pretty cool thing (excluding the cost, which is about $150).

Profgrrrl hates the thing. Why?
So how would you feel about taking a test with a camera hanging over your head? Would it give you a bit of anxiety? Would you feel self-conscious if you had to scratch your nose? If a question was so hard that you start to cry, would you feel twice as bad knowing it was being caught on camera? Would you spend time wondering if someone really was at the other end, watching your every move? If they were recording you? And who could access those recordings?

Now, I'm not a paranoid. Really, I'm not. But I this device looks like a disaster for anyone with test anxiety and just sends a message of "you'll cheat unless we go to great lengths to control you" to everyone else.
The question of how these videos will be used is a valid concern, but online courses already send the message of "you can cheat as much as you want and we don't care," so by doing something to actually stop that cheating we're hardly accusing everyone of cheating. This is the same "guilty until proven innocent" claim made by people against plagiarism-hunting software, and it doesn't hold up here any better. And, how is being observed through this camera any different from having a professor (and dozens of other students) in the room watching you when you take a test on campus?

So, given that profgrrrl has taught online classes, how has she dealt with exams?
I recognize that students could their books in front of them -- so I take a cue from my stats prof who gave us timed open book tests with questions that required critical thinking skills. You could use the book as a resource, but if you didn't already know this stuff there was no way you could look it all up and figure it out in the given time. And if someone else takes the test for the student? Well, I sometimes would ask test questions that related to discussions we had had in class (sneaky, eh?). I also can triangulate test performance with discussion board performance readily enough.
Asking a couple of test questions related to discussions is not going to prevent cheating. What's to prevent a student from being on their cellphone with another student (or with someone who's taken the course before) while they take the exam? What if a student does well on tests but hates writing discussion board posts (or vice versa); should we accuse a student of cheating when they've been posting mediocre discussion board posts but then do well on an exam? What if the student finally studied? That's hardly the way to catch a cheater. And none of this even addresses the problem of how to know whether the person who posted the discussion board posts (or the exam taker) is even the student enrolled in the course.

Administering exams online in an unproctored environment is tantamount to giving free points. If those exams are the major basis of the grade in a course, then I would seriously question whether those grades accurately reflect student learning in the course.

Unless more things like these cameras come on the market, I see no good way to evaluate student learning in online courses other than to require them to come onto campus and be evaluated in a proctored environment.

No comments: