In-class response system update: the negatives

As I've discussed before (here, here, and here), I'm using an in-class response system for the first time this semester in my 180-student zoology lecture. The system is designed to encourage participation in a classroom setting. Students answer questions using handheld transmitters, and the answers are automatically tabulated on the computer at the front of the room. See this post for a more thorough explanation of the system.

I've now used the system for four weeks, and overall am quite pleased (Educue's Personal Response System is the brand I'm using). However, there have been a few speed bumps and potholes on the road, and I'll talk about those in this post. In a future post I'll discuss the benefits.

Installation was deceptively easy. We used Velcro to attach the four receivers to the wall, and connected the units to the computer with cables provided by the manufacturer. Unfortunately, the Velcro failed. During my first lecture with the PRS system installed, one of the receivers at the front of the hall fell off the wall, bouncing loudly on a metal stool. All of the four receivers have now fallen off the wall at least once, and to prevent more loud lecture interruptions I've been taking down the receivers after each lecture.

It appears that the Velcro has been strong enough to hold the units onto the wall; the problem is with the glue attaching the Velcro pads to the wall. I've ordered some mounts for the units that screw into the wall ($7 each from Educue), and these should solve the problem.

The second major problem is that students have had difficulty acquiring the transmitters. The transmitters were bundled with all new textbooks for my course, but there were initially no transmitters available separately in the bookstore, as the bookstore staff wanted to force students to buy new books and were worried about returns. They finally agreed to stock the units separately, but ordered only a few units at a time and priced them with a high markup. I still have a few students who are waiting to get their transmitters; primarily these are students who bought them earlier and subsequently lost them, but it's been taking a week or more for students to get the units.

The third problem has been with units breaking. I've had three students report that their unit has broken after only four weeks of use. I suggested that the students go to the bookstore or the manufacturer and see if they could get a replacement, and I believe that they've been able to acquire replacements without a problem (but am not certain). The failure rates seem relatively high, and if the system was being used for quizzes or tests this would have been a significant problem. Additionally, students regularly forget (or misplace) their transmitters; in nearly every lecture I've had at least two or three students who didn't have their transmitters on hand.

Even with functioning units in class it's not clear that every student is able to respond to every question. The transmitters are a bit difficult to aim, and each individual receiver can only receive one signal per 1/10th of a second (or so), meaning that if multiple students simultaneously send a signal to one receiver only the first student's signal will be received. To account for this, the student transmitter ID numbers are displayed on a grid on-screen whenever a response is received. However, since I need to devote the lower half of the screen to the question text, the grid can get pretty small and with a large class it's difficult to see when each student's response comes in. I've had students repeatedly say that they're unsure whether their answers are being received by the system.

The software that Educue ships with the unit is another problem. After a bit of time playing with the program I've been able to use it easily in class, but it's far from intuitive. There are at least seven separate types of files that can be loaded or saved at one time: session files, settings files, question properties files, name files, class files, answer files, and grade book files, to name the ones I can think of quickly. I've had the program stop receiving signals at least twice, and had some annoying errors (and data loss) when the program tried to save files to a directory I didn't have access rights to.

These units are often marketed to be used as quiz and testing tools (touting paperless, automatic grading), and based on the problems that I've seen so far I'd have to say that they would probably not make a good testing tool. The two other in-class response system users I talked with over the weekend at the conference had very similar opinions, so I don't think I'm alone here.

So, what are the systems good for? They're excellent for encouraging participation and conducting formative assessment. Even considering all the negatives, I've enjoyed using them this semester, and plan on continuing to use in-class response systems in the future.

[Update: I've posted a discussion of the benefits of the in-class response system here]