Qwizdom: a very negative first impression

[This post reviews Qwizdom's Q4 RF in-class response system, comparing it with Interwrite's IR in-class response system.]

Introduction:

This semester I'm trying out a new (to me) brand of in-class response system. Today was the first day I used the system live in my class, and things did not go well. Before I talk about what went wrong, let me give readers some background.

In-class response systems are great for encouraging participation in classes. To use the system each student in the room buys (or is given) a hand-held transmitter that wirelessly sends the student's response to a receiver at the front of the room; a computer collects all the answers and then displays a histogram of the students' responses. I absolutely love using in-class response systems, and have written a number of posts on them (see my archive here, or just skip to my summary post of the negatives and positives of the systems). I use these systems primarily to encourage participation and quickly evaluate whether students are understanding the material. My grading is very simple: each question the students answer (correctly or incorrectly) is worth one point.

For the past two years (four semesters), I've used Interwrite's PRS system. Interwrite's student transmitters use infrared (IR) to send signals to the computer's receiver (the transmitters are glorified remote controls). Using infrared to send signals has two big drawbacks: each student must have a direct line of sight to the receiver (and thus multiple receivers must be installed in large rooms), and there is no possibility of two-way communication (e.g., the student transmitters can't confirm that the receiver has received the signal).

Qwizdom's RF unit:

The new system I'm using is made by Qwizdom, and it transmits signals via radio frequency (RF), allowing for non-line-of-sight two-way communication. RF transmitters hold much promise for the future of in-class response systems, if only because installing the system in a large room is as simple as plugging a small receiver into the USB port of a computer.

Qwizdom's Q4 transmitter (seen here) has lots of useful buttons; it comes with a 10-key numberpad complete with dash and decimal point, allowing instructors to ask both multiple-choice questions and numeric questions (which would be great for math or physics classes). Other transmitters on the market (eInstruction's RF unit and Interwrite's IR unit) either offer only multiple-choice answers (A-F; eInstruction) or have a numberpad but lack a decimal point (eliminating many possible numeric answers; Interwrite). Interwrite is coming out with a RF unit that does have a full numberpad with decimal point, but I have not used it.

While RF may be the wave of the future, Qwizdom's product has some major problems:

• Price: the student transmitters are sold to bookstores for $49 each; after the bookstore marks them up they'll probably sell for well over $50, making them prohibitively expensive (if I hadn't been able to give each student a free transmitter this semester, I wouldn't be using the system).
  
• Student ease-of-use: I spent more than 20 minutes in lecture today trying to show the students how to use the system and ask my first two questions. Students have to enter both a session ID and a student ID to use the system; many students had trouble doing this.
• Software: Interwrite's software was bad enough that I complained about it in this post, but Qwizdom's (Qwizdom Interact v2.1) is far worse. In fact, I'd say it's awful. I spent hours this week attempting to beat it into doing an extremely simple task, and even with tech support's help it still failed.
  

Qwizdom's Software:

By far the largest negative about Qwizdom's system is the software (though the price of student remotes is also far too high). In-class response system software has two basic tasks: to facilitate asking questions during a lecture (or other meeting) and then collate the data gathered and display it in a graphical form in class (e.g., a graph of student responses) and in a textual form after class (e.g., transfer it to a grade book or allow it to be exported to be analyzed by other programs). Qwizdom's software does none of these well.

The first problem with Qwizdom's software came when I attempted to try it out in my office. It became apparent that if I wanted to record the answers each transmitter submitted (and associate them with some ID number) then each user of the transmitter needed to log in with a user ID. Each user had to have an ID created in the software before the user could log in. The software does have the ability to import data from csv files, but only in a very defined format (about a 10 column-wide spreadsheet, one of whose required columns is gender), and thus I had to convert my roster into their format.

The requirement to have each user of the system already in the program's database before they can use the system is extraordinarily annoying. First, it absolutely prevents use of the system in the first week or two of the semester, as students may not even be officially enrolled in the class, and thus instructors wouldn't know to add them to the system. Second, it means that instructors have to tediously enter student data into the system. It would be much better if users could key in their ID numbers and names whenever they joined a session that they weren't already users in.

There is a way to allow an audience to not have to log in with user IDs, but if this method is used, each transmitter is assigned an (apparently) random ID number; this ID number varies between sessions, meaning that there is no way to track each student's use of the system and thus no way to evaluate student performance.

As a side note, Interwrite's system gets around this entire user ID problem by assigning each transmitter a factory-specified unique ID number that is always stored along with the answer the transmitter submits (so even if you never know who was using the transmitter, you can at least track that transmitter's answers between different sessions).

But getting student IDs into the program was only the first headache. After I did that, my next goal was to figure out how to ask a quick question in lecture. To give a little more background, I have embedded my in-class response system questions into my PowerPoint lecture slides; I go through around 25 to 35 slides per lecture, with maybe 2-5 question slides. This adds up to more than 800 slides over the course of the semester and more than a hundred individual questions. In other words, I've put a lot of work into my PowerPoints, and I'm not about to change programs anytime soon. But that's exactly what Qwizdom's software mandates - their program is designed solely to ask questions that users have presented using their in-program presenter (a complicated-looking clone of PowerPoint). To ask questions you have to make a slide for every question, and specify on that slide exactly what the answer is.

The software does have the ability to ask spontaneous questions that haven't been entered in advance (which eliminates the requirement to use their presenter program), but using this option has its own problems. First, the program doesn't store spontaneous question responses in the gradebook, and doesn't make them available anywhere for review. The only place the responses are stored is a csv file in the program's system directory tree, but even in that file the responses have been separated from the student IDs. And, even worse, when the spontaneous question option is used, the program's control window (which takes up the top 1/6th of the screen) is hard-coded to show up in front of all other Windows' windows, including full-screened PowerPoint slideshows. In short, the spontaneous question portion of the program is functionally useless.

The only solution I could come up with to allow me to ask the questions built into my PowerPoint lectures was to create a dummy presentation in Qwizdom's system with blank questions in it. I showed my PowerPoint slideshow in front and ran Qwizdom's software in the background, but even this worked poorly. Qwizdom's software wanted to run full-screen; when I shrunk it down so it only filled half the screen, the font in Qwizdom's screen distorted awfully, and the response graph was created off-screen. So, what I had to do today in lecture was ask the question in PowerPoint, then alt-tab out to Qwizdom's screen, un-maximize Qwizdom's window and resize it so that it only filled half the screen, wait for students to respond, re-maximize Qwizdom's window, click on a menu item to show the student response graph, click on another menu button to close the response graph, and finally alt-tab back into PowerPoint to continue my lecture. Clunky, slow, and stupid.

Even more hilariously, Qwizdom has a built-in timer that counts down when you start a question; when the time reaches 0 a big "Time's up!" gray window pops up for a few seconds. However, students can still answer the question after "Time's up!", and the time is not adjustable at all during the presentation (it's set only when the entire presentation starts); there is no way to pause, extend, or otherwise change the time per slide during a show. I asked Qwizdom's tech support about this and they reported that it was just a dummy timer that I should ignore. Useless.

In contrast, while I still had to run Interwrite's software in the background behind my PowerPoint slideshows, it didn't require me to make a dummy presentation, didn't need to be un- and re-maximized for each question, had an extremely easily adjustable timer (time for each question could be adjusted by 15-second increments both before a question was asked and while the time was counting down), and the response graph auto-popped up when the timer hit zero.

Even after I clunked my way through the lecture, I still ran into one last problem: the software requires that each question have one right answer worth some set value of points and some wrong answer worth fewer points. Since I want every student to get a point for the question even if they didn't get the right answer, this means that I can't even use my grading scheme with this software. The program did allow me to leave the answer to a question blank (not set), but whenever I did that the question became worth 0 points, and so the gradebook showed 0's for every student in my course, regardless of how many questions they answered. The program won't even tell me how many questions each student answered.

Conclusion:

On the surface, Qwizdom's system appears to be extremely polished. The hardware looks very modern and is very functional. However, Qwizdom's software is atrocious. The software might be OK if you were doing some pre-programmed corporate spiel and didn't mind using Qwizdom's built-in presenter, but it utterly fails when presenting with PowerPoint or whenever you want to do anything even slightly out of the ordinary.

In fact, this week's experiences have been so bad that I have no hesitation whatsoever in giving Qwizdom's product my first-ever Radagast Stamp of Disapproval.