Thursday, February 16, 2006

Qwizdom: a very negative first impression

[This post reviews Qwizdom's Q4 RF in-class response system, comparing it with Interwrite's IR in-class response system.]


This semester I'm trying out a new (to me) brand of in-class response system. Today was the first day I used the system live in my class, and things did not go well. Before I talk about what went wrong, let me give readers some background.

In-class response systems are great for encouraging participation in classes. To use the system each student in the room buys (or is given) a hand-held transmitter that wirelessly sends the student's response to a receiver at the front of the room; a computer collects all the answers and then displays a histogram of the students' responses. I absolutely love using in-class response systems, and have written a number of posts on them (see my archive here, or just skip to my summary post of the negatives and positives of the systems). I use these systems primarily to encourage participation and quickly evaluate whether students are understanding the material. My grading is very simple: each question the students answer (correctly or incorrectly) is worth one point.

For the past two years (four semesters), I've used Interwrite's PRS system. Interwrite's student transmitters use infrared (IR) to send signals to the computer's receiver (the transmitters are glorified remote controls). Using infrared to send signals has two big drawbacks: each student must have a direct line of sight to the receiver (and thus multiple receivers must be installed in large rooms), and there is no possibility of two-way communication (e.g., the student transmitters can't confirm that the receiver has received the signal).

Qwizdom's RF unit:

The new system I'm using is made by Qwizdom, and it transmits signals via radio frequency (RF), allowing for non-line-of-sight two-way communication. RF transmitters hold much promise for the future of in-class response systems, if only because installing the system in a large room is as simple as plugging a small receiver into the USB port of a computer.

Qwizdom's Q4 transmitter (seen here) has lots of useful buttons; it comes with a 10-key numberpad complete with dash and decimal point, allowing instructors to ask both multiple-choice questions and numeric questions (which would be great for math or physics classes). Other transmitters on the market (eInstruction's RF unit and Interwrite's IR unit) either offer only multiple-choice answers (A-F; eInstruction) or have a numberpad but lack a decimal point (eliminating many possible numeric answers; Interwrite). Interwrite is coming out with a RF unit that does have a full numberpad with decimal point, but I have not used it.

While RF may be the wave of the future, Qwizdom's product has some major problems:
  • Price: the student transmitters are sold to bookstores for $49 each; after the bookstore marks them up they'll probably sell for well over $50, making them prohibitively expensive (if I hadn't been able to give each student a free transmitter this semester, I wouldn't be using the system).
  • Student ease-of-use: I spent more than 20 minutes in lecture today trying to show the students how to use the system and ask my first two questions. Students have to enter both a session ID and a student ID to use the system; many students had trouble doing this.
  • Software: Interwrite's software was bad enough that I complained about it in this post, but Qwizdom's (Qwizdom Interact v2.1) is far worse. In fact, I'd say it's awful. I spent hours this week attempting to beat it into doing an extremely simple task, and even with tech support's help it still failed.
Qwizdom's Software:

By far the largest negative about Qwizdom's system is the software (though the price of student remotes is also far too high). In-class response system software has two basic tasks: to facilitate asking questions during a lecture (or other meeting) and then collate the data gathered and display it in a graphical form in class (e.g., a graph of student responses) and in a textual form after class (e.g., transfer it to a grade book or allow it to be exported to be analyzed by other programs). Qwizdom's software does none of these well.

The first problem with Qwizdom's software came when I attempted to try it out in my office. It became apparent that if I wanted to record the answers each transmitter submitted (and associate them with some ID number) then each user of the transmitter needed to log in with a user ID. Each user had to have an ID created in the software before the user could log in. The software does have the ability to import data from csv files, but only in a very defined format (about a 10 column-wide spreadsheet, one of whose required columns is gender), and thus I had to convert my roster into their format.

The requirement to have each user of the system already in the program's database before they can use the system is extraordinarily annoying. First, it absolutely prevents use of the system in the first week or two of the semester, as students may not even be officially enrolled in the class, and thus instructors wouldn't know to add them to the system. Second, it means that instructors have to tediously enter student data into the system. It would be much better if users could key in their ID numbers and names whenever they joined a session that they weren't already users in.

There is a way to allow an audience to not have to log in with user IDs, but if this method is used, each transmitter is assigned an (apparently) random ID number; this ID number varies between sessions, meaning that there is no way to track each student's use of the system and thus no way to evaluate student performance.

As a side note, Interwrite's system gets around this entire user ID problem by assigning each transmitter a factory-specified unique ID number that is always stored along with the answer the transmitter submits (so even if you never know who was using the transmitter, you can at least track that transmitter's answers between different sessions).

But getting student IDs into the program was only the first headache. After I did that, my next goal was to figure out how to ask a quick question in lecture. To give a little more background, I have embedded my in-class response system questions into my PowerPoint lecture slides; I go through around 25 to 35 slides per lecture, with maybe 2-5 question slides. This adds up to more than 800 slides over the course of the semester and more than a hundred individual questions. In other words, I've put a lot of work into my PowerPoints, and I'm not about to change programs anytime soon. But that's exactly what Qwizdom's software mandates - their program is designed solely to ask questions that users have presented using their in-program presenter (a complicated-looking clone of PowerPoint). To ask questions you have to make a slide for every question, and specify on that slide exactly what the answer is.

The software does have the ability to ask spontaneous questions that haven't been entered in advance (which eliminates the requirement to use their presenter program), but using this option has its own problems. First, the program doesn't store spontaneous question responses in the gradebook, and doesn't make them available anywhere for review. The only place the responses are stored is a csv file in the program's system directory tree, but even in that file the responses have been separated from the student IDs. And, even worse, when the spontaneous question option is used, the program's control window (which takes up the top 1/6th of the screen) is hard-coded to show up in front of all other Windows' windows, including full-screened PowerPoint slideshows. In short, the spontaneous question portion of the program is functionally useless.

The only solution I could come up with to allow me to ask the questions built into my PowerPoint lectures was to create a dummy presentation in Qwizdom's system with blank questions in it. I showed my PowerPoint slideshow in front and ran Qwizdom's software in the background, but even this worked poorly. Qwizdom's software wanted to run full-screen; when I shrunk it down so it only filled half the screen, the font in Qwizdom's screen distorted awfully, and the response graph was created off-screen. So, what I had to do today in lecture was ask the question in PowerPoint, then alt-tab out to Qwizdom's screen, un-maximize Qwizdom's window and resize it so that it only filled half the screen, wait for students to respond, re-maximize Qwizdom's window, click on a menu item to show the student response graph, click on another menu button to close the response graph, and finally alt-tab back into PowerPoint to continue my lecture. Clunky, slow, and stupid.

Even more hilariously, Qwizdom has a built-in timer that counts down when you start a question; when the time reaches 0 a big "Time's up!" gray window pops up for a few seconds. However, students can still answer the question after "Time's up!", and the time is not adjustable at all during the presentation (it's set only when the entire presentation starts); there is no way to pause, extend, or otherwise change the time per slide during a show. I asked Qwizdom's tech support about this and they reported that it was just a dummy timer that I should ignore. Useless.

In contrast, while I still had to run Interwrite's software in the background behind my PowerPoint slideshows, it didn't require me to make a dummy presentation, didn't need to be un- and re-maximized for each question, had an extremely easily adjustable timer (time for each question could be adjusted by 15-second increments both before a question was asked and while the time was counting down), and the response graph auto-popped up when the timer hit zero.

Even after I clunked my way through the lecture, I still ran into one last problem: the software requires that each question have one right answer worth some set value of points and some wrong answer worth fewer points. Since I want every student to get a point for the question even if they didn't get the right answer, this means that I can't even use my grading scheme with this software. The program did allow me to leave the answer to a question blank (not set), but whenever I did that the question became worth 0 points, and so the gradebook showed 0's for every student in my course, regardless of how many questions they answered. The program won't even tell me how many questions each student answered.


On the surface, Qwizdom's system appears to be extremely polished. The hardware looks very modern and is very functional. However, Qwizdom's software is atrocious. The software might be OK if you were doing some pre-programmed corporate spiel and didn't mind using Qwizdom's built-in presenter, but it utterly fails when presenting with PowerPoint or whenever you want to do anything even slightly out of the ordinary.

In fact, this week's experiences have been so bad that I have no hesitation whatsoever in giving Qwizdom's product my first-ever Radagast Stamp of Disapproval.


Radagast said...

Importing comments:

Matt Davy
Interesting as I use Interact for one of our clients. Have no problems, in fact nearly all the issues addressed in the blog are simply untrue for this software. Whether this is due to upgrades I can't say. But I have had no problems converting previously created powerpoint question slides into Interact question slides (takes 1 button click) you do not have to assign anything to the remotes, you can simply turn them on and go if you want.
Im using ver
December 17, 2008, 7:28:48 AM PST – Like – Reply

Heath Carter
I am sorry to hear you spent a lot of money on something that has caused such a problem. I research and market SRS to many schools and I have used them all. If you are looking for the best on the market you should look into the Dukane A-Click SRS and their Media Control Software. I in-service this system and it is revolutionizing the classroom. It combines easy to use and unlimited power over the classroom. Full numerical pad, positive-negative numbers, fractions, decimals, a large 4-line LCD Screen, Durable radio buttons, and CHECK THIS OUT...3 programmable buttons you can do whatever you want with.

Shoot me an email if you are not already ruined on the SRS idea and I can tell you more. Please people I urge you to do the research before you buy.
May 13, 2008, 11:39:24 AM PDT – Like – Reply

Jack Furr
Having used Qwizdom for many years (starting with the old square "clickers" and having finally weaned myself off the old software (Qwizdom TA) and to the Interact software. I have to say that it works well in the K-12 setting it was designed for (I teach HS science 20+ yrs). I have not gotten my hands on the Actionpoint software yet (hint here Darren but I find the Interact software quite useable. My students ask to use it (qwizdom) daily and get easily bored with the other systems I have tried with them. As with anything really good, it takes time to learn a new system, and more time to learn it well. Just my two cents.
May 2, 2008, 6:08:21 PM PDT – Like – Reply

Action Point is all I have been using for the past 3 years. I use Qwizdoms native software for the games, but any PowerPoint you have can easily have questions inserted into it, thus making it extremely easy to update your current .ppt to include interactive questions with full reporting functionality. One problem I had was after running problem free for better than a year a recent updated ActionPoint version designed for use with Office07 lists every slide in the presentation listed as a question on the report thus skewing the %. I was assured this will be fixed in the next version. All in all I am a believer. If you would like more info on my experience drop me an email.
July 20, 2007, 12:01:49 PM PDT – Like – Reply

Henry Cordy-McKenna
Qwizdom sucks hard! The handsets are good but the software is awful. They have developed ActionPoint which is looking promising but they take an absolute age to develop new updates.
July 17, 2007, 12:57:14 PM PDT – Like – Reply

Darin is indeed correct; I have not tried out Qwizdom's new software. The main reason for this is that my campus has now standardized on a single system, and thus I no longer have the opportunity to try out other systems in my classroom. Additionally, taking up a manufacturer's offer for a demo would be awkward, given that I'm a pseudonymous blogger.

So, long story short, I haven't given either Qwizdom's or eInstruction's software a try recently, so can't say much about them. And, given that I didn't have anything new to say on the topic, I felt the best course of action was to remain silent.

That said, trying to figure out a way to do a side by side comparison of the various manufacturer's would be neat ...
April 18, 2007, 11:12:25 PM PDT

Radagast said...

Importing comments:

Darin Beamish
No, it is eight months later and the moderator of this blog has not taken us up on our offer to try out the Actionpoint software (of course there is no obligation to do so). This post is based on software designed for a different market (k-12) and does not relate to any current Qwizdom software program or pricing. Additionally, in independent reviews by K-12 schools and Technology and Learning Magazine, Qwizdom was chosen as 'the system of choice by both teacher and students'. In fact, Qwizdom won the Technology and Learning awards for the Q4 remotes and Interact software that is mentioned in this post. We are more than happy to have our system compared directly to any other system.
April 13, 2007, 9:08:35 PM PDT – Like – Reply

Carlton Watson
I wanted to know if you took the folks at Qwizdom on the offer to use the Actionpoint software if so how was it?
August 27, 2006, 8:06:32 AM PDT – Like – Reply

Darin Beamish
I see multiple sales reps from eInstruction posting here so I thought I would weigh in from a different side....

Have you tried Qwizdom's Actionpoint software? It integrates with Powerpoint presentation software and is a much simpler software option for higher ed than the Interact software (or any other non-Powerpoint application for that matter). I would be happy send you a free copy if you are interested in trying it out.

Also, you noted that students needed to input both a student ID and session ID. That is not correct. You only need one or the other. The keypads can be associated to student IDs so permanently (by initial student input on the keypad...not through a website association process) and after than they only need to input a session id (ie a room number). The tech support can walk you through any of these features if you are interested.

Let me know how I can help.
July 17, 2006, 9:14:00 PM PDT – Like – Reply

Dustin K. Frank
Would still love to have you test out our equipment and give us some feedback. Let me know if you are interested once you get back from vacation.
June 12, 2006, 8:58:47 AM PDT – Like – Reply

I might suggest trying to sell it on the net (e.g., eBay); other schools are probably still using the system.
April 25, 2006, 8:38:20 AM PDT – Like – Reply

I agree with you completely. I had to buy one this past semester and use it in one of my classes, but we have problem after problem with them and now my teacher has decided not to use them again. What am I supposed to do with it? I can't sell it back to the bookstore and I can't sell it to any students taking the class in the future.
April 25, 2006, 7:46:07 AM PDT – Like – Reply

Dustin K. Frank
Whenever things slow down and if you are interested, we would like for you to test our software and give us your feedback. You seem very knowledgeable and we would appreciate your constructive evaluation of CPS versus the competition.

I look forward to hearing from you.
March 30, 2006, 9:52:16 AM PST – Like – Reply

Thanks for the responses; given how things are going, I probably will be shopping around later this year.

I'd never heard about the lifetime purchase option for eInstruction; that changes the cost/benefit equation some. Oddly, I haven't been able to find anything about the lifetime purchase on eInstruction's website; it only lists the cost as $15/term.

There is one thing I do want to clear up: the reps who were spreading misinformation were not official eInstruction reps; they were textbook reps trying to sell eInstruction remotes bundled with new textbooks.

[And, for those readers who aren't familiar with these systems, note that the two comments above are written by people working for a company that competes with Qwizdom.]
March 15, 2006, 10:41:05 PM PST – Like – Reply

Radagast said...

Importing comments:

Luke Tevebaugh
Hi Radagast,

I'm Dustin's counterpart in Florida. You will do very well to work with Dustin if you should choose to give CPS a try. He's a good honest guy. He won't pretend to know everything and will find the answer you need asap. Check out my website if you are interested in seeing some of the support many of our users have taken advantage of. I built these videos and eInstruction has also provided a very nice set of training videos at Holler if I can do anything!
March 15, 2006, 9:15:10 AM PST – Like – Reply

Dustin K. Frank
I'm sorry to hear that you were not properly informed about the Higher Ed CPS model. I would love to share with you more about CPS and even allow you an opportunity to try it out. Our software will work great with your PPT files and is very easy to use. We do offer an RF model which is very nice, does allow for numerical answers as well as multiple choice. We also have a new remote available for next fall that will have a fraction and symbol button.

You are correct in that we do charge a fee per semester for activating our remotes but we offer mutliple plans depending on the age of the student. Older students may choose to go by semester since they only have 1,2 or 3 left but a younger student could activate for a lifetime which runs about $40-$50 for a lifetime activation. Not sure why that was not shared with you but I would love for you to test out our software and share your feedback. You sound very aware of the programs that are out there and I feel you would like our software and it's user friendliness.

If you are interested, I'd love to speak more with you about testing our software against the others you've experienced.
March 14, 2006, 7:39:28 PM PST – Like – Reply

I've looked at eInstruction's equipment, but have never used it.

The biggest complaint I have about eInstruction is that they use somewhat deceptive sales practices; they sell the transmitters for almost nothing (less than $15, I think), but then charge a $15 "activation fee" every "term" the students want to use the transmitter. So, if a student ends up using the transmitter for six semesters in a row, they end up paying $90 for the privilige of using the transmitter they bought.

What really irks me is that I've had multiple sales reps "forget" to tell me about the activation fee when describing the system's costs.
February 18, 2006, 6:17:01 PM PST – Like – Reply

Have you tried the classroom performance system. It is awsome. They have the simple remotes with the keys and also a remote with a lcd screen on the remote. I have seen all the other remote systems by this is the best I have ever found.
February 18, 2006, 4:06:32 PM PST