Usability Reflections: Screening and Tasking Test Candidates
This week’s assignment involved conjuring up a number of screening questions and tasks for potential test candidates.
Ultimately, the goal was to use screening questions to narrow the selection pool down to a manageable size, then build a plan of what the selectees would actually be doing.
Takeaway
Big massive assumptions up front
Naturally, my deliverable was rife with some pretty bold assumptions such as:
- Cash is an uncommon payment method
- Store Pickup is an uncommon delivery method
- Special instructions are an uncommon order customization
Back in 2008, one of my duties was to compile reports which my boss would then brief to his own boss. He’d say the exact same thing every time I reported status on something without careful verification:
Make sure we get confirmation on that, because you know what happens when you assume: You make an ass out of u and me!
MGySgt Bosarge (original source unknown)
Master Guns Bo was a hoot and an awesome guy to work for, so I think he wouldn’t mind me making an exception here. For all I know those things might be as common as 50% of all orders, but without access to company and industry metrics, it’s pretty hard to make the call. I can just go off of the cues I see within my own social bubble. But to deliver a coherent submission without a lot of umms and uhhs, I think it’s best to be decisive.
Coming up with good screening questions was a little tough
It’s tempting to chalk this one up to my own social ineptitude, but I think there might be something else about this that makes it harder. There is a conscious balancing act necessary to find questions strict enough to exclude candidates that probably would not provide good insights (e.g., people that are nothing like the end user, or people who will never use the product under any circumstances) and not end up with a test pool of candidates who are genetically engineered to be identical. Then, you combine that with trying to hit an arbitrary number of questions and not have them all be nonsensical. :)
That said, I think this is one of those things that gets easier once you have some idea of what works in general. Iterating over questions asked previously should yield a good baseline that’s a good enough starting point for most projects.
Attachments
