Usability Reflections: Summative vs. Formative Usability Testing
This week’s assignment was to justify the suitability of using either a Summative or Formative Evaluation methodolody for conducting usability testing given a scenario where a project has already entered the development phase.
Summa-what?
The Usability Body of Knowledge does a better job of defining these terms than I could:
Formative Evaluation
A type of usability evaluation that helps to “form” the design for a product or service. Formative evaluations involve evaluating a product or service during development, often iteratively, with the goal of detecting and eliminating usability problems.1
Summative Evaluation
Usability evaluation of a complete or near-complete design under realistic conditions that can be used to determine if the design meets specific measurable performance and/or satisfaction goals, or to establish a usability benchmark or to make comparisons.2
The Takeaway
General understanding of the what, absolute confusion about the when
The line of understanding definitely blurred around trying to figure out the appropriate timing of a formative evaluation. The textbook made it seem like it’s something that’s done iteratively in the Design Phase to test multiple solutions to one or more known problems, ala “here’s a bunch of ideas for fixing ${problem_name}, test, fix, repeat, this one works best, kthxbye”. In other words, you already know the problem and are testing out a bunch of solutions. Then, other resources made it seem like it’s something that can be done at any point in the design and development phases to find problems. I’d have thought that by the time you’re underway with development, you’re kind of past the here-are-five-copies-of-the-same-interface-please-tell-me-which-one-works-best phase.
In the end, I went with the summative evaluation methodology. I figure that since development had already begun, it would make more sense to test the users with minimal interference and determine what needs to be fixed.
Ex post factoid: I’m already kind of doing small-scale formative evaluations
On an interesting note, I did learn that (very informal) formative evaluations are already part of my current workflow. When building new UI features/components that don’t have existing designs, I tend to make multiple prototypes and pass them to other members of the product team for their input.