Question 780:

Asked on July 14, 2009

Tags: Six Sigma , Usability

1

Answer:

No answer provided yet.It turns out there really isn't much out there on six-sigma and usability as you've found.

1. The best place to start is the paper a colleague and I wrote a few years ago on usability and six-sigma. It deals with many of the question's you're raising and walks though the calculations along with the type of data. The link to that paper is here: Sauro, J. & Kindlund E. (2005) "Making Sense of Usability Metrics: Usability and Six Sigma" in Proceedings of the Usability Professionals Association (UPA 2005) Conference Montreal, Canada.
   
   The only other book which gets close to this topic is Measuring the User Experience by Tullis and Albert (they references my work on the topic) but the book talks more about basic issues in usability measurement.  See also Sauro, J. & Kindlund E. (2005) "A Method to Standardize Usability Metrics into a Single Score." in Proceedings of the Conference in Human Factors in Computing Systems (CHI 2005) Portland, OR for more on six sigma and usability.
   
   
2. Tasks are generally activities which make up a meaningful unit of work--something a user would recall doing. They are more than just basic units of work like pick an item from a menu. Some examples of tasks would be: booking a rental car, entering an expense in an expense reporting system, logging hours in a time-reporting system, adding a user to a security system, entering a journal entry for accounting, logging a support request etc.
3. Typically tasks are measured not in keystrokes per minute, but just in minutes. So for all the tasks listed above, the measure of efficiency is time to complete the task (in seconds or minutes).  I'm assuming you're referring to statistical confidence levels. The confidence level is separate from the unit of measure. Typically we work with 90 or 95% levels of confidence in generating confidence intervals and using t-test. 
4. The confidence level is independent of the tasks completed. You would define your tasks based on what's critical to the customer. If a customer/user needs to withdraw money from an ATM in 30 seconds, then this would be 1 task, measured in seconds, with a specification limit of 30 seconds.  Often when we define usability tests, we identify 5-10 tasks which account for a reasonable amount of functionality. For example, when testing the usability of an expense reporting system, we have the user: enter an expense report, duplicate an expense report, enter a more complicated expense report with mileage, modify the preference locations. These would be the tasks.