Why and when five test users aren’t enough

  • Woolrych A
  • Cockton G
  • 83

    Readers

    Mendeley users who have this article in their library.
  • N/A

    Citations

    Citations of this article.

Abstract


Nielsen's claim that ``Five Users are Enough'' [5] is based on a statistical formula [2] that makes unwarranted assumptions about individual differences in problem discovery, combined with optimistic setting of values for a key variable. We present the initial Landauer-Nielsen formula and recent evidence that it can fail spectacularly to calculate the required number of test users for a realistic web-based test. We explain these recent results by examining the assumptions behind the formula. We then re-examine some of our own data, and find that, while the Landauer-Nielsen formula does hold, this is only the case for simple problem counts. An analysis of problem frequency and severity indicates that highly misleading results could have resulted when the number of required users is almost doubled.
Lastly, we identify structure and components of a more realistic approach to estimating test user requirements.

Author-supplied keywords

  • usability testing
  • user differences

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Alan Woolrych

  • Gilbert Cockton

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free