How Many Participants Do You Need for an Open Card Sort? A Case Study of E-commerce Websites

Abstract

Open card sorting is the most used method for developing user-centered information architectures. One important question for every HCI method is how many users to involve. Existing studies that address this question for open card sorts have involved trained professionals sorting content items of rather specialized domains. In addition, they employ data analysis approaches that might decrease the confidence one can place on the reported findings. This paper investigates the minimum number of participants for open card sorts performed on a general public website domain (e-commerce). In specific, it involves 203 and 210 participants sorting content items of two real-world e-commerce websites. Results from all the participants were compared with those of different-sized and randomly selected samples of the participants. It was found that 15 to 20 participants is a cost-effective way to obtain reliable open card sort data for general public websites.