Categorical analysis of cross-cultural survey data: Effects of clustering on chi-square tests

Abstract

This paper examines the effects of clustering of societal units on chi-square goodness of fit and Independence tests using a variety of categorical variables drawn from the Standard Cross-Cultural Sample (SCCS). Analytical results and the corresponding estimation equaaw for sample design effects are gven for both types of tests. Cluster sample de&n effects for individual SCCS variables are generally larger than unity, indicating that the risk of type I errors are greater, often considerably so, than the five percent level usually assumed by rcsear&ers. Collapsing across categories because of small sample sizes intensilies the problem. De-sign effects for two-way contingmcy tables reveal similar problems of highly Xated type I error rates for the usual chi-square tests of independence These empirical findings suggest that the extensive use of chi-square tests with cross-cultural data over the past few decades has incorrectly lead to the rejection of many null hypotheses that are in fact true. Deflating the observed chi-square values by the average of the generalized design effects, or by the average of the design effects for individual cell counts, was found to reduce type I error rates closer to the assumed levels.

Author-supplied keywords

Cite this document ^(BETA)