Assessing probabilistic reasoning in verbal-numerical and graphical-pictorial formats: An evaluation of the psychometric properties of an instrument

Abstract

Research on the graphical facilitation of probabilistic reasoning has been characterised by the effort expended to identify valid assessment tools. The authors developed an assessment instrument to compare reasoning performances when problems were presented in verbal-numerical and graphical-pictorial formats. A sample of undergraduate psychology students (n=676) who had not developed statistical skills, solved problems requiring probabilistic reasoning. They attended universities in Spain (n=127; f=71.7%) and Italy (n=549; f=72.9%). In Italy 173 undergraduates solved these problems under time pressure. The remaining students solved the problems without time limits. Classical Test Theory (CTT) and Item Response Theory (IRT) were applied to assess the effect of two formats and to evaluate criterion and discriminant validity. The instrument showed acceptable psychometric properties, providing preliminary evidence of validity.