Investigation of Two Preliminary Analysis-Altering Elements: Ordinate Scaling and DPPXYR

Abstract

The purpose of this pre-registered study (Peltier & McKenna) was to conceptually replicate if the truncation of the ordinate and DPPXYR increased analysts’ estimation of a functional relation and magnitude of treatment effect. Visual analysts (n = 27) evaluated eight data sets reporting null (n = 2), small (n = 2), moderate (n = 2), and large (n = 2) effects. Each data set was graphed six times with manipulations of the ordinate and DPPXYR, resulting in 48 ABAB graphs. We estimated two separate three-level mixed effect models with variations nested in datasets and nested in participants to evaluate the impact of graph characteristics for (1) confidence in determining a functional relation and (2) the estimated magnitude of the treatment effect. We included ordinate scaling and DPPXYR at level 1 and graph effect size at level 2, including all interactions. Overall, graph manipulation consistently did not impact confidence in a functional relation. Results suggest mixed findings for graph manipulation on the estimated magnitude of the treatment effect. Findings will be couched in current literature and recommendations for graph construction and future research will be discussed.