Paper versus digital data collection for road safety risk factors: Reliability comparative analysis from three cities in low- And middle-income countries

Abstract

Background: Rapid advances in mobile technologies and applications and the continued growth in digital network coverage have the potential to transform data collection in low- and middle-income countries. A common perception is that digital data collection (DDC) is faster and quickly adaptable. Objective: The objective of this study was to test whether DDC is faster and more adaptable in a roadside environment. We conducted a reliability study comparing digital versus paper data collection in 3 cities in Ghana, Vietnam, and Indonesia observing road safety risk factors in real time. Methods: Roadside observation of helmet use among motorcycle passengers, seat belt use among 4-wheeler passengers, and speeding was conducted in Accra, Ghana; Ho Chi Minh City (HCMC), Vietnam; and Bandung, Indonesia. Two independent data collection teams were deployed to the same sites on the same dates and times, one using a paper-based data collection tool and the other using a digital tool. All research assistants were trained on paper-based data collection and DDC. A head-to-head analysis was conducted to compare the volume of observations, as well as the prevalence of each risk factor. Correlations (r) for continuous variables and kappa for categorical variables are reported with their level of statistical significance. Results: In Accra, there were 119 observation periods (90-min each) identical by date, time, and location during the helmet and seat belt use risk factor data collection and 118 identical periods observing speeding prevalence. In Bandung, there were 150 observation periods common to digital and paper data collection methods, whereas in HCMC, there were 77 matching observation periods for helmet use, 82 for seat belt use, and 84 for speeding. Data collectors using paper tools were more productive than their DDC counterparts during the study. The highest mean volume per session was recorded for speeding, with Bandung recording over 1000 vehicles on paper (paper: mean 1092 [SD 435]; digital: mean 807 [SD 261]); whereas the lowest volume per session was from HCMC for seat belts (paper: mean 52 [SD 28]; digital: mean 62 [SD 30]). Accra and Bandung showed good-to-high correlation for all 3 risk factors (r=0.52 to 0.96), with higher reliability in speeding and helmet use over seat belt use; HCMC showed high reliability for speeding (r=0.99) but lower reliability for helmet and seat belt use (r=0.08 to 0.32). The reported prevalence of risk factors was comparable in all cities regardless of the data collection method. Conclusions: DDC was convenient and reliable during roadside observational data collection. There was some site-related variability in implementing DDC methods, and generally the productivity was higher using the more familiar paper-based method. Even with low correlations between digital and paper data collection methods, the overall reported population prevalence was similar for all risk factors.