Variability, Validity and Operator Reliability of Three Ultrasound Systems for Measuring Tissue Stiffness: A Phantom Study

Abstract

Introduction Ultrasound elastography is a method of measuring soft tissue stiffness to detect the presence of pathology. There are several ultrasound elastography devices on the market. The aim of this study was twofold. Firstly, to determine the validity of three different ultrasound systems used to measure tissue stiffness. Secondly, to determine the operator reliability and repeatability when using these three systems. Materials and methods Two observers undertook multiple stiffness measurements from a phantom model using three different ultrasound systems; the LOGIQ E9, the Aixplorer, and the Acuson S2000. The phantom model had four cylindrical-shaped inclusions (Type 1-4) of increasing stiffness values and diameter embedded within. The background phantom stiffness was fixed. The mean, standard deviation, and coefficient of variation (CV) were calculated from measured stiffness readings per diameter per inclusion. Intra-observer variability was assessed. The validity of the measured stiffness value was assessed by calculating the difference between the measured elasticities and actual phantom elasticities. Bland-Altman plots with limits of agreement were used to display the inter-observer agreement. The intraclass correlation coefficients (ICC) were used to measure intra-observer, inter-observer, and inter-system reliability. Results Each observer undertook 1020 measurements. All three systems generally underestimated the stiffness values for the inclusions; the higher the actual stiffness value, the more significant the underestimation. The percentage difference between measured stiffness and actual stiffness varied from -79.1% to 12.7%. The intra-observer variability was generally less than 5% for observers using the LOGIQ E9 and the Aixplorer systems but more than 10% over the stiffer inclusions (Types 3 and 4) for the Acuson system. There was 'almost perfect' intra-observer reliability and repeatability for both the LOGIQ E9 and the Aixplorer systems; this was 'moderate' for the Acuson system over specific inclusions. For all systems, there was 'almost perfect' inter-observer reliability and repeatability between Observer A and Observer B. The inter-system reliability and repeatability were 'almost perfect' between the LOGIQ E9 system and the Aixplorer system but 'poor' and 'moderate' when the Acuson system was matched with the LOGIQ E9 system and the Aixplorer system, respectively. Conclusion This study has demonstrated that the Acuson, LOGIQ E9, and Aixplorer ultrasound systems have low variability, high reproducibility, and good intra-observer and inter-observer reliability when used to measure tissue stiffness. However, they all underestimated the stiffness values during this in vitro study. This study also revealed that not all ultrasound systems are comparable when measuring tissue stiffness, with some having better inter-system reliability than others. Ongoing standardization of technology is required at the manufacturer level.