Evaluating and comparing tools for mobile device forensics using quantitative analysis

Abstract

In this paper we have presented quantitative analysis technique to measure and compare the quality of mobile device forensics tools while evaluating them. For examiners, it will provide a formal mathematical base and an obvious way to select the best tool, especially for a particular type of digital evidence in a specific case. This type of comparative study was absent in both NIST’s evaluation process and our previous work (Evaluation of Some Tools for Extracting e-Evidence from Mobile Devices). We have evaluated UFED Physical Pro 1.1.3.8 and XRY 5.0. To compare the tools we have calculated Margin of Error and Confidence Interval (CI) based on the proportion of successful extractions from our samples in different scenarios. It is followed by hypothesis testing to further strengthen the CI results and to formally compare the accuracy of the tools with a certain level of confidence.