Developing and Applying a Framework for Assessing the Research Quality of Qualitative Project Methods in the EU Project SECUR-ED

Abstract

Qualitative research plays a vital role in political development and in the design of statutes and directives. Consequently, ensuring the quality of this research is important. However, the current literature on evaluation of quantitative and qualitative research does not reflect this importance, and we have identified a need to establish guidelines for evaluating qualitative research projects for quality. Therefore, and based on existing research, we have developed a framework for assessing the research quality of large complex projects using qualitative methods. In our study, as presented in this article, we operationalize and apply the framework to evaluate six specific methods in the large European research project, Secured Urban Transportation—A European Demonstration (SECUR-ED); each method is assessed according to the quality criteria of “transferability,” “systematic design/reliability,” and “transactional validity.” Overall, we find that half of the SECUR-ED project methods demonstrate thorough documentation and transferability, and that half of the methods lack consistent usage and therefore score low on both reliability and validity. We also find that one method, the capacity mapping matrix, scores high on all quality parameters. Accordingly, we suggest that future European Union (EU) projects replicate the documentation efforts demonstrated in relation to several of the SECUR-ED methods, and consider the capacity mapping matrix as “best practice” standard. We conclude that the framework represents a novel approach to quality assessments of qualitative project methods across research topics and contexts.