Qualitative research methods in health technology assessment: A review of the literature

Abstract

Background: Qualitative research involves the collection, analysis and interpretation of data that are not easily reduced to numbers. These data relate to the social world and the concepts and behaviours of people within it. Qualitative research can be found in all social sciences and in the applied fields that derive from them, for example, research in health services, nursing and pharmacy. These research methods are not a recent invention but their application in health technology assessment (HTA) or health services research may be novel. In order for commissioners and researchers to utilise these methods and gain valuable knowledge from the results obtained, it is important that they understand the principles of qualitative methods and the way that they may be used to set benchmark standards. Objective: The objective of this review was to examine the nature and status of qualitative methods in relation to their potential uses in HTA. Methods: The search tools often used for systematic reviews were not appropriate for this review as it would be necessary to cover the equivalents of MEDLINE in a range of disciplines and applied fields, many of which do not have databases of comparable coverage. In addition, important methodological writing in the field of social science started long before indexing for computer databases, and much of the most significant work has been published in books rather than journals. Having set the boundaries and organised the categories for this review, therefore, the authors read as widely and as comprehensively as was feasible in the time available. The authors have compared different researchers' approaches to the same issue and examined the work of authors who offer different perspectives. Perspective: Idealists versus realists. Qualitative work is often identified with idealism while quantitative work is identified with realism. However, most qualitative researchers accept that there is an objective, material world, as do realists, but question our ability to know this directly. In the social sciences, what people perceive or believe is the basis of their actions rather than what an impartial observer might think was actually true. Qualitative versus quantitative methods: The goal of all research in HTA should be to establish knowledge about which we can be reasonably confident, and to provide findings that are relevant to policy makers and practitioners. Therefore, decisions about whether qualitative or quantitative methods (or a combination of both) are most appropriate to a particular research problem should be made on the basis of which approach is likely to answer the question most effectively and efficiently. Qualitative methods are useful in the exploratory stages of a research project, where they will often help to clarify or even set the research question, aid conceptualisation and generate hypotheses for later research. Qualitative methods may also be used to interpret, qualify or illuminate the findings of quantitative research and to test hypotheses. Qualitative research is particularly useful to policy makers and planners by providing descriptive information and understanding of the context in which policies will be implemented. Sampling and generalising: In sampling decisions in qualitative research, pragmatic considerations should be integrated with sampling in a systematic way just as in quantitative research; opportunistic sampling should be avoided if possible. The emergent nature of qualitative research means that sampling decisions need to be made throughout the study; such decisions should again be systematic and principled. Where the aim is to build or develop theory, subjects should be selected accordingly; such theoretical sampling makes use of existing theory to make predictions, and then seeks subjects who allow the researcher to test the robustness of such predictions under different conditions. Methods of qualitative research: Participant observation. Participant observation can be used to study the impact of technologies upon the routine functioning of the setting in which they are to be implemented. Participant observation may also be used to review health technologies currently in practice, and has the potential for uncovering the process through which professional inputs are transformed into patient/client outcomes thereby identifying opportunities for modifying current practice to improve outcomes. Interviews: Qualitative interview techniques are used, particularly in exploratory research, to study the range and complexity of ideas and definitions employed by individuals and groups involved in the implementation of health technologies. Both qualitative and quantitative interviewing share the same fundamental problem, however, in that they rely upon interviewees' reports and such reports are necessarily constrained by the context in which they are collected. Written records: The analysis of written records has an important contribution to make to our understanding of the processes and consequences associated with new technologies. In addition, documents such as health diaries may provide important data on the implementation of health technologies. Conversation analysis: The techniques of conversation analysis can provide detailed data on the impact of new technologies upon healthcare settings, the organisation of professional work and the interactions between health professionals and patients. Research ethics: The same ethical principles apply to qualitative and quantitative research in HTA. The mechanical application of ethical codes developed in the context of biomedicine may be unduly constraining in qualitative research and may distract from those ethical risks which are specific to qualitative research. Covert research will rarely, if ever, be justified in HTA. Such research is likely to be a betrayal of trust and a gross invasion of privacy. Assessment of qualitative research: The same assessment criteria of validity and relevance are appropriate for both qualitative and quantitative research in HTA. Relevance: The relevance of HTA research is related to its potential generalisability to groups or settings beyond those studied. Given that most qualitative research is based on a single case or only a small number of subjects, the generalisability of qualitative research is achieved by the generation of theoretical statements, which may guide policy makers but remain to be tested through application in other contexts. Data handling: HTA commissioners should look for evidence that applicants intend to use systematic methods for coding and handling their qualitative data and that methods proposed for analysing such data are appropriate to the research objective. Computerised analysis packages for qualitative data offer an efficient way of handling qualitative data sets and may improve the rigour of the analysis by facilitating searches for falsifying evidence. However, such programs should be used only as a means of facilitating the analysis process rather than carrying out the analysis, which depends upon the theoretical sensitivity of the analyst. Judgements about the validity of research depend upon being able to form a judgement of the research process. Researchers therefore need to provide a detailed record of their methods. Given the non-standardized nature of qualitative research, such records are likely to be more elaborate than in reports of quantitative research. The trustworthiness of data analyses is enhanced where researchers can demonstrate that they have considered alternative plausible explanations for their data. The validity of research findings is enhanced where the researchers increase our understanding of all members in a setting and do not present one-sided accounts. Confidence in the validity of findings is increased where there is evidence of researcher sensitivity to the ways in which the data have been shaped by the researchers' presence. While the practices of respondent validation and triangulation may increase the comprehensiveness of a study, neither can be treated as tests of the validity findings. Conclusion: There are strengths and limitations to qualitative approaches as there are to quantitative methods. However, where qualitative research is conducted properly and data analysed thoroughly, this approach can provide valuable information on the implementation and impact of health technologies on both health professionals and patients.