LITMIR.BIZ

CHAPTER 1 Introduction to Critical Appraisal

      Critical appraisal is the process of assessing the strengths and weaknesses of published studies. It involves a structured and rigorous evaluation of all the stages of the research, from design to analysis and interpretation. All studies have imperfections, so the question is not if there are flaws, but whether they are likely to be important. Critical appraisal is a method of systematically examining research studies to determine the worth of their findings.

The aims of critical appraisal

Critical appraisal assesses two issues: bias in study design or conduct, and the value that the findings have for clinical practice. Bias occurs when systematic errors distort the results of a study. For example, a clinical trial should provide a good estimate of the benefit of a treatment, but weaknesses in design or conduct could produce a misleading estimate. There are many types of bias and these are explored through a series of questions that examine the main deficiencies in research.

      The value of findings is the extent to which they will provide benefits for patients or the general population. Many factors contribute to the value of research findings. For a clinical trial, this would include the size of the treatment benefit, the importance of the outcome measure to patients, and whether the patients to be treated in a clinical setting are similar to the participants in the study. As with bias, there are sets of questions that assess value.

Three stages of critical appraisal

      The critical appraisal of a paper is conducted in three stages: identify the research design, select the appropriate appraisal tool for that study, and apply the tool to assess the bias and the value of the research. This section introduces these stages, with subsequent chapters explaining in detail how to carry them out.

      Identify the research design

      Research studies can ask different types of questions. Some may be concerned with the effectiveness of treatments; others may investigate the likely prognosis of an illness. The research design should be matched to the research question. To evaluate treatment effectiveness a randomised controlled trial would be used, whereas prognosis would be investigated by a cohort study. The other designs are surveys, case–control studies, cohort studies that test interventions, and systematic reviews. Chapter 3 of this book provides guidance on identifying the correct design.

      Select the appropriate appraisal tool

Critical appraisal tools have been developed to enable a forensic evaluation of research studies. They provide lists of questions that probe all aspects of the methods used and the results obtained. Judgements can then be made about the bias and value of the research. This book presents appraisal tools for the six most common research designs in medical research (Chapters 6–11).

      A key feature of the questions on bias is that they distinguish between critical and important sources of bias. For each research design there are a few crucial steps which, if conducted poorly, are very likely to lead to misleading findings. Flaws in the other steps are less likely to cause serious bias. Only when several of these flaws occur are serious concerns raised.

      Value, the usefulness or worth of a finding to individuals and to society, is more difficult to assess. Assessing the potential for improving health is usually straightforward, but often some pieces of information, such as the importance of the findings to patients, are not available. Decisions about value can use information from several sources and require judgement to balance many complex issues.

      Apply the tool to assess the bias and the value of the research

      The critical appraisal questions can only be answered by identifying key pieces of information from relevant sections of a paper. Chapter 2 provides a simple method for extracting the key pieces of information from a research paper. Careful reflection on what the questions reveal enables an assessment of the quality of the research study.

      The appraisal questions presented in this book are structured to lead to one of three answers: yes, no, or not enough information to decide. Selecting the appropriate option is not a simple tick box process. Critical appraisal involves pausing to think about the implications of each limitation, and whether it could seriously affect the interpretation of the study's findings. Sometimes there will be insufficient information to answer a question. As decisions in medicine affect patients, it is usually better to be cautious in drawing conclusions.

      In summary, critical appraisal assesses the bias and the value of research studies. By identifying the research design, the appropriate appraisal tool is selected. It asks probing questions about the design of the study and the way it was conducted, analysed, and interpreted. Chapter 2 describes a simple method for extracting the information from published papers to facilitate critical appraisal.