Explain common biases in observational studies
Understand the need for critical appraisal of observational studies
Understand general criteria to assess observational studies
Appraise design, methods, and analytical approaches in observational studies
Evaluate results of observational studies with results from randomized controlled trials
Observational studies have played an important role in medical literature. Although well-designed and conducted randomized controlled trials (RCTs) are the gold standard, they may not be feasible in many situations. For instance, rare events or events that take long time to develop are often examined by observational studies because of the constraints of time and resources needed to conduct an RCT to evaluate such events. Observational studies also have a unique role in research into the harms of medical interventions. For those events, conducting RCTs would not be ethical because it is unethical to impose a known harm on anyone. In recent years, observational studies have played an increasingly important role in comparative effectiveness research, with the additional benefit of providing more “realistic” effects in daily medical practice.
Despite their wide application in medical literature, the quality of observational studies can vary largely, affecting both internal and external validity of the findings. Consequently, it is important to be able to critically review findings from observational studies instead of blindly accepting and interpreting the finding as evidence. This requires familiarity with observational study designs, their inherent limitations, and critical evaluation of the findings. This chapter will first provide a brief review of the biases and confounding arising from observational studies that may threaten the internal validity of their findings. Next, it will introduce formal criteria for critical assessment of various aspects of an observational study, which include the study design, methods, and analytical approaches. This chapter ends with suggestions on how to compare results of observational studies with RCTs when discordance in findings is encountered.
Like clinical trials, observational studies are “susceptible to error, bias, and confounding that may lead to erroneous finding(s) and/or conclusion(s) in the study.”1 Because observational studies are not randomized, bias caused by confounding factors is a particularly important threat to the integrity of observational studies and should be carefully controlled for and/or examined in the study. The following discussion briefly reiterates different types of biases and confounding encountered in observational studies.
Bias results from systematic errors in the way study subjects are selected, measured, and analyzed. Systematic errors will result in inaccurate estimates which lead to invalid conclusions. Similar to clinical trials, bias can be introduced at every stage of an observational study and it would be impossible to discuss all potential biases in detail. The ...