What is the Best Evidence?
Save as PDF
Imagine this: you are in your office scrolling through your unread emails before your patients arrive. An email catches your attention with the subject line “kale consumption decreases glaucoma risk,” so you tap on it. The email opens to an article summarizing a new study that concluded: “Patients who consumed 2 cups of raw organic kale per day were less likely to experience vision loss from glaucoma”. “Fantastic,” you say to yourself. You put your phone in your pocket and head to see your first patient, who happens to be a high-risk glaucoma suspect for whom you monitor yearly. You do a fist pump because you are so excited to give your patient this new, hot-off-the-press insight.
How do you know if this is a good recommendation for this patient?
In our first column of Follow The Science, we discussed the role of evidence-based medicine (EBM) in serving as a decision-making tool for our patients. To summarize, EBM is the use of the current best research evidence filtered through clinical expertise and patient preferences. But how do we know what qualifies as “current best research evidence”?
Back to our imagined study on kale and glaucoma risk. Let’s say you did recommend 2 cups of raw organic kale to your patient that day. Miraculously, the patient complied and consumed all that kale for the next several years. In addition, they never showed glaucomatous progression per their visual fields or ocular coherence tomography despite their intraocular pressures consistently > 24 OU. Can you now say, “I bet it was the kale that kept them from progressing”? Will you now start recommending kale to every other glaucoma suspect or glaucoma patient because of this one instance? Can information from this single case be used as evidence to support the general recommendation to every glaucoma suspect?
To answer those questions, it is helpful to refer to the traditional hierarchy of evidence pyramid. The evidence pyramid is a way of visually representing the validity of different types of evidence. As shown in Figure 1, the bottom comprises case reports, the middle consists of observational cohort studies, and near the top are experimental randomized controlled trials (RCT) with systematic reviews and meta-analyses at the top.1
Figure 1: The hierarchy of evidence pyramid.
What is important about the pyramid as it pertains to appraising and applying evidence for clinical practice is understanding that the information gleaned from lower levels is much more prone to bias and much less generalizable to broad patient populations. Likewise, evidence extrapolated from higher levels of the pyramid is less prone to bias and more generalizable to broad patient populations. There are exceptions to these statements, of course (for example, a poorly designed or industry-biased RCT may not be better than a well-designed observational cohort study), and there is ongoing controversy in the EBM research community as to whether systematic reviews belong at the top.2 Nonetheless, these broad principles should be used to assess all kinds of evidence.
In future editions of “Follow The Science,” we will unpack in greater detail why observational research is more prone to bias and less generalizable to broader populations than, say, the RCT. But for now, let’s think back to our questions about the hypothetical kale study and our single case experience.
You have probably already (correctly) asked yourself, from which type of study was the “kale consumption decreases glaucoma risk” conclusion derived? Additionally, can this type of study tell us if “kale consumption will decrease the risk of glaucoma” in our patient?
A similar study to our fictitious one appeared in 2016 in JAMA Ophthalmology. This observational prospective cohort study (level 3 on our evidence pyramid) concluded, “higher dietary nitrate and green leafy vegetable intake was associated with a lower POAG risk.” 3
Observational studies are fantastic tools for uncovering associations between seemingly disparate characteristics of large groups of study participants. They are excellent tools for observing (hence the classification) insights that need to be further studied and experimented with to determine causality. However, correlations found in observational research cannot definitively prove causation. This is true even in large cohort studies with thousands of participants where complex statistical methods are applied. The only way to definitively prove causality is by using experimental studies with randomization. Although randomization does not eliminate bias that plagues observational research, it significantly reduces it. Again, more on this in future columns.
Therefore, in our hypothetical case and study, both observational studies (one cohort and one single case study), the correlation of kale consumption to the reduction of glaucoma progression does not prove causation. The only way we can honestly tell our patients with glaucoma to “eat more kale for your glaucoma” is if we randomized two groups of people, one to kale consumption and the other to some sort of placebo and followed them over time to compare each group in terms of glaucoma progression. Until then, we can talk to our patients about the interesting association, but we should refrain from recommending it with any degree of certainty.
But our patient is proof that it works, right? In his fantastic book “Bad Science,” Ben Goldacre says it best: “The plural of anecdote is not data.” In other words, one single case (the bottom of the pyramid) cannot be generalized to a broader population. What other variables confounded the results? Corneal thickness, family history of glaucoma, demographics, and overall metabolic health could more likely be correlated with the results (no glaucoma progression) than kale consumption.4
Hopefully, you find, as I have, the practical usefulness of the hierarchal pyramid of evidence. Whether you read an article from this edition of the Journal of Medical Optometry, an industry magazine, or you see a friend post results from a study on social media, please remind yourself of the pyramid and determine from where that piece of evidence comes. How generalizable is this evidence? How biased could this evidence be? The pyramid gives you a framework to answer quickly, “Is this the best evidence?”
Literature cited:
- Paul M, Leibovici L. Systematic review or meta-analysis? Their place in the evidence hierarchy. Clin Microbiol Infect 2014;20:97–100.
- Murad MH, Asi N, Alsawas M, Alahdab F. New Evidence Pyramid. Evid Based Med 2016:21:125-127.
- Kang JH, Willett WC, Rosner BA, Buys E, Wiggs JL, Pasquale LR. Association of Dietary Nitrate Intake With Primary Open-Angle Glaucoma: A Prospective Analysis From the Nurses’ Health Study and Health Professionals Follow-up Study. JAMA Ophthalmol. 2016;134(3):294-303.
- Goldacre, B., & Farley, R. (2010). Bad Science: Quacks, Hacks, and Big Pharma Flacks. Farrar, Straus and Giroux.
Dr. Klute owns and practices at Good Life Eyecare, a multi-location practice in Eastern Nebraska and Western Iowa. He is a fellow of the American Academy of Optometry and is certified by the American Board of Certification in Medical Optometry. He writes and lectures on primary eye care practice management, evidence-based medicine, glaucoma, and dry eye disease.
