How Do You Find Good Evidence for Wellness Claims?
Q&A repost: Where to look for sound science, red flags for alternative treatments, what to do when someone swears their non-evidence-based wellness protocol works, and more.
Right now the public-health world is rightly freaking out about the nomination of Robert F. Kennedy Jr. to lead the department of Health and Human Services (HHS). Kennedy has promoted numerous baseless conspiracy theories, such as the false claim that the FDA is “suppressing” products like raw milk and ivermectin, when in reality those things have been studied and found to be harmful or ineffective. He has claimed people are being “unknowingly poisoned” by seed oils, which is false—there’s no good evidence to suggest seed oils are harmful, and considerable research showing they may have health benefits, such as reducing levels of unfavorable blood lipids. And he’s spent the past 20 years sowing public distrust in vaccines. Though he disavows the “anti-vax” label, claiming to be in favor of “safe vaccines,” he’s repeatedly and continually ignored the evidence that FDA-approved vaccines already meet rigorous safety standards, instead spreading the falsehoods that they’re not tested for safety in clinical trials or studied using “good science” (they very much are).
For more on Kennedy’s harmful misinformation, here are a few recent pieces worth a read:
The Sanewashing of RFK Jr. (The Atlantic)
12 Questions for RFK Jr. (American Council on Science and Health)
My Conversation with Robert F. Kennedy Jr. (
)Installing Kennedy at HHS would be profoundly damaging for public health and health policy in the U.S. If you agree, please consider calling your Senators and urging them to vote no on his confirmation.
Whether or not he’s confirmed, his nomination has already given him a bigger platform and allowed him to spread misinformation to a wider audience—possibly including people in our lives. So today’s Q&A is a deep dive into how to assess the evidence behind wellness and diet claims.
The first question is available to all subscribers (about where and how to find research studies about wellness claims, especially for beginners). Then there are 3 bonus questions for paid subscribers (about how to avoid falling into a wellness trap when deciding whether to try a fad or alternative treatment for chronic illness, how to figure out what you truly like to eat/do outside of wellness culture, and how to respond when someone swears that their wellness ritual is the reason they feel better—but you know it’s most likely a placebo).
If I want to find research studies for a wellness or diet claim, how to start and where best to look, especially for beginners?
—Tyann
Great question! PubMed is the best place to start for finding scientific research on health and wellness, for beginners and experts alike. This U.S. National Library of Medicine database contains pretty much any health-related scientific study worth its salt—as well as many that aren’t, so keep that in mind. Just because you see something in PubMed doesn’t mean it’s good science; you’ll have to do more investigating to find out. But if you come across a purported piece of scientific evidence for a health/wellness claim and it’s NOT available in PubMed, consider that a red flag.
Once you’re there, put in your search terms and then filter the results to get relevant studies. You’re looking for randomized controlled trials (RCTs), which are studies in humans that compare a particular intervention to a placebo—the “gold standard” of scientific research, and the only type of study that can possibly tell us about causation. You don’t want studies that are merely observational, or studies conducted in nonhuman animals or in cells (known as in vitro studies) or computer models (in silico studies). Those types of studies are helpful to other scientists looking for promising lines of research, but they’re not designed to tell you anything about the clinical efficacy of a potential treatment in humans.
To quickly find studies with the appropriate design, look in the left rail where it says “Article Type” and check the box next to “Randomized Controlled Trial.” Even RCTs can have their problems (as we’ll discuss), so your search doesn’t end here, but this will provide a good jumping-off point.
Even better (especially if your search returns many RCTs): uncheck that box, and then check the ones next to “Systematic Review” and “Meta-Analysis.” These study types synthesize results from all of the existing research and are likely to be some of the best available evidence, though that’s not a guarantee—if the underlying data are flawed, you could end up with a “garbage-in, garbage-out” meta-analysis. We’ll discuss how to recognize a few different forms of “garbage” below.
If you try searching for RCTs and get zero results, that’s a big red flag. It means that the wellness claim in question is based either on early-stage research (observational studies, research in animals, cell studies, computer models), or on no science at all, but just anecdotes. In any case, to me that’s often reason enough to consider the claim unsubstantiated and move on. (The emotional pull these claims can have is another story, which I’ll address in the second answer.)
If your searches do return some results, now you can start digging in to learn more. Clicking the title will bring you to what’s called the abstract, which is a free summary of the study.
A helpful next step is to look at who funded the research. Many journals now include a conflict-of-interest statement at the end of the abstract, so you don’t have to read the full study to see it. If you see a conflict of interest—“Author X has consulted for XYZ Vitamins, Inc.” or “Researcher Y is on the board of ABC Diet Co.” or “Funding for this [diet-drug] study was provided by TKTK Pharmaceuticals”—the study has a high likelihood of bias, so it’s reasonable to disregard the results and move on to the next study. But if you see a “no conflict” statement like the one below, you can proceed to the next line of inquiry.
Next you’ll want to look at the number of participants and the study population. If an RCT was conducted in only a handful of people, it might be promising early evidence in favor of a certain treatment—or it might be a fluke that isn’t reproducible in bigger studies. Similarly, if an RCT was conducted in a population you’re not a part of—e.g. men in their 70s, and you’re a woman in your 30s—then the results may not be applicable to you. In any case, we won’t know for sure until follow-up studies are conducted in more representative groups.
There are some additional things to consider when assessing scientific research, as I discuss in the final chapter of The Wellness Trap, so I’d recommend checking that out for the full picture. But the above steps should be a good start in weeding out many dubious wellness claims.
Thanks for the great question, Tyann, and thanks to all the free subscribers for reading! Paid subscribers can stick around for 3 bonus Q&As, and everyone can ask their own questions for a chance to have them answered in an upcoming newsletter.
What should we ask ourselves if we want to try a wellness fad? As a person with chronic pain and living with chronic illnesses, sometimes there is very little offered in the way of treatment. How can I keep myself from falling into a trap, to borrow your word, when deciding whether to try an “alternative” treatment?
—Shelley