How Do You Find Good Evidence for Wellness Claims?
Q&A: Where to look for sound science, red flags for alternative treatments, what to do when someone swears their non-evidence-based wellness protocol works, and more.
Hey there!
This is the first edition of the Rethinking Wellness newsletter, which will be coming out every other Wednesday to start (on the weeks we don’t have new podcast episodes). Thanks so much to all the new folks who’ve just joined us, including my lovely longtime subscribers who came over from my previous newsletter, and those who found me through CBS Mornings or another recent media appearance!
In these newsletters I’ll be answering audience questions about wellness culture, including whether particular diets are warranted, the truth about supplements and other protocols, how to spot wellness scams, and lots more. You can ask your own question here for a chance to have it answered in an upcoming edition, and some of those questions may end up in bonus episodes of the podcast as well.
The first question today is available to all subscribers (about where and how to find research studies about wellness claims, especially for beginners).
Then there are 3 bonus questions for paid subscribers (about how to avoid falling into a wellness trap when deciding whether to try a fad or alternative treatment, how to figure out what you truly like to eat/do outside of wellness culture, and how to respond when someone swears that their wellness ritual is the reason they feel better—but you know it’s most likely a placebo).
Without any further ado, let’s get to the questions!
If I want to find research studies for a wellness or diet claim, how to start and where best to look, especially for beginners?
—Tyann
Great question! PubMed is the best place to start for finding scientific research on health and wellness, for beginners and experts alike. This U.S. National Library of Medicine database contains pretty much any health-related scientific study worth its salt—as well as many that aren’t, so keep that in mind. Just because you see something in PubMed doesn’t mean it’s good science; you’ll have to do more investigating to find out. But if you come across a purported piece of scientific evidence for a health/wellness claim and it’s NOT available in PubMed, consider that a red flag.
Once you’re there, put in your search terms and then filter the results to get relevant studies. You’re looking for randomized controlled trials (RCTs), which are studies in humans that compare a particular intervention to a placebo—the “gold standard” of scientific research. You DON’T want studies that are merely observational, or studies conducted in nonhuman animals or in cells (known as in vitro studies) or computer models (in silico studies). Those types of studies are helpful to other scientists looking for promising lines of research, but they’re not designed to tell you anything about the clinical efficacy of a potential treatment in humans.
To quickly find studies with the appropriate design, look in the left rail where it says “Article Type” and check the box next to “Randomized Controlled Trial.” Even RCTs can have their problems (as we’ll discuss), so your search doesn’t end here, but this will provide a good jumping-off point.
Even better (especially if your search returns many RCTs): uncheck that box, and then check the ones next to “Systematic Review” and “Meta-Analysis.” These study types synthesize results from all of the existing RCTs and are likely to be some of the best available evidence, though that’s not a guarantee—if the underlying RCTs are flawed, you could end up with a “garbage-in, garbage-out” meta-analysis. We’ll discuss how to recognize a few different forms of “garbage” below.
If you try searching for RCTs and get zero results, that’s a huge red flag. It means that the wellness claim in question is based either on early-stage research (observational studies, research in animals, cell studies, computer models), or on no science at all, but just anecdotes. In any case, to me that’s often reason enough to consider the claim unsubstantiated and move on. (The emotional pull these claims can have is another story, which I’ll address in the second answer.)
If your searches do return some results, now you can start digging in to learn more. Clicking the title will bring you to what’s called the abstract, which is a free summary of the study.
A helpful next step is to look at who funded the research. Many journals now include a conflict-of-interest statement at the end of the abstract, so you don’t have to read the full study to see it. If you see a conflict of interest—“Author X has consulted for XYZ Vitamins, Inc.” or “Researcher Y is on the board of ABC Diet Co.” or “Funding for this [diet-drug] study was provided by TKTK Pharmaceuticals”—the study has a high likelihood of bias, so it’s reasonable to disregard the results and move on to the next study. But if you see a “no conflict” statement like the one below, you can proceed to the next line of inquiry.
Next you’ll want to look at the number of participants and the study population. If an RCT was conducted in only a handful of people, it might be promising early evidence in favor of a certain treatment—or it might be a fluke that isn’t reproducible in bigger studies. Similarly, if an RCT was conducted in a population you’re not a part of—e.g. men in their 70s, and you’re a woman in your 30s—then the results may not be applicable to you. In any case, we won’t know for sure until follow-up studies are conducted in more representative groups.
There are some additional things to consider when assessing scientific research, as I discuss in the final chapter of The Wellness Trap, so I’d recommend checking that out for the full picture. But the above steps should be a good start in weeding out many dubious wellness claims.
Thanks so much for the great question, Tyann, and thanks to all the free subscribers for reading! Paid subscribers can stick around for 3 bonus Q&As, and everyone can ask their own questions for a chance to have them answered in an upcoming newsletter.
What should we ask ourselves if we want to try a wellness fad? As a person with chronic pain and living with chronic illnesses, sometimes there is very little offered in the way of treatment. How can I keep myself from falling into a trap, to borrow your word, when deciding whether to try an “alternative” treatment?
—Shelley