Who knows what to believe anymore?

Every day, there’s a new sensational declaration in the news that something you do, eat, or believe is killing you.

• Eat this!
• Avoid that!
• Make sure to do this!
• Stop doing that!
• Lose weigh by buying this!
• Bulk up by ordering that!

It seems like no matter WHAT you do, it’s easy to find an article that tells you about all of the things you’re doing wrong or things you need to buy.

With well-researched arguments that cite seemingly credible  sources, how can you tell which side is right and which side is wrong?

On top of that, you’ll probably start wondering with each article you read:

Are these people telling the truth? Do I need to change my lifestyle?

With one of the Rules of the Rebellion being “Question everything (including Nerd Fitness),” it’s about time I showed you the process by which I accept or reject a particular study or news story.

After all, we’re nerds!  We analyze the nuances of characters in video games and books; why shouldn’t we do the same with the news?

So put on your Sherlock Holmes hat, grab that comically overized magnifying glass, and let’s do some super-sleuthing, gumshoe!

Step One – Set Aside Personal Biases

This one is brutally tough.

If you’re going to look at a study or article, it’s important to remember that your own personal biases are going to come into play.  It’s easy to read articles that support your beliefs and let unwelcome facts slide. It’s easy to instantly discount any article that threatens some of your own personal beliefs. The confirmation bias, the reason you’ve argued a point you know deep down is wrong, is a tough thing to shake.

I’m a normal dude just like you with my own set of beliefs and biases.  For example, because I’m a big fan of the Paleo Diet, its tough for me to remain unbiased while reading articles and studies that present an “anti-meat” or “pro-grain” angle.  I’m also a huge fan of weight training but not a big fan of distance running, so I’m generally biased towards articles that support the former and discount the latter.

For that reason, I do my best to scrutinize everything I read from both sides of any debate, as I want to present the best possible advice I can to the rebels of Nerd Fitness.  I try to open my mind and understand that new information will come out that might change how I feel about a particular subject.  I encourage you to do the same.

Now, Along with an open mind, it’s important to have one other thing…

Step Two – Put on your skeptic goggles

These are my skeptic goggles.  I generally get funny looks when I wear them in public.  

When I come across a very bold and dramatic statement about food consumption or health practices (whether or not they support your current beliefs), my first initial thought should be:

“things that make me go HMMMMMMMMM…..”

(sorry for getting that stuck in your head for the rest of today).

The truth is that like any other company, news stations and sites have a bottom line, and are  responsible for driving ads and creating revenue.  For that reason, their headlines tend to err on the dramatic side without actually presenting evidence or backing up their sensationalist assertions.    They’ll often cite a study, extracting just a single line, and claimed to have “proven” something.

It’s not uncommon to see a headline like: “STUDY SHOWS THAT _______________ INCREASES CHANCE OF DEATH!” In reality, the obscure study tested rats and not humans, and never actually made the claim that the news author declared.

You don’t need to wear the same skeptic goggles as me, but feel free to get creative and pick up a pair at your nearest imaginary goggle store.

Step Three – Interrogate the author

Let’s say you stumble across an article that says “Study Proves _____________ Causes 20% Increase in Mortality.”

After you’ve donned your goggles, it’s important to determine the validity of the news source and the author who wrote it.

Interrogate the author and figure out if they’re is telling the truth!  No, don’t kidnap, tie up, and shine a bright light on the author’s face. Read the article and decide if the author passes the “is this person legit?” test.

Let’s be honest: it’s one thing if it’s written by a Pulitzer-prize winning reporter, published in the New York Times, and cites a study from a nationally recognized health research company. It’s another if it’s written by somebody with an agenda citing a study funded by a food company to produce the results they’re after.

It’s often pretty simple to see if the author writing an article has an agenda or strong bias. For example, in Monday’s “Time to Retire the Low-Carb Diet Fad“, the author cites a study that talks about carbs, food cholesterol, and blood cholesterol. A sentence like this jumps out at me immediately:

“But let’s face it – most of us know in our hearts that eschewing a breakfast of whole grains and fruit crowned with a dab of yogurt for a greasy pile of sausage, bacon, and eggs is not the road to health.”

You can almost feel the disdain in her writing for the “greasy pile of sausage, bacon, and eggs!”  However, just discounting an author for having a bias doesn’t solve your problem.  We all have biases, and authors are often paid to present their side of the story, especially if it proves the point of a study that recently came out.

Once you’ve qualified whether or not the author presents a logical argument, it’s important to check their facts, as oftentimes the sources they cite are misrepresented.

Step Four – Study the study, like a boss

If it’s an article about food, diet, or exercise, it will almost certainly back up it’s claims with sources of recently conducted studies.  

In my years of studying health and fitness and dissecting articles and studies, a few things have become incredibly clear to me:

• Just because it’s a medical study from a famous university does NOT mean it’s accurate.
• Correlation does NOT prove causation. Oftentimes, conclusions are drawn in studies that show a correlation between two things (eating habits and mortality, for example), but showing that two things are related doesn’t prove that one causes the other. There may be a correlation between wearing a storm trooper uniform and death, it doesn’t mean the uniform CAUSES death.
• Scientists and doctors can have preconceived biases that work hard to prove their own beliefs, even if the facts don’t back them up.
• Diet studies are often grossly inaccurate, as they rely on people self-reporting their eating habits.  Human error, embarassment over writing down certain types of foods, and a host of other variables (that aren’t tracked) can lead to vastly different results.

Red Meat Mortality and Consumption was conducted by doctors from Harvard and compared health and eating habits of thousands of people to declare:

Red meat consumption is associated with an increased risk of total, CVD, and cancer mortality. Substitution of other healthy protein sources for red meat is associated with a lower mortality risk.

This study was covered in practically every major news network around the world with various levels of bold  headlines.

EESH. If you’re a meat eater, or considering switching away from eating meat, this seems pretty cut and dry.

Harvard researchers: check.

Doctors: check.  

Decisive conclusion drawn from study: check.

However, as you’ve hopefully learned from the lessons above, these studies aren’t always what they seem.  This study presents information designed to make us think “this is fact.”  However, a closer look at the actual data and results can make or break the credibility of a study. The devil is in the details. Don’t let the fancy PHD’s and Harvard scare you away from digging deeper. So let’s dig deeper.

Goggles on! 

• It’s an observational study - this means that they simply observed people’s eating habits over a number of years by having them fill out a confusing questionnaire every few years.  If you take a look at the questionnaire, you’ll notice how easily it could lead to false data and incorrect responses through simple human error.
• The conclusions are interpretive, not concrete. As stated above, just because two things are correlated doesn’t mean that one proves the other.  For a really weird example,  Members of the Church of the Flying Spaghetti Monster will point out that the decrease in pirates over the past few centuries has coincided with an increase in the oceans temperatures.  Using the Harvard Study logic above, the ocean’s rising temperature is a direct result of less pirates!
• People lie!  Ever had to fill out a doctor form that asks you how many drinks you have per week, and you check the “less than three per week,” when its more like “a few beers a day?”  You can bet your ass people lied on their food questionnaire forms.
• There is a wide range in what is considered “unprocessed meat.”  According to the survey, “hamburger” would qualify as unprocessed meat – which would then also include hamburger patties from McDonalds and taco fillings you’d get from Taco Bell.  No difference is made between the best organic grass-fed meat and the worst processed meat.
• As Gary Taubes tells us about this article: ”Very few epidemiologists would ever take seriously an association smaller than a 3- or 4-fold increase in risk. These Harvard people are discussing, and getting an extraordinary amount of media attention, over a 0.2-fold increased risk.”  The correlation is tiny when comparing meat-eaters to non-meat eaters, and could be well within the “margin of error” of reporting when realizing these results come from confusing questionnaires sent to people who may or may not have reported accurate results!

But what you don’t actually have time to do the research on these topics and read all of the studies that are citied?

Find research Yodas that you do trust.

Step Five – Go To Sources You DO Trust

Since entering the fitness and health world years ago, I’ve come across two sources that I turn to time and time again when it comes to the debunking or qualifying of a particular news story or study:

• Gary Taubes - I’ve found Gary’s books, “Good Calories, Bad Calories” and “Why We Get Fat” to be two of the most well researched and logical pieces of writing and reporting when it comes to dietary information.  Since then I’ve started to read the rest of his articles and found them to be logical, well written, and well-cited.
• Denise Minger - Denise runs a website called RawFoodSOS, and I discovered her blog while doing research on the China Study.   I found her thorough analysis of the data from the China Study to be beyond incredible, and have since eagerly awaited any articles she writes about new studies.   If you are a data-analysis nerd, her site is a GOLD MINE.

Whenever I see a new study, my first thought is to see if either Gary or Denise have analyzed the data and covered it themselves.  Both of them have repeatedly earned my trust and respect and thus I’ll turn to them to get help when I don’t have the attention span to analyze pages of medical studies myself.

If you have enough time today, I HIGHLY recommend reading the two articles cited next.

Here’s Gary’s rebuttal to the “Red Meat Will Kill You” study. with a few excerpts:

“So when we compare people who ate a lot of meat and processed meat in this period to those who were effectively vegetarians, we’re comparing people who are inherently incomparable. We’re comparing health conscious compliers to non-compliers; people who cared about their health and had the income and energy to do something about it and people who didn’t.

They might as well be comparing Berkeley vegetarians who eat at Alice Water’s famous Chez Panisse restaurant once a week after their yoga practice to redneck truck drivers from West Virginia whose idea of a night on the town is chicken-fried steak (and potatoes and beer and who knows what else) at the local truck stop.”

Here is Denise’s fantastic rebuttal to the Red Meat article over on Mark’s Daily Apple:

“The folks eating the most red meat were also the least physically active, the most likely to smoke, and the least likely to take a multivitamin (among many other things you can spot directly in the table, including higher BMIs, higher alcohol intake, and a trend towards less healthy non-red-meat food choices).”

Now, these are my two go-to sources.  I’d love to hear if you have come across any in your own journeys through this wonderful world of bold statements, scientific studies, and medical trials.

Lessons Learned Today

If you learn nothing else today, let it be this:

Don’t believe everything you read, no matter the source!  Do your own research, identify possible biases, and then decide for yourself if its something you want to accept.  If you are truly concerned eating something or not eating something is bad for you, conduct your own personal trial.  Get your blood checked, change one variable, and then get blood work done again a few months later and analyze the results.

If you don’t have the time, resources, or money to conduct these experiments on yourself, do the best you can without going too far off the deep end in one direction or the other.  You’re smart – make the best choice you can with the information you have, and don’t be afraid to learn from how your body acts and feels.

And there we have it: You’ve now graduated from the Nerd Fitness School of Skepticism.  Your diploma is in the mail and will arrive in 3-5 business days!

How do you deal with information overload?

How do you decide what to believe and what to discard?

Any thorough researchers you’ve discovered that I should add to my reading list?

Goggles on!

-Steve

### 

 Photos: Lego Fitness Skeptic Sherlock Homes, Rose Colored Glasses, Yoda and Danbo Fitness Skeptics , Closed book and magnifying glass, Words and glass, Lego Fitness Detective and Magnifying Glass