More on Scientific Studies

As a follow-up to Monday’s article about scientific studies, why they are often contradictory and the need to understand how they are structured to assess their credibility, I came across this excellent example in my reading file.

Remember the key points from last time:

• Negative results, those that show something doesn’t work or there is no danger, often don’t get a lot of publicity – even if knowing that information could be important;
• Lack of funding for unexciting project leads to reduced efforts by researchers to replicate or validate previous studies;
• When studies are not replicated and validated the strength or their results is questionable, and they are likely to be overturned;
• Small sample sizes are prone to yield weakly supported results;
• Even with a larger sample, a one-time study may produce a statistical fluke rather than a real scientific finding;
• Finally, a lot of charlatans on TV and the Internet describe themselves as scientists and use scientific-sounding jargon to make a sale.

Related to the last item, when a sufficient number of Americans buy into a certain belief, companies will go to great lengths to meet the resultant demand.  They usually have no vested interest in educating the public when they can use junk science to sell as much product as real science.

Now comes the example from Business Insider.  The headline reads, “Scientists who found gluten sensitivity evidence have now shown it doesn't exist.”  The same people whose small study in 2011 concluded that “gluten-containing diets can cause gastrointestinal distress in people without celiac disease,” a condition they called non-celiac gluten sensitivity, went back to question their own results.  In the follow-up study they tested thirty-seven self-identified gluten-sensitive patients.

Here’s how it went.  The researchers provided every meal for twelve weeks.  The meals cycled through high-gluten, low-gluten, and no-gluten diets.  Without knowing which diet plan they were on at any given time, the participants reported pain, bloating, nausea, and gas to a similar degree for each of the diets.  The researchers could find “absolutely no specific response to gluten” and were forced to rethink their original (2011) conclusions.  The participants “expected to feel worse on the study diets, so they did. They were also likely more attentive to their intestinal distress, since they had to monitor it for the study.”

Does that matter to the food industry? – Of course not!  As long as 30 percent of people want to eat less gluten and gluten-free products produce $15 billion in sales, why should they care?  The good news is for the 1 percent who actually has celiac disease.  They don’t have to search so hard for gluten-free food.  The other 29 percent with a problem that’s all in their head puts pressure on the manufacturers to make sure the labels are big and bold, regardless of the science.

This is the drawback to the labeling argument that regardless of the science everyone has a right to know what they are eating and should be able to make an informed choice.  If that informed choice is based on faulty assumptions, superstition and scientific advice from Facebook friends, it is a drain on society.  When resources are spent to satisfy fantasy fears, they cannot be used to better purposes.

By the way, the example comes from August 2015 showing the truth in the first bullet point above that studies showing no danger get far less publicity.