Flashback - Diet Soda and Depression

[I wrote this over seven years ago about another meaningless study meant to make disturbing headlines for the news media and possibly to attract more funding for the researchers. The only party left out of the equation, as usual, was we, the consumers.] 

I have mentioned before the tendency of journalists and the media to try to get and hold our attention with emotionally charged pictures and stories. We regularly see news about the results of studies that seem surprising, scary, or shocking, but they are often somewhat meaningless. They challenge our ability to remain calm and think rationally about the subject. Here is another example.

“A new study finds that people who drink diet sodas or fruit drinks are more likely to be diagnosed with depression.” The article continues very responsibly to explain the size and nature of the study and to emphasize that a link does not necessarily mean that drinking diet sodas causes depression, but the headline - "Drinking diet soda linked to depression" – has us hooked. (With sugary soft drinks being blamed for obesity - blame the soda, not the person drinking it - our choices are narrowing.)

Thinking critically about it, we know that correlation is not the same as causation. "Linked to" isn't the same as "caused by." We may wonder what we are supposed to do with this information – stop drinking diet soda so to avoid depression or start drinking lots of coffee, which the article tells us may have the opposite effect? I don’t think it works that way.

Later in the article they say, “more research is needed.” So what was the point?

Why should we even care about these kinds of studies that give preliminary findings, or publicize findings before they are presented for formal review, or rely heavily on self-reporting as opposed to objective observation?  As I pointed out before, we don’t have the time or energy to be worried about everything, so there is nothing really useful about such news. But it does make for catchy headlines.

[After all this time nothing has changed. Basically meaningless preliminary studies continue to be published and highlighted in the news almost daily, and it won’t stop until a large number of Americans wise up to the tactics.

On second thought, one thing has changed. Judging from comments on social media, some people do have time to worry and express an opinion about everything, but that's another topic. ]

A/B Testing in Education

Back on April 20 of this year, I wrote a detailed explanation of experiments. Because we so often see in the press “breaking news” of the “latest study,” it’s important for critical thinkers to have a good understanding of what makes a study valid. Sample size and composition are important. Many press releases are based on less than 100 observations from a narrow population but reported as if they apply to everyone. 

Another major flaw is that when the scientists or doctors report a correlation, a strong relationship between a drug or practice and an outcome, the news reports it as if there is a cause/effect relationship, which is much less likely and harder to prove. That’s why the most common phrase at the conclusion of these studies is that more research is needed, but this may be an afterthought in a news report if it's mentioned at all.

I read lately about a slightly different type of experiment. It is a natural experiment and has become more common in this era of big data. Instead of inviting participants into a lab, dividing them into two groups, giving a treatment to one group and leaving the other alone and  then comparing results; a natural experiment sets up different groups in real life by feeding them different information and measuring the reaction.

Google, Facebook, major advertisers and others with access to a huge population of followers can vary their messages and compare results. They don’t need volunteers, users of the sites become unknowing subjects.

An ad to one group may have a blue background and the other a green background. Which gets the most clicks or likes or referrals or sales? An advertiser may use different pictures or wording in two different regions of the country. Which gets a better response? Someone calls the number and asks for Mary, not knowing that Mary is a code for a particular radio or TV ad in Chicago as opposed to a different one there or somewhere else. This practice is called A/B testing, and it’s happening constantly, especially on the major social media platforms. What are the best colors, pictures, format, headline wording, etc. to get clicks, shares, donations or return visits? 

It sounds like sophisticated manipulation with no apologies – psychological warfare and an invasion of privacy. But there may be several positive uses for it. I thought of this while watching the Jeopardy! Teachers Tournament last week.

According to Everybody Lies, a book about big data applications, a company called EDUSTAR that makes educational software for kids did a kind of A/B test that yielded surprising results. “One lesson plan that many educators were very excited about included software that utilized games” to teach fractions. They found out that a more standard approach, not using games, yielded better student understanding.

On Jeopardy! right after the first commercial break, the teachers sometimes respond to questions by describing their methods in the classroom, and each seems to takes a different approach. They have their little tricks and games, and they are doing what they are comfortable with or what has perhaps worked for them in the past. These ideas seem to get an enthusiastic reception, but who knows if they really work or which is better for producing better-educated kids – presumably the point of our schools.

They can’t all be right about getting the best outcome. This seems like a perfect setting for a similar kind of A/B testing by increasing the sample size from one to many classrooms distributed over many different schools. Maybe some techniques work better than others. It would be interesting to find out. Of course some teachers would be uncomfortable with this, but surely the objective of education is to make kids more competent, not to make teachers comfortable. Just a thought.

Understanding Experiments

When most people hear the word experiment, they picture a scientist in a lab coat with bubbling beakers and test tubes of mysterious liquids. This is not the real meaning. As a result of this misconception, news of the latest study is usually misleading.

An experiment is a rigorous process of testing an idea with the intention of either solving a problem or of improving a situation. Necessary first steps include defining the problem to be solved or determining how results will be measured. No one can claim improvement without measurements to compare (before and after). In the first case, the experiment is successful if the problem goes away. In the second, success is judged by the measurable amount of improvement.

When a well-designed experiment is successful, the implication is that the same solution can be applied widely to other situations: to solve the same problem elsewhere or to achieve the same amount of improvement. (Usually others replicate the experiment before findings are accepted.)

To achieve satisfactory results, any experiment must be well designed. Sloppy studies lead to problematic conclusions, ones that can’t be counted on to solve anything. Even the best experiments can yield bad information just based on the amount of diversity in the world and the fact that fluky things happen. 

Many experiments or studies the public is exposed to in the media relate to drugs or other remedies, and a strict procedure must be followed to ensure the conclusions are valid, otherwise the drugs or other remedies get on the market without proof that they are safe and effective.

In drug studies researchers try to choose a sufficiently large sample, because the larger the sample, the lower the chances of getting some oddball results just because you happened upon some atypical participants. A bigger sample tends to average out any unusual individual readings. 

The next step is to divide the sample into two parts that look as much alike as possible: same proportion by sex or race or education or income or background or location or any other feature that might affect the results. Sometimes this can only be done by random assignment to one group or the other. It's better if researchers have a good understanding of all the characteristics that might influence results and can make the two groups look as much alike as possible relative to those characteristics.

One group is treated – given the pill or the information or other treatment – the other, the control group, is given a fake pill (placebo) a sham treatment or left alone.

Afterward the groups are compared statistically to see if the change (hopefully an improvement) in the first group is significantly better than the change in the second. (Yes, there is often a change in the control group merely because the know they are participating in the experiment and believe the placebo is a real remedy.)

Ideally the people doing the testing are not aware of who is in which group (double-blind).

We know from the news that even these careful experiments can go wrong. Sometimes drugs are withdrawn from the market due to problems discovered only after they are released, released to a much larger sample size. That is why I have come down so hard and so often on vitamins and other dietary supplements where the law exempts them from the need for any research at all to prove safety and effectiveness. (They rely on endorsement, not proof, and use weasel words to imply effectiveness.)

Many times in the past I have also criticized experiments because they have been sloppy about their design. Sometimes they don’t define the problem until after the test is done and announce the findings with a press release. Sometimes the samples are too small due to budget constraints or laziness. Sometimes they rely on self-reporting so there is no real measurement. 

Finally businesses and educators like to say they are experimenting. In both cases it is rarely true. They don't set up two groups to compare. Then they measure by gut-feel.

Businesses are in too much of a hurry to do it right. They just try things, sometimes multiple things at the same time, so who knows which ones have a positive or negative or neutral influence on the final outcome.

In education they are still arguing about measurement. Teachers don’t want to be paid based on test scores, but haven't suggested a more acceptable criterion to measure their results. Yet school systems continue to try new methods and approaches without ever satisfying the first step – how to objectively measure real improvement.

Without understanding experiments, it’s too easy to be fooled by people who don’t know what they are doing and by people who do but are just trying to sell us a bill of goods. 

Walk Faster, Live Longer?

A very interesting article in JAMA a number of years ago requires the application of critical thinking to avoid making a misinterpretation.

 A team of scientists put together a summary of nine studies over the years totaling almost 35,000 subjects over the age of 65, residents of various nursing facilities . Each study was trying to find out whether measuring a person’s walking speed could help predict how much longer they would live.

The data came from the various researchers measuring how fast people walked over a short measured course, between 8 and 20 feet long, using a stopwatch and converting the speed to feet or meters per second. They then followed up years later to see how well the measured speed at the time of the test correlated with how much longer the individuals lived. (The authors refer to it as “gait speed.”)

The published conclusion, when they put the information from all the studies together: “In this pooled analysis of individual data from 9 selected cohorts, gait speed was associated with survival in older adults.” The graphs accompanying the article show this quite clearly, and the findings held true for both men and women and for people of different races. The top group walked the course at 1.6 meters per second, which converts to about 3.6 miles per hour. (For a longer distance this would mean a pace at which someone could walk a mile in about 17 minutes.)

How is this useful? “First, gait speed might help identify older adults with a high probability of living for 5 or 10 more years, who may be appropriate targets for preventive interventions that require years for benefit.” It could help assess whether a particular medical procedure would be worth the risk – there is always a risk, especially for older people. “Second, gait speed might be used to identify older adults with increased risk of early mortality, perhaps those with gait speeds slower than 0.6 m/s.” It is a simple and informative way for “assessing expected survival to contribute to tailoring goals of care in older adults.” Furthermore, it is a simple measurement that can be easily done in a nursing home corridor by non-professionals.

How is this not useful? Correlation is not causation. Someone seeing this article and not using critical thinking might conclude that beginning to walk faster was a way to guarantee longer life. After all, the graphs on the page clearly show that those who walked faster lived longer. Except for the fact that exercise is good for anyone, this conclusion is not necessarily so; it’s not what the analysis shows.

The authors theorize that the correlation exists because, walking “requires energy, movement control, and support and places demands on multiple organ systems, including the heart, lungs, circulatory, nervous, and musculoskeletal systems.” A slower walking speed could indicate some damage or a larger than normal energy requirement for other reasons. Both could be a general indicator of lessened vitality. It makes sense that those who were going to live longer were just capable of walking faster, and did so.

Unfortunately, scientists did not prove that increasing our walking speed will increase our life expectancy. (In fact, people walking faster while texting could easily decrease their life expectancies.) Getting more exercise, including walking, and generally taking care of ourselves will. There is rarely one easy answer. The real secret to good health appeared in this space over five years ago and, unfortunately requires some degree of discipline.

Those Sore Muscles

Last time I discussed a book I hadn’t read, but based on a couple of reviews, I found some positive things to say about the advice it gave about taking responsibility. (I also threw in some snarky comment about the title.)

This time I want to discuss a book that I actually did read. Published quite recently, it’s called Good To Go, by Christie Ashwaunden. The subtitle is: What the athlete in all of us can learn from the strange science of recovery. It covers a variety of tools and techniques for physical recovery after participating in a sporting event or hard practice session. Examples are taken from professional athletes, Olympic-level competitors and serious sports enthusiasts. The book is an investigation of what works, what has real scientific backing vs. what is based on marketing, hype, and urban myth. It contains many surprises, but reinforces the need for critical thinking, the need to challenge our deeply held beliefs in everything we do.

The introduction states the problem clearly. “The explosion of recovery products and services can seem ridiculous.” Although “everyone intuitively knows what recovery is and how to achieve it,” the industry has found a way to make it much more complicated, expensive and time-consuming than it needs to be or ever has been.

Promoters of these new methods usually use stories rather than scientific facts, and stories told by celebrities or sports stars are most effective. The advertised science behind them is often based on poorly constructed studies with very small sample sizes. Only the ones with positive results see the light of day with the rest (often the majority) filed away. The book investigates many of these products and services to see what really works. I will cover just a few.

One such product with little scientific justification is the sports drink phenomenon, a huge industry with many devoted fans. One expert characterized some of their studies as verging on “comical.” The author takes on the whole idea of hydration, pre-hydration and electrolyte supplements and comes to the conclusion that the best defense against dehydration is to drink when you are thirsty. Any other approach can result in waste or even harm. 

She gives the example of running with her dog. When they get home, she gives her access to a water bowl. The dog decides whether or not to drink. The author has “never needed to inspect the color of the dog’s urine or give her an emergency IV for low fluid levels.” We make it overly complicated, yet hydrate is common advice from all sides every summer.

Similar advice goes for sports nutrition. She gives the example of Usain Bolt, the fastest man in the world, filling up on chicken nuggets during the Olympics. They may not have been the most nutritious meals, but “when your muscles are hungry for fuel, they don’t care where the energy comes from.... What should I eat after exercising? The easiest answer is: whatever your body is hungry for.” 

She also discusses the promotion of such ideas as infrared saunas, explaining that heat is heat, and any magical powers attributed to infrared, such as reducing inflammation, boosting growth hormones, clearing toxins or boosting the immune system are unsupported and have received negative attention from the FDA. Yet marketers and pro athletes push overpriced infrared saunas, cold lasers, massage beds and pajamas, based on the “sciency” sound of it all.

Subsequent chapters talk about sleep and supplements. Sleep is vital for recovery. Keep it uncomplicated; our body helps us figure out how much is enough. I have addressed the problems with supplements many times in the past. (See here and here). They are unregulated, can be dangerous and are poorly researched. They have a reverse approval process compared to prescription drugs. No one has to prove they are safe and effective. The FDA only gets involved when they prove to be harmful, as in the case of ephedra, where a hundred people died taking it to increase energy. “Yet it took the FDA ten years to ban it.”

There is much more to learn from this book. But I found it an excellent example of applied critical thinking with a host of interesting surprises.

One More Time – The Placebo Effect

Way back in early 2012, I first brought up the placebo effect – the tendency of the body to heal itself when belief in the cure is strong. How it works is still a mystery but that it does work is indisputable.

The example I used over 6 years ago was a $30 performance wristband, one endorsed by famous athletes and touted to improve athletic performance. In a test people were given the advertised wristband and exposed to some physical performance and balance tests. The before and after results showed some improvement. Then they were given a similar-looking one-dollar replacement band and told it was also special, performance on physical and balance tests improved comparably. Conclusion: it wasn’t the band at all; it was the perception that they were getting some extra, outside help – mind over matter.

Since then I have mentioned the placebo effect in essays warning about vitamins, acupuncture, forest bathing, chiropractic, healing crystals, gluten-free diets, cryotherapy, homeopathy, ear candling, essential oils and a few other subjects. (Wow, that’s a lot of toes to step on in only six and a half years!)

The point is not to make people angry and defensive, but to make them aware of the difference between science and marketing, and how the placebo effect can lead to an erroneous conclusion that a treatment really works.

When a cure is proposed, competent scientists will split a relatively large group in two, randomly assigning one to test the treatment while giving the other a placebo. Both groups usually show some improvement due to the placebo effect. To validate the treatment however, the improvement of the treated group must be significantly better than that of the control group. Otherwise they declare the treatment “no better than a placebo,” that is, no better than no medicine at all.

The placebo effect explains many of the endorsements we hear from celebrities, friends and neighbors. They sincerely (and enthusiastically) believe whatever they are recommending has beneficial effects, but with no scientific evidence it may be “no better than a placebo.” When you buy one of these products, the money spent is money wasted. That’s why the small print in ads, where they explain how the FDA has not approved their magic formula, they often add something like “results will vary.”

That takes us to a health story from England where the BBC reported a pure placebo experiment. With help from University of Oxford, they tried to see if they could “cure real back pain with fake pills.”

One hundred people with severe back pain were asked to participate in a study of a powerful new painkiller. What they didn’t know is that everyone would be given realistic looking pills that were really placebos, “capsules containing nothing but ground rice.”

The blue-and-white-striped pills “came in bottles, carefully labeled, warning of potential side effects and sternly reminding patients to keep out of the hands of children” to further the impression that this was powerful stuff.

Three weeks later researchers found “nearly half of our volunteers reported a medically significant improvement in their back pain” and that those who spent a little more time with a doctor merely discussing the pills were more likely to improve. In one case, a man went off his morphine but continues to take the ground-rice pills.

Placebo effect is not a trick that works only on the gullible. Taking a placebo can cause the body to release endorphins, natural painkillers. And sometimes subjects improve even when they know they are taking a placebo.

But that doesn’t mean the gullible aren’t tricked. In the earlier case of wristbands, even after the people learned that there was no difference and the improvement in both cases was psychological, they still wanted to buy the $30-wristband over the equally effective $1 bands. It’s like when people think the same wine tastes better when the bottle has a higher price tag. 

It does no good to understand science when critical thinking fails to kick in.

Living Longer – The Secret To A Long Life

Everyone wants to know the secret to living longer, and many, many answers to that question are tossed around in the news media and on social media every day.  This is an important issue to most Americans. It is a matter for critical thinking, making sure the proposals pass the test of common sense, for perspective, making sure the potential gain is worth the investment, for economic understanding, knowing that a dollar spent on this pursuit cannot be spent elsewhere, and for discipline, having the ability to stick to a plan for the long term. (Note: any secret to longevity that has no long-term component doesn’t pass the critical thinking test.)

The next three or four installments of this page will cover an investigation of a few of these secrets to longevity. So where shall we start – how about with taking more vacations?

A news article about a study in Finland that, curiously enough, got to Yahoo News by way of India, tells us that men who take vacations live longer. Specifically, “compared with those who took more than three weeks, men who took three weeks or less annual leave from their regular work schedule were found to be 37 per cent more likely to die early.”

The study included 1,222 middle-aged executives, who were randomly assigned to a test group or a control group. Every four months they gave the test group advice on healthy living, including subjects like diet, exercise and not smoking. The control group was left alone. “Shorter vacations were associated with excess deaths in the [test] group,” but made no difference in the control group. They concluded: “stress reduction is an essential part of programmes aimed at reducing the risk of cardiovascular diseases.” 

Wow, where do I start? To be fair, I couldn’t find a link to the paper presented, so have no idea about how long the study lasted – hopefully, it was more than a few years, or how they defined excess deaths. But there are other flaws in the reasoning.

First, the sample was a decent size, but they cut it in half for no apparent reason. They assigned men randomly to each group. Randomization is a good course of action when you have no idea about the background of the subjects, but with more information it is preferable to assign similar people to each group to try to balance out what researchers call confounding factors. Some of those factors may have been family or work situations that could make taking vacation more stressful or impossible, such as divorce or personal/corporate bankruptcy. So randomizing into two groups is puzzling.

Then I wonder if vacation time was the only variable they tested for – that would seem odd given that they also provided healthy living information to only one group. Otherwise they may have been testing for a number of variables and found an “association” only with vacations. I have written elsewhere about how this practice can lead to some unreliable, even crazy results.

And who funded the study? Was it the Finnish tourism industry?

 Finally, the conclusions they reached cannot be applied to any large population – of Finland or anywhere else. They tested only male executives. It is comparable to a college professor testing a group of undergraduates, called a sample of convenience, and publishing the results as if they applied to ordinary citizens across the country. No can do! Do male Finnish executives have anything in common with female Detroit autoworkers? In fact, the conclusions did not even apply to their own so-called control group. How does not receiving healthy living information affect how stress-reducing a vacation is? Half their experiment backed their findings and the other half didn't!

I’m sure it made the news because everyone would like to exercise a little confirmation bias and latch on to a scientific study telling them what they want to hear: that not taking a three-week vacation can be a health hazard. Unfortunately, it’s not that simple.

So, if we want to live longer, we must look elsewhere, like to those magic pills, foods or beverages packed with vitamins and minerals – right? Not so fast! A word or two about that next time.

Why So Much Bad Science

Last time I featured the latest news about palm oil, that it might be carcinogenic (or maybe not).  I used it to show how the debate about various cooking oils along with butter and margarine has been going round and round for several decades.  First A is bad for you so use B instead; and then we hear that the scientists have changed their minds about B.  We get lists, ordered from good to not so good, of oils (or another category of food) from different sources giving different information.  But each seems to have good reasons for their opinions based on some scientific study. 

Confusion is compounded when the news media pick up the story of a new study just out with new information of danger or benefits or else contradicting an older study.  The palm oil story reminded me of a published paper by a Stanford statistician in August 2005 “Why Most Published Research Findings Are False.”

One important reason for his claim is that scientific experimentation is never absolute.  When trying to find a relationship between a behavior or treatment and a result, several problems arise.  Some relationships may happen by chance alone.  The common standard of 95% correlation coefficient leaves a 5% chance that the result is incorrect.  The standard allows for one in twenty to be a false positive.  But many other factors conspire to push the number of false findings much higher.

First, correlation does not imply causation.  Even if one thing happens right after another, or they both vary in the same way, there may be no relationship.  This site gives several examples, one where a simple nutrition questionnaire found a (strong statistically significant) correlation between eating eggrolls and dog ownership and between drinking coffee and cat ownership.  Obviously eating an eggroll doesn’t result in an irresistible urge to get a dog.

Some other factors that may lead to errors include: 

• Using smaller sample sizes – this is often done for efficiency and economy; it’s easier and cheaper when fewer people are tested.
• Settling for a smaller effect size – the link to cancer or other outcome may be very slight; but if they find any link at all, they are tempted to get it published to beat others.  Science is very competitive for both funding and credit.
• Considering a greater number of relationships – the more you look for the more you will find.  In June 2015 I gave the example of a paper claiming a positive relationship between eating chocolate and losing weight.  Although the author did not make up any data, he later admitted that he had compared so many variables that one or two were bound to show a relationship just by chance.  It so happened that he found this very counterintuitive and alluring relationship, chocolate and weight loss, and published it as a tongue-in-cheek report.  Unfortunately, before he could correct the record, news agencies around the world ran with it as breaking scientific news.
• Greater flexibility in experimental designs – a lack of standardization in the design of a test can cause results to vary.
• Greater financial and other interests and prejudices, especially in a hotter scientific field – scientists have personal interests and biases just like all other humans.  They may unconsciously try harder to make the data fit their theory.  They may not check a supportive result because it is the answer they and their sponsor or employer are looking for.

Another problem is that scientists are less likely to spend time replicating research to confirm findings.  It’s hard to get funding for merely repeating someone else’s work.  It’s also not as interesting, and you are less likely to get headlines or other recognition. 

Fortunately, this idea of testing the results of others is catching on.  In 2011 a group of psychologists from around the world tried to replicate findings of 100 published papers from 2008.  They could reproduce the results from only 39.  Furthermore, “Daniele Fanelli, who studies bias and scientific misconduct at Stanford University in California, says the results suggest that the reproducibility of findings in psychology does not necessarily lag behind that in other sciences.”  They confirmed that twelve-year-old assertion that just because it’s published in a reputable journal doesn’t necessarily mean it’s true. 

So where does it leave critical thinkers who want to rely on scientific data instead of social media rumors or endorsements and anecdotal evidence from advertisers?  We can’t jump to conclusions based on every “latest discovery” seen on Dr. Oz or the news.  Be patient.  Look for large sample sizes and confirming tests.  In this fast-paced technological age, it is doubly difficult to separate the valid from the bogus, but virtually none of the breaking news requires an instant response.

Bogus Warnings

They may come in the form of spam e-mails or desperate pleas on social media, but the bogus warnings are out there.  The people who send them seem to be sincere in their concern.  The people who forward them seem to be lax in their interest to do a little simple research to determine their veracity (Critical Thinking).  But there seems to be no shortage of warnings about avoiding certain foods to ensure our future good health.  Maybe everyone wants to perform a heroic act saving their friends and the population in general, but the result is really a bunch of Chicken Littles crying, “The sky is falling,” and distracting everyone from actual serious concerns.

One example comes by way of social media from a site called Healthy Holistic Living.  The article is undated, but was picked up by a number of other sites in August 2014.  The headline, “The Noodles that Cause Chronic Inflammation, Weight Gain, Alzheimer’s and Parkinson’s disease,” is a real attention grabber.

It tells of a “first-of-its-kind experiment” by a doctor at Massachusetts General Hospital who “used a pill-sized camera to see what happens inside your stomach and digestive tract after you eat ramen noodles, one common type of instant noodles.”  He found that “even after two hours, they are remarkably intact” which may put a strain on your digestive system.

But wait, there’s more.  They also contain poison!  The noodles contain TBHQ, “a synthetic chemical with antioxidant properties – not a natural antioxidant.”  It’s a commonly used ingredient in processed foods of all kinds but is also used in lacquers, perfumes, varnishes and pesticides.  They go on to warn of a bunch of possible nasty outcomes from eating any non-homemade instant noodles.

This seems very bad on the surface. Has the FDA dropped the ball and the only ones who care about it are the healthy-holistic-living people?  From the rest of the article, the bias becomes clear.  To them all processed foods are bad and potentially poisonous.  This experiment from Mass General just reinforces that view.  So they took that information and combined it with a study from South Korea, where they eat a lot of ramen noodles, showing some adverse health effects.  They published their conclusion linking the noodles to high blood pressure, high blood sugar levels, increased risk of heart disease, stroke and diabetes.

The problem is that the South Korea study relied on self-reporting, results have not been reproduced and instant noodles haven't been isolated as a single factor in adverse health outcomes.  The study itself says findings are not necessarily applicable to subjects outside that country.

This rumor apparently was resurrected by NextShark website, published in July 2016 attributing the South Korea study to Harvard (to try to add some credibility).  An old, non-scary study was rebranded to get attention or confirm an agenda.

A similar scary rumor had to do with Trisodium phosphate (TSP) in children’s breakfast cereals.  Although it can be used in high concentrations for cleaning walls before painting, it is "generally recognized as safe" by the FDA and is also approved for use by food safety standards agencies in the European Union.  It is not a paint thinner, solvent or acetone as the author would have us believe.  It’s probably safer for the children than all the sugar found in those cereals.

I guess two good questions come from this.  Shouldn’t we do some research before panicking and reposting these continual rumors?  Are we going to trust the FDA with the mission of keeping us safe from dangerous foods, some painter who noticed similarities with his work on cereal box ingredients list, or a website with a specific agenda of disparaging all processed foods?  The FDA does make a few mistakes, but I think they are much more reliable over the long run.  It’s a matter of critical thinking and a little perspective – calm down, get a grip and investigate before spreading bogus warnings.

Scientific Studies

A link to an interesting YouTube segment has been going around.  John Oliver talks about scientific studies in his humorous and irreverent way.  But many of the points he makes should be taken seriously.

Television and FaceBook are filled with studies and many of them seem to contradict each other.  The problems he points out include that scientists are under pressure from their academic institutions or employers, and negative results don’t get published – even if it would be important to know those negative results (to keep from wasting money, for example).  The confusion about which study to take seriously comes from the lack of efforts to replicate previous studies, a very important step in the scientific process.  If a study cannot be replicated by peers, as well as reviewed to ensure proper procedures were followed, its result is not considered valid.  But the pressure to find something new and exciting reduces the availability to get funding for replication studies, meaning those one-time results may have been a statistical fluke rather than a great scientific finding.

To get those all-important, positive, newsworthy results, researchers may resort to various gimmicks, such as using a small sample size, testing for so many variables that at least one will by chance show a statistically significant result, publishing results from lab rat studies as if they are equally valid for humans (which they are not) or publishing a press release with a sexy headline hoping the journalists will not dig too deeply into the substance of the limited experimental findings.  In addition, there are a lot of charlatans on TV and on the Internet describing themselves as scientists and using a lot of scientific-sounding jargon to sell whatever they have to offer.

This is an important subject, and if treating it humorously gets the point across, all the better.

After spending the 19 minutes watching that YouTube version of the Last Week Tonight episode from HBO, I soon saw an example of the problems.  The next day a Health Minute episode on local TV news featured a new study proclaiming that one minute of vigorous exercise was as good as a 45-minute moderate workout.  “Not having the time to exercise is no longer an excuse,” they announced.  I couldn’t find the same piece on the Internet, but instead found this New York Times wellness blog with exactly the same message.  The headline read:  “1 Minute of All-Out Exercise May Have Benefits of 45 Minutes of Moderate Exertion.”

Well, if you read only the headline, you have gotten two things:  another excuse to take it easy and a bunch of bad information.  As it turns out some folks at McMaster University in Canada chose 25 out-of-shape young men, took some biometric data from each and randomly split them into 3 groups:  one to ride moderately on an exercise bike for 45 minutes 3 times a week for 12 weeks, one to ride for only 10 minutes with three 20-second bursts of intense riding, and one to do nothing out of the ordinary (the control group).  They compared readings at the end of the test and found that the physical improvement for groups one and two were comparable and better than the control.

This is good information for those interested in interval training effectiveness, but the sample size was so small (8 per group) that the room for statistical error based on individual differences was huge.  Also, dividing groups randomly is a good default when there is not a better way to do it, but with only 25 people, it shouldn’t have been that difficult to come up with a more careful method.

Without getting too deep into the experimental design, it’s pretty easy to see that this was a small test without much rigor and certainly not what the headlines would lead us to believe.

This is why critical thinking is so important.  The news media, even the reputable NY Times, don’t care so much about the details as they do about catchy headlines to sell newspapers and airtime.  We can easily be led astray.  We can waste money or go down a dead-end path with our lives by putting our faith only in those studies that seem to tell us what we want to believe anyway.  And if you don’t like the study this week, just wait for the next one to come around.