Popular Posts

Showing posts with label anecdotal evidence. Show all posts
Showing posts with label anecdotal evidence. Show all posts

Friday, June 12, 2020

Flashback - Extending Unemployment Benefits

[Early in 2012 I used a question about the motivational aspects of unemployment benefits to show how sloppy news organizations are about data gathering. This kind of "research" continues to this day without being challenged, while Congress raises the issue of further extending unemployment benefits in light of the coronavirus lockdown. ]

First, this is not about the pros and cons of extending unemployment benefits. It’s about using critical thinking to deal with the information we receive on any subject. This just happens to be a good example. 

At the end of this CNN Money article they ask: “Do you receive unemployment insurance? Have you not been looking for a job as hard because you are getting benefits? Email [address] with your contact information and you could be contacted for an upcoming story or video.”

Two thoughts. As I have pointed out many times before, testimonials or anecdotes are not evidence. These reporters will receive e-mails and sort through them probably to find ones to support the story they want to tell. Second, even if they condensed the information received and delivered a summary, this is an extremely poor method of sampling. Their data would not be representative and any statistical conclusions would be meaningless. How many people do you seriously think will e-mail to say that they are just sitting around until their unemployment runs out, making no effort to find a job?

This practice of finding an individual example and selling it as the plight or experience of many is common among reporters, politicians and advertisers. They want to put a face on the crisis, but it borders on deception. Often the purpose is to gather support for a cause or product by eliciting an emotional reaction from the viewers or voters. It steers us away from the kind of critical thinking and analytical problem solving required to get us (the US) out of the mess we are in.

[By the way, if I had been inclined to write about the motivational effects of unemployment benefit extensions, I need look no farther than a National Bureau of Economics Research working paper in 2015 after the recession. Surprise, surprise! They estimated “the abrupt end of unemployment benefit extensions led to 1.8 million additional new US jobs....”, that is, new workers!]

Monday, January 20, 2020

What is Income Inequality?

What is the opposite of income inequality? It certainly can’t be absolute equality of income. Without being more specific, how can the term mean anything?

For years we have heard about goals like clean water and clean air. The US has been moving in the right directions. According to the EPA, “Since 1970, implementation of the Clean Air Act and technological advances from American Innovators have dramatically improved air quality in the U.S.  Cleaner air provides important public health benefits.” Follow the link to find a graph of nine measured pollutants, excluding CO2. It shows a downward trend since 1990 for all, excluding unusual events like dust storms and wildfires. Overall, pollutants have dropped significantly despite the fact that “Americans drove more miles and population and energy use increased.”

All the pollution levels dropped below a line labeled “Most Recent National Standard” as of five years ago. Note that there is a standard that defines an acceptable level of particles in the air. Totally pure air is not a practical or realistic goal.

Similarly, the 1974 Safe Drinking Water Act “resulted in major changes in the way drinking water is managed and treated in the United States, and it achieved substantial measurable benefits in risk reduction and public health protection.”

On a side note, these facts do not seem to sway Americans influenced by media reports designed to stimulate concerns about hypothetical risks related to drinking water. A survey from 4 years ago showed that 56% of Americans “were concerned or very concerned about their tap water, 77% were regular users of bottled water, and 43% used some type of home treatment device.” The water industry and regulators have their work cut out for them figuring out how to promote acceptance and confidence in public drinking water supplies “and to maintain the public’s support for needed improvements.”

Once again there are measurable standards. Customers of public water systems receive a report from their water utility every year showing how the water supply measures up to various standards. Again, absolute purity is unachievable.

The same is true for lakes and rivers. “Human health ambient water quality criteria represent specific levels of chemicals or conditions in a water body that are not expected to cause adverse effects to human health.” 

Applying this to income inequality leaves critical thinkers puzzled.

The news media emphasizes stories about the shrinking middle class. “A graph from Reason, however, shows that about 50 years ago, 53 percent of people were middle-income, making between $35,000 and $100,000 per year. Although that statistic has since fallen to 42 percent, the reason [behind the change] is that many people moved into upper-income brackets. The share making more than $100,000 rose from 8 percent to almost 28 percent. (These numbers are inflation-adjusted.)” It’s shrinking, but not in the direction the news media and politicians imply.

Stories about income inequality don’t mention this information. Instead they attempt to stir up envy toward the super-rich, at least some of the super-rich. Notice that the emphasis is always on bankers, CEOs and hedge fund managers, not on sports stars, celebrities, Internet influencers or software developers. We are supposed to be irate that some people are rich, undeserving jerks, but others deserve wealth because they entertain us. Often the people who report these stories are rich themselves.

Furthermore, the evidence that income inequality negatively effects economic growth or social mobility is based on correlation, and, as every student of statistics knows, correlation does not mean causation. Whether or not income inequality is a drag on economic growth remains debatable.

But beyond that, it makes no sense to get upset about income inequality until it is defined and a goal it set. Anything that can’t be measured can’t be said to improve (or get worse). With no goal, there is no definition of victory. (This can also be said about so many other vaguely defined points of contention such as diversity, sustainability, and social justice.)

Friday, December 27, 2019

Flashback - Ignoring The Facts

[This one comes from January 2012 and is still one of my favorites. People, who are involved in a test that proves that a $30 item is not effective, continue to insist on buying one. Note, they didn't just hear on TV or read on the internet that the item is bogus - they were actually involved in testing it and shown the results. It didn't work as advertised, but they still wanted one regardless of what their eyes were telling them because of all the hype they had heard elsewhere! Being swayed by marketing promotions and celebrity endorsements is not wise behavior.]

While looking for a different article on the Placebo Effect, I found this interesting video. As you know, the Placebo Effect causes people to recover or show improvement even when they are given only a fake remedy, the familiar sugar pill. The power of the mind is amazing.

I recall an article a number of years ago stating that the improvement within the control groups in certain drug studies increased as the power of the real medicines increased. They were getting the sugar pill but believed they were getting the latest, stronger medicine, so they experienced even better results. It was all psychological. It is easy to understand how this reaction could play into the hands of "snake oil" salesmen. They sell you a useless cure and let your mind do the rest. Then they get a few believers who help them sell more. That's why I always say, "Endorsements are not evidence."

Here is the  CBS News video testing the claims of a particular performance wristband, one endorsed by famous athletes, and touted to improve athletic performance. When test subjects were given the advertised wristband, their performance on physical and balance tests did improve. But when they were given a one-dollar replacement band and told it also was special, performance on physical and balance tests improved comparably. The video also explains how the bracelet company sets up tests to ensure the perception of improved performance.

The scariest part is the ending where, despite the fact that the $30-wristband claims had been debunked, shown to be no more effective than the $1 bands or even not wearing a wristband at all, the participants still wanted one!

This is not critical thinking! This is the kind of decision-making that leads people to spend money on worthless items instead of using it wisely. Then they wonder why they get into debt and can't afford to retire. It's the kind of thinking that leads us to vote for candidates based on endorsements or personal appearance or charisma instead of the leadership and ideas we need. It's the kind of thinking that contributes to the decline of America!

Monday, May 21, 2012

Earthing - Keep Your Feet on the Ground


When I read about this last week, I could only think, “Where has people’s critical thinking gone?  Earthing, proclaimed as “the greatest health discovery ever” and “right up there with the discovery of penicillin,” is the practice of walking outside barefoot or in some other way increasing contact with the earth to allow us to absorb free electrons from the earth’s surface leading to improved health.  There is a detailed explanation of how this works along with – you guessed it! – lots of endorsements.

How many times have I repeated that endorsements or wonderful stories are not proof of effectiveness nor is selling supposed remedies as ancient wisdom, old knowledge rediscovered, or the unlocking of some secret?  When I read this, several questions immediately came to mind.  Why are the earth’s electrons any different from other free electrons?  Why did our ancient ancestors who had more contact with the earth have a much shorter life expectancy?  Why don’t our doctors tell us to walk outside barefoot more often – some kind of conspiracy to withhold the information or are they just tired of giving tetanus shots?  If electrons from the earth enter a body so easily, why can’t you slowly drain a flashlight battery by holding both ends between your fingers?  Where is the research?

I actually did find some research.  Here is one paper describing a particular study.  It had 12 participants.  The researchers grounded their beds and measured cortisol levels as well as participant reports of pain, stress and sleep dysfunction.  This seems like pretty shabby experimental design.  How were the 12 chosen?  Such a small sample size limits the statistical validity.  Where was the control group and “double-blind" setup to guard against placebo effect?  Self-reporting is prone to error.  This isn’t even in the same league as FDA requirements for any approved treatment.  It’s almost like getting 12 people together for an endorsement party.  Other studies are cited, but to what end?  Their website lists 13 bullet point benefits and implies that it's just the tip of the iceberg.  Snake oil, anyone?

The disclaimer on the site is most telling:  Products and information on this site are not intended to diagnose, treat, cure, or prevent any disease.  Of course not, that would probably be illegal.  (I don't doubt their sincerity, since everyone just walking around barefoot would fit their theory and not profit them a dime. but sincerity is never proof of validity.)  In any case, they strongly imply throughout that health benefits abound while making it easy to order their $60 mats, $200 kits and $300 bed sheets. 

One hundred dollars here, one hundred dollars there – it all adds up -- and a non-skeptical, non-critical thinking society then finds itself wondering how can I afford to retire or why am I part of a $1 trillion college debt crisis?

Monday, April 23, 2012

Psychic Healing



In February a Colorado woman was accused of stealing nearly $300,000 from clients and ordered to serve five years in prison and to pay restitution.  According to testimony, she told her clients that she needed their cash to draw out the bad energy and needed to check their credit cards to see how frequently the number 6 appeared.  She kept the cash and used the credit cards at local department stores.  She was found guilty on 14 counts of fraud and 2 of tax evasion. 

Imagine how desperate people must be to consult a psychic healer to help them solve their problems!  Despite articles like this one, and many others besides, that show psychic healing as well as other so-called psychic abilities are unsupported by scientific theory or actual results, Americans still pursue this as an avenue of relief.  In this case they were lucky and may get some of their money back.

Is it fair to paint all psychics with the same brush, labeling them all frauds?  Perhaps not, I’m sure some sincerely believe in the abilities they claim.  Still, James Randi, magician and debunker of things psychic, has for many years offered a $1 million prize to anyone who can demonstrate true psychic powers under controlled conditions.  He has gotten few takers and no winners.  Yet police sometimes call in psychics to help solve crimes.  If they don’t, they risk being accused by victims or their families of not doing “everything possible.”

Back in 1996 a nine-year-old girl for a science project devised a method of testing the validity of therapeutic touch, an “energy medicine” which claims to promote healing and reduce pain and anxiety when therapists move their hands over a patient to manipulate his or her “energy field.”  The girl contacted 21 experienced practitioners, sat them behind a screen and asked them which of their hands was experiencing the energy field from her hand.  Their success in detecting the energy field was no better than random guessing.  Results of this experiment were written up in the Journal of the American Medical Association.  Yet the State of California awards continuing education credits to nurses who learn this technique and “an American Hospital survey conducted in 2005 noted that about 30% of 1400 responding hospitals offered [and presumably charge for] therapeutic touch.”  


What we see here are further examples of uncritical acceptance of alternative practices based on anecdotal evidence rather than scientific testing.  This seems to be the trend.  We are encouraged to be open-minded, but there is a difference between being open-minded and foolish when it comes to our health and how we spend our money in general.

Monday, January 9, 2012

Can You Live on $50,000 a Year?

An interesting article from NBC asks that question, but, as in all cases, the article must be read carefully.

The reason they pick $50,000 is that this is the new household median income, and (adjusted for inflation) it has fallen over the past few years.  They ask readers to send in their stories.  Just as I said last time, this idea of telling someone’s story is typical of reporters and often misleading.  They are trying to get examples of people to show how they are struggling to make it.  (This is my assumption, that they are trying to highlight another "crisis." I admit that I am reading into their motives.)  None of these examples, however, will be typical, because later in the article they explain that the median income of a family, two or more related people living together, is about $62,000 and that of a single person is just under $30,000.  So a family at $50K would be below average and a single person at $50K would be above.

Nevertheless, my main point comes from the interesting graph near the top of the article.  It shows the median income (adjusted for inflation) over the last 50 years.  Despite the recent dip, the overall trend is upward.  Looking at the big picture, the long term, in other words using perspective, we see that on average we are better off than we were in 1965, 1975, 1985, 1995, even 2005 (in pure dollars, adjusted for inflation).  If we could get by then, we should be able to get by now – unless our needs and expectations have change significantly.  As usual, there is no crisis, just a problem of behavior.

What drives the question of whether we can make it on $50K is the underlying assumption that we must be on a continuous upward slope, with no year-to-year dips.  As our resources increase, so do our appetites - our wants, not our needs.  The graph shows that the world doesn't act that way.  The upward slope is punctuated by dips and sharp rises.  Why should we expect different?  This is what, not too many years ago, led so many people to be lured into buying houses they eventually couldn’t afford.  They were encouraged (by bankers and realtors) to assume that their income and the value of their house would both move on a continuous upward path.

The article presents a picture of America self-deceived with the notion that life will steadily get better, with no temporary setbacks, either personally or nationally.  How unrealistic!  Based on that assumption, wants from years ago become the needs of today and we seem unable or unwilling to adjust.  Our requirements keep growing, and what was sufficient to live on comfortably ten or twenty years ago, no longer is.  As C. S. Lewis so well put it:  “Whatever men expect, they soon come to think they have a right to:  the sense of disappointment can, with very little skill on our part, be turned into a sense of injustice.”  What's needed is a sense of perspective.

Friday, January 6, 2012

Extending Unemployment Benefits

First, this is not about the pros and cons of extending unemployment benefits.  It’s about using critical thinking to deal with the information we receive on any subject.  This just happens to be a good example.

At the end of this CNN Money article they ask:  “Do you receive unemployment insurance? Have you not been looking for a job as hard because you are getting benefits? Email [address] with your contact information and you could be contacted for an upcoming story or video.”

Two thoughts.  As I have pointed out many times before, testimonials or anecdotes are not evidence.  These reporters will receive e-mails and sort through them to find ones to support the story they want to tell.  Second, even if they condensed the information received and delivered a summary, this is an extremely poor method of sampling.  Their data would not be representative and any statistical conclusions would be meaningless.  How many people do you seriously think will e-mail to say that they are just sitting around until their unemployment runs out, making no effort to find a job?

This practice of finding an individual example and selling it as the plight or experience of many is common among reporters, politicians and advertisers.  They want to put a face on the crisis, but it borders on deception.  Often the purpose is to gather support for a cause or product by eliciting an emotional reaction from the viewers or voters.  It steers us away from the kind of critical thinking and analytical problem solving required to get us (the US) out of the mess we are in.

Monday, August 29, 2011

Critical Thinking Requires a Little Understanding of Statistics

I made a comment a couple of weeks ago about people spending more to buy organic or all-natural products at the grocery store (and other places) without being able to justify the added cost with some tangible benefits.  Of course one of the benefits is the good feeling that they are taking better care of their families, but if the evidence shows that there is no difference, it’s money wasted, money that could be used in other ways to make the family’s life better.

What is evidence?  Is it a news item on TV or the Internet citing a study that showed health benefits?   Listen closely and we find that many things that were bad for you a couple of years ago are now good for you (e.g. coffee, wine and chocolate) and some of the things that were good for you are now either bad for you or have no effect whatsoever – several supplements fall into this category and within the past month a doctor warned that it could be dangerous to be drinking a lot of water when you are not thirsty.  (This came with a shocking headline, “Can Water Be Bad For You?”)  So the statements and studies come and go.  Some evidence is trustworthy and some is not.  Just because someone wrinkles his nose and sneers at the limitations of “Western Medicine,” it doesn’t mean that everything the Chinese did for thousands of years is better, or even effective.  We can’t necessarily take for truth what we hear or read without at least some preliminary investigation.  And we shouldn’t follow the example of the boxer in Paul Simon’s song – “A man hears what he wants to hear and disregards the rest.”

There is a lot to consider when we hear reports.  Sometimes the author of a study wants to beat others in order to get credit for a discovery by issuing a press release with “preliminary result.”  If these results later don’t hold up we never hear about it.  Studies often contradict each other, because they are designed differently or some are poorly designed or those tested differ in some way, or the effect is borderline and the results can go either way.  Some studies are sponsored (paid for) by organizations that favor a particular outcome.  Is the sample large enough or chosen properly?  In health-related studies, it’s a long way from lab rats to people and most conclusions cannot be applied directly.  

Do interest groups or news agencies give us enough of this information to make an informed decision?  Usually not, and if they do it's at a point where most people have stopped reading.  Sometimes the headlines are slanted in a way to get our attention or sway our beliefs.

So it’s left up to us to try to understand the science and statistics – a tall order.  Here is an article I found recently to help work through some of the issues.  Still, remember when faced with claims of the miracle cure or the cleaner, safer, healthier product, it’s better to be a little skeptical.