Sample Size of One

Three weeks ago at Noblesville West Middle School in central Indiana, a student left the classroom and returned with two handguns.  He shot one student and a teacher, who heroically tackled him and wrestled the guns away. He was soon taken into custody and charged with attempted murder.

About two weeks later the news came out that the shooter would not be charged as an adult because he is only 13 years old. Indiana law allows children as young as twelve to be charged as an adult for murder but the cut-off for attempted murder is fourteen. Of course, legislature is springing into action to correct this apparent injustice. US News reported that the speaker of the house asked state lawmakers to review the law.

This is a good example of what we used to call in business a sample size of one and is typical of government action.  Someone sees an error, a problem or an accident and wants to fix it.  But that same person, and others who look on, never ask how common is it. Is it common, or is it a fluke?  That never seems to matter.  We don’t ever want it to happen again, regardless.  Laws are changed and new regulations are put in place. Energy and resources are spent on one-off issues just to show that action has been taken.

This reaction is not limited to lawmakers. People do this every day when they accept the word of a friend or relative about the effectiveness of some product or practice instead of looking at the research.  The endorsement of a celebrity or even some unnamed doctor (who it turns out may be part owner of the company) will sway them to send away for a miracle cure advertised on television or on the Internet.

So many times in the past when reviewing scientific research or health studies, I have emphasized the need for an adequate sample size. The exact calculation is somewhat complicated, but a good rule of thumb is that a sample with fewer than 100 subjects is questionable. A small sample with fewer than 30 usually calls for a special statistical approach. But studies make the news where researchers or pollsters work with 15 or 25 (perhaps not even carefully chosen) people and publish results. No matter, a single occurrence cannot even show a trend.

Remember, plane crashes and volcanoes make the news; the 100 deaths a day from auto accidents don’t. And when newsworthy things happen, no one ever stops to ask the questions: what percent are we talking about, how important is it to insure against a similar occurrence or how big is the problem?

So a single incident of a school shooting in Indiana where the shooter happened to be under fourteen leads to a review of the law. I’m sure the lawmakers will feel good about avoiding possible miscarriages of justice in the future, no matter how rare it may be. Do they think that the next thirteen-year-old is going to stop to consider the enhanced penalty, or is it just a matter of fairness to punish a kid, who is probably in far more need of some kind of psychological intervention than decades in prison?

It often takes only a sample size of one to get people so upset and excited that they start demanding action.