Why FDA Approval?

With the desperate search for a coronavirus vaccine and medicines to reduce the seriousness of the disease comes a lot of discussion about FDA approval. One side argues that speed is of the essence and we need to accelerate the process. The other side argues that we can never be sure of a drug’s effectiveness without proper testing. (This will reflect some of the information from last week’s entry, “Understanding Experiments.”) 

The FDA admits, “It takes on average 12 years and over US$350 million to get a new drug from the laboratory onto the pharmacy shelf.” The first step requires about three years of laboratory testing before testing on humans. “Only one in 1000 of the compounds that enter laboratory testing will ever make it to human testing.” 

Next comes a three-phase process. First they must test for safety. Next they need 100 to 300 volunteers to test for basic effectiveness. Finally 1000 to 3000 patients in clinics and hospitals are monitored for effectiveness and to identify adverse reactions. Each phase can easily take years to complete. (And some people wonder why drugs cost so much and why the companies fight and scheme to protect their patents.)

In the interim some drugs that have been previously approved for other diseases are being tested on coronavirus patients. This practice of prescribing drugs off-label is really quite common in the US. WebMD writes, “Off-label prescribing isn't necessarily bad. It can be beneficial, especially when patients have exhausted all other approved options.”

This is why FDA approval is so valuable and why makers of so many pills, salves, ointments and medical devices are so eager to try to take credit for having cleared this hurdle. Otherwise the law requires them to post the familiar disclaimer: “The statements made regarding these products have not been evaluated by the Food and Drug Administration. These products are not intended to diagnose, treat, cure or prevent any disease.” (But the disclaimer is familiar only to those who bother to read the fine print.)

If companies fail to do this, it’s not the FDA but the Federal Trade Commission (FTC) that comes after them for misrepresentation, as they did in this case – one of many that show up almost weekly.

“Under a settlement with the Federal Trade Commission, the marketers of an electrical nerve stimulation device called Quell have agreed to pay at least $4 million and stop making deceptive claims that the device treats pain throughout the body when placed below the knee and is clinically proven and cleared by the Food and Drug Administration (FDA) to do so.” (From an FTC press release.)

I checked their website and found that they no longer show the device as FDA approved (to treat or cure), but still do try to give that impression through endorsements and research posters. They continue to mention the FDA at least twice on the site, referring only to registration, not approval – big difference. In summary, the “FTC alleged that the defendants lack scientific evidence to support widespread chronic pain relief claims…” but the site lists no such disclaimer. It’s big business and a $4 million fine pales in comparison to a $350 million development cost.

That it is “100% drug free and non-surgical” while it “stimulates sensory nerves with safe and precise electrical pulses to trigger a natural pain relief response” appeals to people with chronic pain trying to avoid addictive drugs. It’s understandable how such a device would tempt people, and if placebo does the trick it might be worth $300 plus an on-going expense for fresh electrodes.

Like so many other health and medical decisions, avoiding such questionable products is all about a little research and critical thinking.

Understanding Experiments

When most people hear the word experiment, they picture a scientist in a lab coat with bubbling beakers and test tubes of mysterious liquids. This is not the real meaning. As a result of this misconception, news of the latest study is usually misleading.

An experiment is a rigorous process of testing an idea with the intention of either solving a problem or of improving a situation. Necessary first steps include defining the problem to be solved or determining how results will be measured. No one can claim improvement without measurements to compare (before and after). In the first case, the experiment is successful if the problem goes away. In the second, success is judged by the measurable amount of improvement.

When a well-designed experiment is successful, the implication is that the same solution can be applied widely to other situations: to solve the same problem elsewhere or to achieve the same amount of improvement. (Usually others replicate the experiment before findings are accepted.)

To achieve satisfactory results, any experiment must be well designed. Sloppy studies lead to problematic conclusions, ones that can’t be counted on to solve anything. Even the best experiments can yield bad information just based on the amount of diversity in the world and the fact that fluky things happen. 

Many experiments or studies the public is exposed to in the media relate to drugs or other remedies, and a strict procedure must be followed to ensure the conclusions are valid, otherwise the drugs or other remedies get on the market without proof that they are safe and effective.

In drug studies researchers try to choose a sufficiently large sample, because the larger the sample, the lower the chances of getting some oddball results just because you happened upon some atypical participants. A bigger sample tends to average out any unusual individual readings. 

The next step is to divide the sample into two parts that look as much alike as possible: same proportion by sex or race or education or income or background or location or any other feature that might affect the results. Sometimes this can only be done by random assignment to one group or the other. It's better if researchers have a good understanding of all the characteristics that might influence results and can make the two groups look as much alike as possible relative to those characteristics.

One group is treated – given the pill or the information or other treatment – the other, the control group, is given a fake pill (placebo) a sham treatment or left alone.

Afterward the groups are compared statistically to see if the change (hopefully an improvement) in the first group is significantly better than the change in the second. (Yes, there is often a change in the control group merely because the know they are participating in the experiment and believe the placebo is a real remedy.)

Ideally the people doing the testing are not aware of who is in which group (double-blind).

We know from the news that even these careful experiments can go wrong. Sometimes drugs are withdrawn from the market due to problems discovered only after they are released, released to a much larger sample size. That is why I have come down so hard and so often on vitamins and other dietary supplements where the law exempts them from the need for any research at all to prove safety and effectiveness. (They rely on endorsement, not proof, and use weasel words to imply effectiveness.)

Many times in the past I have also criticized experiments because they have been sloppy about their design. Sometimes they don’t define the problem until after the test is done and announce the findings with a press release. Sometimes the samples are too small due to budget constraints or laziness. Sometimes they rely on self-reporting so there is no real measurement. 

Finally businesses and educators like to say they are experimenting. In both cases it is rarely true. They don't set up two groups to compare. Then they measure by gut-feel.

Businesses are in too much of a hurry to do it right. They just try things, sometimes multiple things at the same time, so who knows which ones have a positive or negative or neutral influence on the final outcome.

In education they are still arguing about measurement. Teachers don’t want to be paid based on test scores, but haven't suggested a more acceptable criterion to measure their results. Yet school systems continue to try new methods and approaches without ever satisfying the first step – how to objectively measure real improvement.

Without understanding experiments, it’s too easy to be fooled by people who don’t know what they are doing and by people who do but are just trying to sell us a bill of goods. 

Appropriately Skeptical

From time to time I see an ad from a local doctor promoting stem cell therapy for pain. It includes one endorsement telling how the office was so easy to work with and how the pain went away. For some time now I have been skeptical about the validity of this promotion. I thought stem cell research was not yet in the mainstream of medical practice, and I know that endorsements are not reliable evidence. It seemed suspicious, so I decided to look into it.

The promotional material on line states in part: “Stem cell therapy uses your body’s most powerful and flexible cells to promote natural regeneration in damaged joints.” It claims to be effective for a list of conditions: surgical wounds, tendon damage, severe muscle strain, arthritis, damaged cartilage, torn meniscus and many other joint and soft tissue injuries and conditions.

“The stem cells … come from your own body … a small quantity of bone marrow from your large hip bone.” (Using the recipient’s own stem cells avoids the need for FDA approval.) The on-site lab processes the harvested cells. The doctor then can “precisely inject your stem cell mixture where you need it … where they morph into new cell types” and provide “long-lasting relief” for “many [but not all] patients.”

My first stop was WebMD with a sub-headline reading: “Unproven, Risky Treatments Mislead Patients to Seek Cutting-Edge Therapy.” International Society for Stem Cell Research (ISSCR) “advises patients to seek only stem cell treatments being tested in clinical trials approved by the FDA”… or some approved smaller studies.

There is a lot of information on the site but it is summarized like this: “Every treatment has some risks. So the question comes down to whether the benefits outweigh the risks. And those studies haven't been done yet.” 

This Healthline report from about 18 months ago says that stem cell therapies are being promoted for a wide variety of conditions, but “very few of these applications have any scientific backing, [however] stem cell therapy for knees has been the subject of quite a few promising studies.” It would be a big improvement over knee replacement, but promising studies hardly qualify a procedure for routine outpatient application. They list the cost of this treatment at “approximately $3,000 to $5,000 per knee, depending largely on geographical location.” 

The FDA gives much of the same information. “Stem cell products have the potential to treat many medical conditions and diseases. But for almost all of these products, it is not yet known whether the product has any benefit—or if the product is safe to use.”

Consumer Reports chimes in with a sub-headline: “A new industry is booming. But critics worry that the treatments are ineffective and dangerous.” Here is a very brief summary: “Stem cells are special cells with the potential to repair damaged tissue and organs.” But “not all of them are possessed of equal power.” According to the current scientific consensus, the therapeutic potential of stem cells taken from fully developed tissues, adult cells, is believed to be much more limited. (Does this include marrow from hipbones? I wonder.)

That Consumer Reports piece is a long article with many warnings about the confusion in the field and the inability of regulators to keep up with the proliferation of clinics, both legitimate and otherwise.

Another reason I am skeptical is that the same doctor offers many other questionable services including platelet-rich plasma (PRP), calling it “one of the most powerful natural healing methods available today.” Some conditions include dry eyes, hair loss and erectile dysfunction.

The Pain Science website opinion published earlier this year describes it this way: “injections bathe troubled cells in a concentrated mixture made from your own blood. Hopefully this stimulates healing where it is otherwise failing … but no one really knows for sure yet.” Regardless, it’s not hard to find someone willing to do it for you, but it’s not cheap. “Without any clear evidence of benefit beyond placebo, PRP is now being marketed aggressively as a cure-all for sports injuries.”

I did this research not because I would consider going to a pain specialist unless I was referred by my primary care doctor. I did it just to show how careful we need to be to avoid spending a lot of money on dubious treatments. 

I didn't name this doctor because I'm sure there are similar ads running all over the country. I am not demonizing any doctor who may truly believe in such treatments, but it is a warning to patients everywhere. 

It is nearly always better and safer to be appropriately skeptical.