14.8: The Scientific Method

Let's now summarize the major points about the scientific process of finding our way to the truth. Scientific sleuths conjecture possible explanations—the so-called scientific hypotheses—then try to test them in order to rule them out. Testing is not normally a passive process of observation. Instead, it is an active attempt to create the data needed to rule out a hypothesis. When a hypothesis can stand up to many and varied attempts to rule it out, we have confirmed the hypothesis. Optimists say we have produced a scientific proof of the hypothesis. But the proof is always tentative because scientists also realize that any hypothesis or theory that is proved today is likely to be shown in the future to be an approximation of an even better hypothesis or theory.

The key idea of the scientific method is that the true hypothesis cannot be ruled out; the truth will survive the ruling-out process. So if we were correct in suggesting that A really does cause B, then observing and testing will eventually produce data inconsistent with the alternative suggestions that B causes A instead, and we can rule out the possibility that some C causes both A and B to be correlated.

This whole procedure of doing science might seem easy at first, but actually scientific proofs are not easy to come by. A major problem is lack of imagination. If you cannot imagine the possible lurking C's, how can you be expected to rule them out? To illustrate, consider why having lung cancer is associated with buying a copy of The New York Times newspaper on Tuesdays. The claim implies that the percentage of lung cancer cases is higher among those people who buy this newspaper on Tuesdays than among those people who don't buy it. Why? Although some associations are accidental, just statistical flukes, this one is not. So which causes which? Is buying the paper on Tuesday causing people to get lung cancer, or is having lung cancer causing the paper-buying behavior? Finding the answer scientifically is much like solving a mystery. Can you solve it?

Here are some suggestions to pursue. What is special about the ink on Tuesday's newspaper? Could there be a cancer-causing agent in the ink? Or should we give up on showing that the buying causes the cancer and try to go the other direction? Maybe cancer victims have a good reason to buy the paper. Maybe Tuesday is the day the paper contains the most articles about cancer, so lung cancer patients are especially likely to buy the paper on that day.

Unfortunately for these suggestions, a little testing will refute them. Perhaps you spotted the error in the reasoning. You were being misled away from the real causal story behind the association between lung cancer and Tuesday Times buying. First, there's a significant association between lung cancer and Times buying on Friday, too. And Wednesday. The real story is that living in a city tends to cause both the lung cancer and the Times buying on Tuesday; it causes their association. Living in a city, any city, makes it more likely that a person will take in cancer-causing agents such as air pollution. Also, living in a city makes it more likely that a person will buy The New York Times on Tuesday, and Wednesday, and other days. Country folks are less likely to read big-city newspapers—on Tuesday or any other day. So, city living is the cause. By asking whether lung cancer causes the paper buying or vice versa, I was purposely committing the false dilemma fallacy. Pointing out that something else could be the cause of both is a way of going between the horns of this dilemma.

What this story demonstrates is that proving something scientifically can be quite difficult because of lack of imagination. Scientific proofs can't just be "ground out" by some mechanical process. Your leap from association to cause is successful to the degree that all the other explanations of the association can be shown to be less probable, and this can require a considerable amount of scientific imagination. This same point can be re-expressed by saying that if somebody can imagine an equally plausible explanation for the association, or a more probable one, then your original inductive leap from association to cause is weak.