Biology teachers often start their courses by reviewing the scientific method. Stripped down to essentials, this means:
1. Formulate a hypothesis.
2. Run an experiment to test the hypothesis.
3. Tentatively accept your hypothesis if the experiment works; otherwise, go back to Step 1.
This story isn’t entirely wrong; genetics really does begin with Mendel’s plant hybridization experiments. If you thoroughly peruse a typical biology textbook, however, much of the content isn’t based on experiments. Take anatomy. Do biology textbooks teach students about the circulatory system by describing experiments? No. Instead, they present and discuss a tidy diagram like this:
Are the textbooks glossing over a pile of experiments from which these diagrams derive? Again, as far as I can tell, no. These diagrams are based not on experiments, but on something totally different: painstaking observation. We don’t know the heart pumps blood because we randomly assigned half the animals in a sample to have their hearts removed. We know the heart pumps blood because a bunch of smart people looked. The same goes for a vast body of anatomical truths. Where’s the experiment testing the hypothesis that the stomach digests food? Or the experiment testing the hypothesis that biceps attach to the radius in the forearm?
My point, of course, is not to criticize biology but to understand it. Naive philosophers of science notwithstanding, biology is packed with useful non-experimental knowledge. In principle, you could experimentally remove animals’ stomachs and show that digestion stops. But you could use the same method to confirm the hypothesis that the brain digests food, because removing the brain also leads to the cessation of digestion (not to mention death). And in any case, what’s the point of these experiments? The prevailing descriptive approach already provides fine answers. Methodological purists who cry, “Science is experimentation” aren’t just silly. If we took their dogmas seriously, we’d have to throw away a vast body of precious knowledge.
The contrast between rhetoric and reality is even stronger in my own field: economics. Economists are vocal proponents of the simplistic scientific method. If they can’t present their work as experimental, they strive to label it “quasi-experimental.” But if you read an introductory economics text, virtually none of the content is based on experiments. Instead, good economics texts are packed with truisms based on calm observation of humanity: incentives change behavior, trade is mutually beneficial, supply slopes up, demand slopes down, excess supply leads to surpluses, excess demand leads to shortages, externalities lead to inefficiency. These lessons are as undeniable as “the heart pumps blood” and “the stomach digests food.” But they’re nevertheless supremely insightful and useful. Designing social institutions without considering incentives is as absurd trying to stuff food down people’s lungs.
But haven’t economists learned quite a bit from experimentation? Of course; my own department is full of experimental economists. My point is that economics, like biology, is packed with precious non-experimental knowledge, too. It’s still science; indeed, it’s some of the best science ever done. We shouldn’t let the genuine triumphs of the experimental method overshadow the rest of the field. And we should staunchly resist anyone who uses methodological dogmas to veto well-established truths – or selectively pretend they don’t exist.
Whether you’re in biology or economics, writing textbooks won’t get you tenure. But the non-experimental knowledge contained within these textbooks could easily be worth more than all the experimental papers published in their respective field the last fifty years. What’s the final verdict? For biology, I’m happy to defer to biologists. For economics, however, I’ve been around long enough to confidently rule in favor of the textbooks.
READER COMMENTS
AlexR
Oct 24 2017 at 3:07pm
Biology has been an experimental science since at least Aristotle. Cutting open an animal, he found organs structured in a particular way and hypothesized that the next time he cut open an animal he’d find similar structures. The diagram of the circulatory system presents a variety of predictions based on experiment, predictions that are tested every time a phlebotomist searches for a vein to make an injection or a heart surgeon opens a chest. Economics, whose “truisms” are likewise based on “painstaking observation,” has been an experimental science since at least Adam Smith. Whether asking college students to pretend to be titans of industry, spurred by the reward of pizza money, is a useful form of economic experimentation is another question entirely.
BZ
Oct 24 2017 at 3:16pm
Makes me harken back to a Don Boudreaux post about predicting the change in the water level of a lake after throwing a small stone into it. He likened the laws of econ to a kind of law of water displacement, where, even where measurement imprecision or concurrent changes in a complex system may make the water-level measurements come out a certain way, still we can know the effect of throwing in the stone per-se.
gwern
Oct 24 2017 at 4:41pm
These examples do not seem very well chosen…
Harvey made extensive use of animal and human experimentation in proving the heart pumps blood: https://en.wikipedia.org/wiki/William_Harvey#De_Motu_Cordis_.28On_the_Motion_of_the_Heart_and_Blood.29
No, it doesn’t. The critical part is the brain stem; you can remove the cortex without affecting surprisingly many things (decorticated animals such as cats can still walk, run, engage in ‘sham rage’ etc), and the entire brain without necessarily killing animals, because the autonomic nervous system is controlled by the brain stem. Hence cases like Mike the Headless Chicken, who lived 18 months without his head and could digest food just fine, ultimately dying (like many humans with ostensible brains) of choking to death: https://en.wikipedia.org/wiki/Mike_the_Headless_Chicken
Perhaps you could ask Dr William Beaumont about that.
Ameek Bhalla
Oct 24 2017 at 10:50pm
In addition to Harvey’s work already mentioned by gwern, I would suggest looking up the experiments of Galen on pigs testing how urinary circulation works.
Curious Chap
Oct 25 2017 at 1:41am
Bryan,
You write here as if the insights gained by non-experimental economics have all been discovered and are now incorporated into basic econ texts. Do you believe that or do you think it is still possible to gain new insights into problems by applying Bastiat-style economic logic to problems?
Toby
Oct 25 2017 at 2:47am
I agree with AlexR on this. The view of what is an experiment is too narrow. The key is that you isolate causes. This requires control of the variable, manipulation, and comparison between manipulating and not manipulating the variable. And control isn’t even necessary if you can be sure, through painstaking observation, that only that variable changes in value.
Grade0
Oct 25 2017 at 8:37am
My understanding of the scientific method as articulated by Francis Bacon in his Novum Organum Scientiarum (New Method of Science; the title selected undoubtely to indicate that this work was replacing Aristotle’s Organum) is that Bacon defined two components to the scientific method – observation and experimentation. Schwarz (2012) explained that Bacon referred to these components as “experientia” and “experimentum” to mean “the unforced observation which we might call experience” and “the contrived experience which we might call an experiment”. It is true that the scientific method is often equated with experimentation and observation is minimized or even ignored – as Bryan observes and illustrates.
Another aspect of Bacon’s work is that he never defined how to conduct this “contrived experience” called an experiment. This occurred 260 years later by Fisher working at the Rothamsted agricultural research station in the UK. Interestingly, most scientific discoveries before Fisher were largely in the physical sciences (physics, chemistry, astronomy), sciences that exhibit little variability. Variability obscures and hides effects, but is an inherent characteristic of agricultural/biological/ecological systems. This is probably why experimental design (DOE) developed out of agriculture and not the more basic sciences of physics, astronomy, or chemistry – they did not need it. What makes the physical sciences difficult is understanding the underlying mechanisms, whereas variable systems have the added difficulty that effects must be first teased out from the variability before mechanisms can be studied. I wonder to what extent economics like biology, ecology, and agriculture?
Schwarz A (2012) The becoming of the experimental mode. Scientiae Studia 10:65–83
Noah Carl
Oct 25 2017 at 9:41am
Excellent post
Nathan Smith
Oct 25 2017 at 2:24pm
Great pushback against the methodological naivete and impoverishment that comes from economists trying to mimic the perceived traits of the natural sciences. But I think the post is still a bit misleading, in that it gives too much legitimacy to parallels between economics and the natural sciences.
A key difference between economics and the natural sciences is that economists can and routinely do use INTROSPECTION as a source of insight. They can do that because economists STUDY humans and also ARE humans, so they can reason from the experience of being human, known introspectively, to general conclusions, or at least predictions, about what human nature is like.
Without introspection, I don’t think even so basic a principle as “people respond to incentives” could be established. What’s an incentive? Something one wants, given only as a reward for certain behavior. But how do we know what people want? How do we know that they want anything, or that the phenomenon of wanting even exists at all? No amount of mere external observation of human faces and actions could reveal it to a creature that really didn’t know about wanting. WE know, because we experience wanting introspectively, and then generalize to the experiences of others.
I agree that the textbooks are full of a kind of reasoning that is healthy and necessary, but increasingly treated as inadmissible in academic journals. But I don’t think the extra ingredient that the textbooks have is non-experimental observation; it’s reasoning from introspection.
Karl Lembke
Oct 25 2017 at 4:28pm
In your description of the scientific method, I’d say step 2 is too narrow. Instead of “Run an experiment” it should say “Gather observations and data”.
So for example, in the case of the circulatory system, there were at one time at least two competing theories: blood seeps from one side of the heart to the other through invisibly small pores, or blood passes through very small vessels from the arteries to the veins. The hard work of (a) showing there was no tendency for blood to seep through the walls of the heart, and (b) tracing pathways from artery to capillary to vein is what caused one theory to be discarded in favor of the other.
Likewise, digestion. One theory was that food simply putrified in the stomach. I’m not sure what good that was supposed to accomplish, but that was what at least some people thought. A lot of measurements that were made showed that substances did pass from the gut into the body, and in this case, experiments showed that you could follow the course of various substances from ingested food to chemical levels in the blood stream.
The point is, there are lots of sciences that are not experimental, but rely on our ability to observe and draw inferences. Astronomy is a science despite our inability to set up supernovas on an experimental basis.
………Karl (B.S. in Physics)
Michael Wulfsohn
Oct 25 2017 at 5:34pm
Quite right. My understanding: the key requirement is evidence, not experimentation. Experimentation can provide very strong evidence, but it’s often an impractical way to build economic knowledge. Economists instead use evidence that is less strong, but still useful. For example, they might observe how well a given theoretical model explains data, and use their observations to refine the theory.
Musca
Oct 25 2017 at 5:49pm
Interesting post. The reason that the two-step summary of the scientific method seems simplistic is that it is entirely deductive.
I was taught science the same way – “somehow” one formulates a hypothesis, and then tests it by comparing implications deduced from it against data.
But where does the hypothesis come from? The “painstaking observation” and introspection mentioned by Alex and Nathan above are referring to induction: assembling individual observations into a generalized statement. Step 0 is then assembling numerous observations into an induced hypothesis, whose specific implications are then tested deductively.
Induction seems to have a “bad rap” in some fields, perhaps because it involves more explicitly philosophical assumptions about human reason’s ability to perceive the world directly (David Harriman has much to say on this). Would someone here familiar with how the history of economics is taught comment on whether “induction” is specifically discouraged or derogated in the literature or pedagogy?
Comments are closed.