![Facts, Opinions, and the Pew Research Center’s Pseudoscience](https://staging.econlib.net/wp-content/uploads/2018/10/facts.jpg)
Hopefully you can’t separate facts from opinions…
Do you remember those grade school exercises where you had to divide a bunch of statements into facts and opinions? The trick to getting an ‘A’ was easy: if a statement could be looked up in a reference book or checked by simple observation—e.g., “Topeka is in Kansas,” “An isosceles triangle has two sides of equal length,” “My sneakers are white,”—you labeled it a fact (even if it was false!); otherwise it was an opinion.
At the time, I gave the exercises little thought, but they should have bothered me. I certainly considered “Hitler was evil,” “Catherine Bach is beautiful,” “Johnny Carson is funny,” and “The food in the school cafeteria is lousy” to all be facts, though none of them could be checked in reference books. On the other hand, “Elvis is alive and working at a Denny’s in Tucson” could be checked, but that didn’t seem like a fact to me.
It wasn’t until my first formal logic class that I thought about those exercises critically. In logic, all statements are claims about the world. Facts are accurate statements (though many facts will never be known because no one will ever prove their accuracy—or even conceive of them), while opinions are expressed statements that are believed to be true but may or may not be facts. “Topeka is in Kansas” is a fact and is an opinion held by many people, while “Topeka is in Virginia” is neither a fact nor—as far as I know—an opinion (because I don’t know anyone who believes that).
Normative and subjective statements can also be facts and opinions, though it can be argued that many of them are vague. For instance, “The food in the school cafeteria is lousy” may be better understood as “Most people dislike the food in the cafeteria.”
One of my logic teachers, Virginia Tech emeritus professor Harlan B. Miller, felt so strongly about this that he wrote in his logic text:
You may have been unfortunate enough, in grade school or middle school, to have been taught to contrast “fact” with “opinion” and to sort statements into these two categories as exclusive and exhaustive. If so, forget it. Try to clear that distinction from your memory banks. It’s nonsense or worse.
Sorting statements as fact or opinion is as sensible as sorting physical objects as green or plastic. Some things are one, some the other, but lots are both and lots are neither. There are lots of facts that are no one’s opinions (the unsaid truths), lots of opinions that certainly don’t correspond to facts (The Sun at your supermarket checkout is full of them), lots of unbelieved falsehoods that are neither facts nor opinions, and lots of things, including all known propositions, that are both fact and opinion.
It’s a fact, and my opinion, that 7 + 7 = 14. It’s my opinion, and a fact, that Hitler was morally depraved. Many (not all) facts are the opinions of somebody. Many (not all) opinions are (correspond to) facts.
So flush the fact/opinion contrast from your system. It should never have been there.
For good measure, he cites this article by Cal State–Sacramento philosophy professor Perry Weddle and admits to the paranoid suspicion that the fact/opinion exercises are part of “a plot to sell ethical emotivism by warping the minds of ten-year-olds.”
I’ve occasionally thought about those exercises over the years, wondering if grade schoolers still do them. I hoped that, as more and more people studied logic, they—especially K–12 educators—had relegated the fact/opinion dichotomy to the dustbin of pseudoscience along with alchemy and phrenology.
Then, this week, I came upon this headline and deck on The Atlantic’s website:
Older People Are Worse Than Young People at Telling Fact from Opinion
Given five facts, only 17 percent of people over 65 were able to identify them all as factual statements.
The article explains that the Pew Research Center’s Journalism & Media unit has been administering its own version of the fact/opinion exercises to adults, then issuing hand-wringing reports on the inability of many test-takers to “correctly” separate the statements. According to Pew,
Factual statements are unambiguous statements that one can prove to be accurate or inaccurate based on objective evidence. Opinion statements are based in people’s beliefs and values, whether political, religious, moral, cultural, or some other belief system, and therefore lack traditional standards of objectivity. And borderline statements have elements of both factual and opinion statements – they can be based in objective evidence, but claims are vague enough that they can neither be unambiguously proved nor disproved by factual evidence, in part because such evidence is often conflicting, incomplete, contested or involves making predictions.
That certainly sounds like the distinction from grade school.
The Atlantic article’s author, Alexis Madrigal, accepts the dichotomy uncritically and argues (torturously) that people’s inability to separate fact from opinion should be blamed on the repeal of the Federal Communications Commission’s Fairness Doctrine, the rise of social media, and the existence of Breitbart, Fox News and MSNBC. A quick Google search finds that Madrigal is not alone in accepting Pew’s fact/opinion dichotomy uncritically; so does Justin Doom at ABC News, Joe Concha at The Hill, Mairead McCardle at National Review Online, John Walsh at Business Insider, Angela Moon at Reuters, and even Rush Limbaugh.
It’s tempting to dismiss this as a mere semantic disagreement between philosophers and logicians on one hand and the Pew folks, media, and K–12 educators on the other. But if you look at Pew’s “test answers” for 10 statements it deems unambiguous facts or opinions, you find that Pew has trouble following its own definitions.
For instance, it classifies the following three statements as “opinions”:
- Immigrants who are in the U.S. illegally are a very big problem for the country today.
- Government is always wasteful and inefficient.
- Increasing the minimum wage to $15 an hour is essential for the health of the U.S. economy.
In public policy, analyses are regularly published arguing that these statements are true or false, and thus facts or falsehoods. These analyses make extensive use of “objective evidence” and “traditional standards of objectivity” such as empirical data and statistical analysis—the stuff Pew claims are the hallmarks of “facts.”
For instance, the notion that illegal immigrants are “a very big problem” rests on ideas about crime and public safety, political and social dynamics, labor economics, and government fiscal effects. Those topics can be evaluated with empirical data, historical examples, economic modeling, and basic mathematics—that is, “traditional standards of objectivity.” My Cato colleagues Alex Nowrasteh and David Bier muster mountains of “objective evidence” on the effects of immigration and provide a careful, reasoned framework for evaluating them, concluding that illegal immigrants are in fact not a very big problem (or any problem at all, on net) for the country today.
Ah-ha, the Pew folks would likely respond, but analysts at the immigrant-wary Center for Immigration Studies likewise muster evidence and assemble a rational framework, and they conclude that illegal immigrants are in fact a very big problem for the country today. It can’t be the case that both Cato and CIS are right—so, Pew reasons, there must be no objective fact of this matter, just opinion. But just because there is disagreement doesn’t mean there’s no fact, just as disagreements between physicists don’t mean there’s no facts in physics. Rather, these disagreements mean the facts are in dispute.
Similar arguments can be made about government wastefulness and inefficiency (two concepts that are well-defined in economics) and whether a minimum-wage increase is “essential” (or even beneficial) to the nation. In fact, all five statements that Pew identifies as opinions and not facts clearly are claims to fact. They are all subject to objective analysis (though, again, that analysis may not settle the issue—at least not today) and are either facts or falsehoods as well as opinions.
What about the statements Pew labels as “facts”? All five do accurately describe the world, but at least one of them is not a “fact” according to Pew’s definition of the term. Consider this statement:
- ISIS lost a significant portion of its territory in Iraq and Syria in 2017.
ISIS certainly lost most of its territory over the last few years. But does the amount lost in 2017 count as “significant”? That depends on what is meant by “significant.” In social science, statistical significance usually (but not always) means there is less than a 5 percent chance the correlation of two variables is the result of random chance. By extension, does that mean ISIS must lose at least 95 percent of its land over the course of 2017 in order for the loss to be “significant”? Or does “significance” in this context depend on “people’s beliefs and values … and therefore lack traditional standards of objectivity”? If we follow Pew’s definition, then this statement is an “opinion,” not a “fact.”
Pew’s dichotomy of fact and opinion is problematic—so problematic that Pew violates its own definitions of those two terms even when it tries to use them carefully. Rather than casting aspersions upon the people who ostensibly misidentify “opinions” as “facts” and vice-versa, Pew should appreciate that those people take value judgments seriously enough to believe they accurately reflect the world we live in—that is, that they are facts.
That leads to the deeper problem underlying the Pew exercise. If value judgments are purely matters of opinion and wholly removed from fact, then most if not all policy disagreements—and most if not all human disagreements—cannot be settled rationally. “Slavery should not be legal in most cases” is not a fact according to Pew, but an opinion that cannot be true or false. Likewise “The Holocaust was immoral,” “Sexual harassment is a big problem in the United States,” and “The immigrant caravan heading toward the United States is a big problem for the country.”
In fairness to Pew, I’m sure this exercise is motivated by a noble idea: to get people to think critically about their value-laden beliefs about the world, which in turn should encourage civility to others with different points of view. But by making fact and opinion mutually exclusive, Pew goes too far, removing the very reason—trying to understand the world—for critical thinking and civility. If opinions can’t be facts, then societal disagreements on these matters can only be settled through power-battles between opposing tribes, with the strongest tribe dictating to everyone else. The Pew Research Center’s nobly intentioned project becomes the perfect clarion for the Age of Trump.
This is not to say that policy disagreements don’t entail differing personal tastes, preferences, and risk tolerances. De Gustibus Non Est Disputandum, after all. That’s why I’m a libertarian who believes in robust individual liberties and limited government—institutions that help people with different preferences maximize their freedom to follow those preferences while living peaceably with others in a well-functioning society. The same is true when the facts are unknown or in dispute. But differing tastes don’t mean that we should deem matters of judgment, values, and beliefs—and all matters subject to disagreement—to be fact-free zones. Doing that would undermine the very importance of these disagreements.
READER COMMENTS
nobody.really
Oct 29 2018 at 3:11pm
Uh … yeah. Except I doubt that we’d find agreement among all philosophers and logicians, or among all Pew/media/K–12 folks. In short, people use words differently, and the same person will use the words differently in different contexts. In addition, even if we could transcend the differences in language usage, I suspect we’d find that people have different beliefs.
Alasdair MacIntyre argues that we cannot understand ANY proposition without the aid of a belief system to provide context. In this sense, we cannot distinguish between a “fact” and the adoption of a belief system (“opinion”) within which to analyze the fact.
But often we converse as if we share a frame of reference. And within a given frame of reference, we might find propositions we could agree on. Thus, a teacher might teach geometry or physics, or even say that you cannot subtract a large number from a smaller one, without explicitly saying that the teacher was speaking strictly within the context of Euclidian geometry, or Newtonian physics, or the domain of whole numbers. The teacher could begin by making an explicit exposition of frames of reference, but experience suggests that this style of communication proves less useful than simply assuming a common frame of reference and plunging ahead.
In sum, I expect that Firey’s grade school teacher to want to help kids distinguish between propositions that we might resolve by reference to data to propositions that we would never resolve solely by reference to data, assuming everyone in the conversation had frames of reference with sufficient similarities. And I expect that Firey’s logic professor intended to speak to a different audience with a different purpose, and thus he employed the words “fact” and “opinion” differently. Welcome to semantics.
Yes. But the notion ALSO depends on the meaning of “a very big deal.” And no amount of empirical data will resolve that question. Ask the person who is listening to Hurricane Maria ripping the roof off of his house whether he regards illegal immigrants as a very big problem, and I suspect you may find that other concerns weigh more heavily on him. Then answer to whether illegal immigration represents a “very big problem” irreducibly depends upon the frame of reference of the person answering the question.
Likewise, whether “Government is always wasteful and inefficient” irreducibly depends upon the respondent’s frame of reference for evaluating waste and efficiency. Whether “Increasing the minimum wage to $15 an hour is essential for the health of the U.S. economy” irreducibly depends upon what the respondent regards as essential, and as healthy. The fact that Firey can identify facts that he regards as relevant to analyzing these questions does not eliminate the subjective factor involved.
This does not render Firey’s analysis wrong; it merely indicates that Firey uses the words “fact” and “opinion” in a manner that differs from (my understanding of) how the grade school teacher or Pew use the terms.
I have to suspect that Firey misses the point of the grade school teacher’s exercise, and of Pew’s. I don’t think anyone has much doubt that plenty of people assert their opinions as facts. But Pew seeks to explore (and promote?) public discourse. And the grade school teacher and Pew (or perhaps I’m just projecting my own views) believe that we aid public discourse when we encourage people to distinguish between propositions that we might hope to establish factually and propositions upon which reasonable people might disagree. And yes, I believe this exercise teaches a kind of moral virtue—the virtue of subjecting my own factual claims to scrutiny and even dispute within the public arena, even when that hurts my feelings.
Again, I don’t regard language usage as right or wrong, but as more or less helpful. I don’t see the utility of adopting Fire’s definition of “fact” as “a value judgment believed firmly.” To the extent that the word “fact” has positive connotations, using a word in this manner tends to defeat the purpose of Pew’s exercise as I understand it.
Andre
Oct 29 2018 at 5:25pm
Agree with nobody. This is particularly helpful: “In sum, I expect that Firey’s grade school teacher to want to help kids distinguish between propositions that we might resolve by reference to data to propositions that we would never resolve solely by reference to data, assuming everyone in the conversation had frames of reference with sufficient similarities.”
Firey seems to neglect the fact that a lot of what he seems to claim as fact are clearly opinions, themselves a combination of component facts. Even if all those component facts were agreed-to-as-“facts” by arguing sides (e.g., number of immigrants, $ spent on immigrants, cost of enforcement, taxes generated, etc.) left vs. right), that doesn’t mean that the combined results of those facts (e.g., “immigration is a problem”) is itself a fact. It is not, because different sides will weigh the component facts (they agree on!) differently.
(Besides, some components are unlikely to be facts, anyway; the point is that even if all you start with is facts, they don’t necessarily yield a factual conclusion, but rather an opinion.)
Taking this paragraph,
“[1]“Slavery should not be legal in most cases”is not a fact according to Pew, but an opinion that cannot be true or false. Likewise [2]“The Holocaust was immoral,” [3]“Sexual harassment is a big problem in the United States,” and [4]“The immigrant caravan heading toward the United States is a big problem for the country.””
None of these utterances are facts. They are all clearly opinions. Most of us, I believe, would hope that for the first two, everyone agrees. That doesn’t make them facts, though. As for the third, the notion that it is a fact is laughable. Since when is there an agreed-upon definition for what is a “big problem”?
As for the last one, the very people who are like to say the caravan is not a big problem might change their minds if its continued progress helps Republicans do much better at the polls next week.
robc
Oct 30 2018 at 10:18am
#1 and #2 are only opinions if we assume their is no absolute standard of morality. If one of your premises is that there is an absolute standard of morality, then #1 and #2 may become facts.
To quote Jefferson, I believe in “the Laws of Nature and of Nature’s God”.
We may not exactly understand what those laws are, just as we don’t understand all of physics, but that doesn’t make things dependent on them opinions. Just facts we don’t understand well.
Mark Z
Oct 30 2018 at 1:56pm
This question of facts seems to be convoluted by the fact that we are discussing two separate characteristics of a proposition as if they were one. “Catherine Bach is beautiful” is a matter of taste, while “Ankara is the capital of Turkey” is a matter of fact; they are both, I would say, opinions, but one is on a purely subjective matter.
When people speak of ‘fact’ vs. ‘opinion’ in politics, I think they are really just referring to the degree of consensus. “Vladimir Putin is the President of Russia” is a fact, while “Russia is the greatest foreign policy threat the US currently faces” is an opinion. But the distinction is not an essential one: each of these statements is either true or false. What (in common parlance) makes the second one an “opinion” is that it isn’t ‘self-evident’ or widely agreed upon whether it’s true or false. That, imo, is all that distinguishes them. One might have the opinion that Putin is not the president of Russia, and that all information to the contrary is propaganda. We’d call this a conspiracy theory, but the only thing that distinguishes a conspiracy theory from a mere opinion is that the former contradicts a nearly universally held opinion.
Epistemological, then, facts and opinions are similar. Even moral claims like ‘X is wrong’ are arguably in the same category, in that they are either true or false (or can be decomposed into a finite set of propositions each of which is true or false). Even if one holds that all moral values are subjective, then one’s position is that ‘X is wrong’ is a false statement, inasmuch as it purports to say something beyond “I don’t like X,” and posits a rule applicable to others.
So, what makes the CIA world factbook saying “Ankara is the capital of Turkey” a fact and the Heritage Foundation saying “socialized medicine is a bad idea” an opinion is merely that virtually everyone would accept former proposition as true and/or the source as being implicitly credible on the question of Turkey’s capital, while only maybe half of the population would agree with the latter proposition and/or accept the Heritage Foundation’s authority on the question at hand.
nobody.really
Oct 30 2018 at 4:46pm
People have different opinions about the is/ought distinction, but I find it a useful rule of thumb: Statements declaring that something “ought” or “should” or “must” (except in the case of logical necessity/prediction), or that express value judgments, or that attribute purpose or intent to non-sentient things, strike me as the kinds of statements that I should not expect other people to embrace purely on the basis of empiricism.
In addition, when I want to appeal to empiricism, I strive to reduce the level of abstraction in my statements. Often I find it useful to use the discipline of E-Prime–especially avoiding reliance on the verb to be. When I re-draft statements to avoid “to be,” I must re-focus on what I mean to convey.
Statements such as “Ankara is the capital of Turkey” lend themselves to varied interpretation. Instead I might say, “According to the Turkish Register, on October 29, 1923, the members of the Grand National Assembly of Turkey voted to establish the central offices for governing the Republic of Turkey in Ankara. To date, I have no basis to believe that anyone has moved these offices.” In this manner, I express what I mean by “capital,” the basis for my conclusion, and the limits of my conclusion.
Now consider the statement “Slavery should not be legal in most cases.” I could translate it into E Prime by saying “The law should not permit slavery in most cases.” Then I might unpack the term “the law” to say something less abstract, such as “Police, prosecutors, judges, and juries, in enforcing the Constitution, legislation, and regulations adopted by government agents, should not permit slavery in most cases.”
But I have a harder time finding substitutes for the word “should.” I might translate the sentence to say, “would produce an undesirable result,” but that still seems opinion-y to me because it relies on the meaning of “undesirable.”
We might avoid offering a personal opinion by factually quoting someone ELSE’s opinion. We might say that “Frederick Douglass opposed slavery in most cases” or “In most cases, slavery conflicts with the Universal Declaration of Human Rights.” But if we intend the reader to adopt the proffered authority as a standard by which to judge slavery, then we return to the realm of opinion.
Thus, I think I have a standard for distinguishing between statements I could expect to promote solely on the basis of empiricism (“facts”) and statements that I would not expect to promote solely on the basis of empiricism (“opinions”).
andy
Oct 31 2018 at 6:42am
I struggled with the ‘slavery’ as well, but it seems to me that it is not that hard to reformulate it in a way it would conflict with agreed-upon ethics. I.e. slavery is incompatible with e.g. ‘equal rights idea’. That seems to be ‘factual’. There is a hidden assumption that most people wouldn’t oppose ‘equal rights’, but that seems to me relatively safe.
nobody.really
Oct 31 2018 at 10:02am
Opinion, fact, and logical inference
I identify a third category, beyond “fact” and “opinion”: logical inference/conclusion.
And yes, I could see a logical inference that “Slavery conflicts with the norm of equality under law.” But US law (and all law?) recognizes a variety of status distinctions that conflict with the norm of equality. The law grants rights to citizens that it withholds from non-citizens, grants tax status to married couples that it withholds from non-married couples, applies tax brackets to poor people that it withholds from rich people, makes driving privileges available to older people that it withholds from younger people, restricts the discretion of felons without imposing similar restrictions on non-felons, etc.
Thus, the statement that slavery conflicts with a norm of equality may arise from logical inference rather than opinion. But the statement “Slavery, unlike a variety of other legal status distinctions, should not be legal in most cases” still relies on a “should.” This still looks opinion-y to me.
“Fact” as consensus
If in 1800 a poll showed that most Americans thought slavery was good, then you would regard it as a fact that slavery was good in 1800? If a poll showed that most Afghanis regard Christianity as the worship of a false god, then you regard it as a fact, within Afghanistan, that Christianity is the worship of a false god? If most people believe in Newtonian physics rather than relativity, or creationism rather than evolution, then you would regard these popular theories as fact?
Again, I don’t regard word usage as true or false, but merely as more or less useful. I don’t find this concept of “fact” very useful. I use phrases such as “widely held” or “widely accepted” for this concept.
cregox
Nov 20 2018 at 3:27am
semantics aside, i find no comments on the one thing we could all probably agree the article got right: that by pew’s own definition, the statement about ISIS is actually an opinion. “to me that’s the bigger topic for discussion!” and that’s a fact about my opinion. 🙂
as for the logic of what fact is and if it should be defined as opposite from opinion… i propose a mixed definition:
facts should always be objectively falsifiable, yes, and they must always be true as well.
meaning facts won’t always be true nor falsifiable, by definition and due to reality’s nature. that’s just what we strive them to be.
if a star just “died”, it will be a fact it is alive until years pass and we are able to become aware of the fact it died. that’s just basic physics today.
and the fact about my opinion isn’t exactly falsifiable… but it’s still true. and it will be until i say otherwise. if i ever change my opinion there, that “to me ISIS is the bigger topic for discussion here”, it will no longer be true or a fact. and it’s hardly a fact to begin with, since the only way to falsify it is asking me and getting a different answer, but it can still be considered a fact since there’s a way to falsify it. even if i die today, there still will be ways.
now, as much as i prefer my own definition there, it does make playing pew’s game much harder. and, to me, that begs for an easy name to this game, which i find both interesting and useful as a thought exercise. the word is “claim”. claim or opinion. they both can be true or false, but only one must be falsifiable by definition.
still, “significant” turns the ISIS claim into an opinion.
does it not?
Comments are closed.