Social scientists are wary about the use of stories as evidence for a claim. Anecdotes that are not backed up with systematic and rigorous comparative data are not trusted. Anecdotes are the weakest form of evidence, but they are often the most persuasive. Even scientists can be moved by a telling anecdote that contradicts a mass of statistical evidence. Anecdotes must always be used with care, precisely because we are psychologically susceptible to them.
Comments by various people on the email@example.com listserv
I have been in meetings comprised of administrators (many with scientific
backgrounds) who, after skimming over the valid, reliable, and carefully
collected data, move on to the anecdotal comments and make important policy
decisions based on the latter.
So they seem to be important, irrespective of value.
Story-telling, the anecdotes, may reflect the belief that individual cases are representative of larger populations. There's a nice article in the New Yorker by Atul Gawande, "The Cancer-Cluster Myth" (Feb 8, 1999, pp. 34-37), in which the author briefly discusses work by David Kahneman and Amos Tversky, who identified "a systematic error in human judgment, which they called the Belief in the Law of Small Numbers...Human beings evidently have a deep-seated tendency to see meaning in the ordinary variations that are bound to appear in small samples" (pp. 36, 37).
In a similar vein is a chapter called "The Achilles' Heel of Human Cognition: Probabilistic Reasoning" in Stanovich, Keith E. (1996) How to Think Straight About Psychology (4th ed) New York: HarperCollins.
I remember hearing a story (many years ago) from one of the "star" public
opinion pollsters (George Gallup, senior; Louis Harris, whoever it was) that
he was on an ocean liner from NY to Britain in the 1950s and over the course
of each day, a crowd formed around the deck chair of one person. When the
pollster asked one of the stewards who the person was, he said it was Ronald
Reagan who was telling the most amazing stories to the assembled crowd. The
pollster went down to hear the stories and found that Reagan went on for the
5 day crossing, telling stories each day, and never repeating himself. In
1980, running for the presidency, Reagan displayed this "talent" again when
he told stories to refute statistical findings. For example, the Dept of
Agriculture had found that something like 90-95% of Food Stamps were being
used for EXACTLY the purpose for which the program had been designed.
Reagan, however, told a story to "refute" these statistics. He had, he
said, been standing in a line at a grocery store in California when some
"young buck" (his words) came up and paid for a T-bone steak with food
stamps, thus demonstrating in Reagan's mind that fraud and abuse was rampant
in the Food Stamp program.
Bill Clinton, by the way, took a page from Reagan's playbook but uses it in a much less devious fashion. In each of this State of the Union addresses, Clinton goes up and down the ladder of abstraction--citing statistics on the successes of AmeriCorps, for example, and then introducing key AmeriCorp volunteers who just happen to be in the audience. I hope that after Clinton goes out of office and the spate of books about his presidency hit the bookshelves, one of these might go into Clinton's speechmaking strategies to see if this style of argumentation was something he developed himself or if it was something taught to him and so could be a good example of the application of social science research to the real world.
Notice the use of anecdote to refute the use of anecdotes.
For a contrarian view of anecdotal data see the next comment.
Dr. Glenn Terrell. "THE SOCIAL SERVICE INITIATIVE: with Special Attention to Anecdotal Data." The Letter of the Pacific Institute Vol. 2 Iss. 2, Spring 1999.
Some may argue that anecdotal data have no place in educational and social science research. This editor believes this to be a misconception of what constitutes useful, scientific data. The necessary criteria for inclusion of data in research in any scientific discipline are a) quantification, i.e., the data must be expressed in terms of a number system, and b) the data must be derived from the measurement of concepts that are clearly defined so that any researcher anywhere in the world can replicate the study in order to support the study or dispute it. Anecdotal data, if expressed in quantitative terms and derived from a clear description of the operations the researcher uses in generating the anecdotes, meet these two criteria. Furthermore, some of the most convincing data we have in support of the impact of our education are found in testimonials.
Here is a link to a story from the New York Times about professors' reactions to student email messages. This story, like much journalism, is based on anecdotal evidence. It would be interesting to have some data to support the story. How many email messages from students do professors get? How much variation is there in that number? Does the number vary by age, gender, subject taught, or other factors? How many professors like communicating with students by email? Anecdotal data are interesting, but they must be viewed with the same critial judgment applied to all data. To: Professor@University.edu Subject Why It's All About Me