Telling stories about the real world

Science is very good, on the whole, at what it does: establishing predictable facts about the natural world. It's probably fair to say that modern science, in so far as there is such a thing, is better at this than pretty much any other system that humans have ever devised. As scientists, we are often tempted to take other areas of human knowledge gathering to task for not behaving enough like science in their attitude to establishing facts about the world. We berate and bemoan the lack or misuse of evidence. We pour (sometimes deserved) scorn on the apparent flimsiness of other enterprises. In two very interesting posts, drugmonkey discusses why journalism and law can be so unappealing to the scientifically trained. I don't disagree with everything he says, and I think that certainly parts of both could stand to be held to more scientific standards of evidence.
And yet, I am always a little cautious about yelling "be more scientific" at people. Scientists as a whole have a bad a case of epistemological "PC gaming master race" syndrome (if I ever make a nerdier analogy, shoot me).  We are convinced of the innate superiority of our means of knowing what we know. This view is hubris. On the one hand, as has been shown time and again by historians, philosophers and sociologists of science, it is remarkably difficult to pin down, at any time in history, a single definition of what constitutes the scientific method. What's more, efforts to do so have often lead to awkward situations, such as Karl Popper's continuous flip flopping over whether evolution by natural selection could be considered scientific. (One of the many reasons I dislike Karl Popper). On the other, there are entire areas of enquiry in which applying the many commonly accepted facets of the "scientific method" (hypothesis testing by experiment, replication) is difficult, impossible, or wrong headed. And no, I'm not talking about your position on omnipotent beings awkwardly obsessed with judging humans. I'm talking about the two things that are (not coincidentally) crucial in both journalism and courts of law: the reconstruction of historical events and  the determination of people's internal states.
Wait, I here you cry, are you claiming history is not a science? Well, no. What I am claiming (and the philosopher Anton Schopenhauer got there first) is that one cannot apply an experimental framework to individual point event in the past, and furthermore that the range of applicability of hypothetico-deductive methods is much more limited. And when attempting to ascertain the truth behind a single event, in the past, never to be repeated (such as a current event, or a crime), what are we left with? Testimony, correlation, hearsay. Also known as a case. Also known as a story. Neither Baconian empiricism, nor hypothetico-deductivism, nor Popperian faslificationism, will help us here. At best, they will allow us to evaluate some (though by no means all) the evidence. In essence, the repeatable experiment is a tool for erasing the singular nature of point events (and it is much more difficult to do then often presented).
This exact problem is one that plagues my original discipline of paleontology. Paleontology is, in many ways, history writ large. Except it is not writ, more sort of left lying around. Many of the events we would like to understand are unobservable, their consequence inferred from disparate sources of evidence observed in the here and now. And the paleontological literature is rife with people grappling with the problem of just how scientific this endeavor is. And they run the gamut, from people twisting Popper's theory into an unrecognizable pretzel to accommodate paleontological historicity, to people consigning vast tracts of what is normally considered part of paleo to the dustbin of unscientific speculation. Neither approach is satisfying, and both end up running into absurdity and intellectual sterility. Paleontology is largely historical, as such the best we can do is marshal evidence for competing narratives, and go with the most plausible. Some of that evidence will fit with more prescriptive definitions of the scientific method, most (and I count most cases of fossil discovery in this category) will not.
As for the reconstruction of the internal states of a person at a given time and place, science isn't even close to figuring that one out. Yet we, as humans, are always in the business of trying to figure out what's going on in another person's head. And, in the West at least, it is literature that has grappled most directly with this problem, through what one could describe as thought experiments, but that we usually call novels. As the philosopher Mary Midgley once wrote:

"That is why literature is such an important part of our lives - why the notion that it is less important than science is so mistaken, Shakespeare and Tolstoy help us to understand the self-destructive psychology of despotism. Flaubert and Racine illuminate the self-destructive side of love. What we need to grasp in such cases is not the simple fact that people are acting against their interests. We know that; it stands out a mile. We need to understand, beyond this, what kind of gratification they are getting in acting this way."

Ironically, two of the authors who most thoroughly embraces this task of literature, Emile Zola and Marcel Proust, both believed that their novels were eminently empirical and yes, scientific.
Abandoning the notion that a one-size-fits-all epistemological framework derived from the physical sciences can be applied across all areas of possible human knowledge is not the same as saying "anything goes". Rather, it is a requirement that we be more rigorous, more critical, more demanding of the evidence and narratives presented to us given the constraints of what it is we are trying to understand and know. Indeed, the uncritical acceptance of a superficially science-y framework can be as dangerous for critical thought as any narrative-based understanding of the world (this is what Richard Feynman described as cargo cult science). We should have more that one tool in our epistemological kit. The world of things that are knowable is complex enough to need them.


PS: While researching this post, I found this delightful Schopenhauer quote we can probably all agree on:
"Newspapers are the second hand of history. This hand, however, is usually not only of inferior metal to the other hands, it also seldom works properly."