"I think you'll find it's a bit more complicated than that." - Ben Goldacre
I've just finished reading Bad Science by Ben Goldacre, an excellent analysis of science scares, quacks and how things go wrong when the media report on science, with a particular emphasis on medical matters (he's a doctor). The book was nominated for the 2009 Samuel Johnson Prize for Non-fiction. As the quote above indicates, he also explores the problems that can arise even in good science - a very healthy approach, I think.
The book has chapters on homeopathy, the placebo effect, 'experts' like Gillian McKeith and Patrick Holford (Check out HolfordWatch), amongst others. Superb stuff, and scary too.
I particularly liked the chapter on Why Clever People Believe Stupid Things. You'll have to buy the book to read the full details, but I'll list the five reasons Goldacre cites for not trusting one's intuition, however tempting it may be.
1. We see patterns where there is only random noise.
2. We see causal relationships where there are none.
3. We overvalue confirmatory information for any given hypothesis.
4. We seek out confirmatory information for any given hypothesis.
5. Our assessment of the quality of new evidence is biased by our previous beliefs.
Add to that availability and social influences, and you have a recipe for the irrational. As Goldacre says:
It's not safe to let our intuitions and prejudices run unchecked and unexamined; it's in our interest to challenge these flaws in intuitive reasoning wherever we can, and the methods of science and statistics grew up specifically in opposition to these flaws.
One often hears the charge that rationalists harbour prejudices just as theists and other 'magical' thinkers do. Well, that's true, but that's why there is a scientific method. When another method comes along that recognises those prejudices as the scientific method does, then we may have something to challenge it. There *is* no other method currently; therefore, what is discovered by science can claim greater epistemic value than anything discovered by another. Where there is a contradiction, we must accept the scientific outcome, if we are acting reasonably.
To expand on the graphic at the top of the story, consider this version, courtesy of the University of California at Berkeley.