Times are tough for young psychologists. That’s what I kept thinking during recent meetings with candidates for a psychology position at Stevens.
Psychology is undergoing a credibility crisis. Last summer, a group called Open Science Collaboration reported that it had replicated fewer than half of 100 studies published in major psychology journals.
The New York Times declared that the report “confirmed the worst fears of scientists who have long worried that [psychology] needed a strong correction. The vetted studies were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory.”
The crisis keeps generating headlines. On Friday, a group of four prominent psychologists led by Daniel Gilbert of Harvard claimed in Science that the Open Collaboration study was statistically flawed and did not prove its claim that “the reproducibility of psychological science is low.” “Indeed,” Gilbert and his co-authors state, “the data are consistent with the opposite conclusion, namely, that the reproducibility of psychological science is quite high.”
Members of the Open Science Collaboration countered that the “very optimistic assessment” of Gilbert’s group “is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data.”
The exchange suggests that psychologists cannot even agree on basic methods for arriving at “truth,” whatever that is. It gets worse. Over at Slate, Daniel Engber reports that the influential theory of “ego depletion”—which holds that willpower is a finite resource that diminishes with use–might have been “debunked.”
Roy Baumeister and three other psychologists presented experimental evidence for the theory in a 1998 paper that has been cited more than 3,000 times. The theory has supposedly been corroborated by hundreds of other studies, and it underpins the 2011 bestseller Willpower, by Baumeister and journalist John Tierney.
But a study of ego depletion involving “more than 2,000 subjects tested at two-dozen different labs on several continents,” Engber says, found “exactly nothing. A zero-effect for ego depletion: No sign that the human will works as it’s been described, or that these hundreds of studies amount to very much at all.”
Spelling out the disturbing implications, Engber notes that “an entire field of study—and significant portions of certain scientists’ careers—could be resting on a false premise. If something this well-established could fall apart, then what’s next?”
Good question, over which all young psychologists are no doubt agonizing. To cheer themselves, they might consider the following four points:
First, there’s nothing new about psychology’s credibility crisis. More than a century ago, William James worried that the field he helped create might never transcend its “confused and imperfect state.”
Second, all scientific fields struggle with replication issues. Behavioral genetics and psychiatry are arguably much less credible than psychology, and string and multiverse theorists don’t even have empirical results to replicate!
Third, psychologists are still doing important, empirically sound work. Two who recently spoke at my school are Sheldon Solomon, co-creator of terror-management theory, which predicts how fear of death affects us; and Philip Tetlock, leader of a study on “superforecasters,” ordinary people who do a better job than many so-called experts at predicting social phenomena.
Fourth, psychology is arguably healthier than many other fields precisely because psychologists are energetically exposing its weaknesses and seeking ways to overcome them.
I can’t wait to discuss all these issues with our new psychology professor.
John Horgan directs the Center for Science Writings. This column is adapted from one originally published on his ScientificAmerican.com blog, “Cross-check.”
