Annual Question:
Thirty-five years ago, I was an insecure assistant professor at the University of California Berkeley—and a curmudgeonly senior colleague from the hard-science side of psychology took me aside to warn me that I was wasting whatever scientific talent I might have. My field, broad-brushed as the soft side of psychology, was well intentioned but premature. Soft-siders wanted to help people but they hadn’t a clue how to do it.
Now I get to play curmudgeon. The recent wave of disclosures about the non-replicability of many soft-side research phenomena suggests that my skeptical elder knew more than I then realized. The big soft-side scientific news is that a disconcertingly large fraction of the news does not hold up to close scrutiny. The exact fraction is hard to gauge but my current guess is at least 25%, perhaps as high as 50%. But historians of science will not have a hard time portraying this epistemic “train wreck” as retrospectively inevitable. Social psychology and overlapping disciplines had evolved into fields that incentivized scholars to get over the talismanic p < .05 significance line to support claims to original discoveries and disincentivized the grunt work of assessing replicability and scoping out boundary conditions. And as the gang of six, Duarte et al, point out in their Behavioral and Brain Sciences article on “ideological diversity will improve social psychology,” the growing political homogeneity of the field selectively incentivized the production of certain types of knowledge: counterintuitive findings that would jar the attentive public into realizing how deeply unconsciously unfair the social order is. This has proven a dangerous combination.
In our rushed quest to establish our scientific capacity to surprise smart outsiders plus help those who had long gotten the short end of the status stick, soft-siders had forgotten the normative formula that Robert Merton formulated in 1942 for successful social science, the CUDOS norms for protecting us from absurdities like Stalinist genetics and Aryan physics. The road to scientific hell is paved with political intentions, sometimes maniacally evil ones and sometimes profoundly well intentioned ones. If you value science as a purely epistemic game, the effects are equally corrosive. When you replace the pursuit of truth with the protection of dogma, you get politically-religiously tainted knowledge. Mertonian science imposes monastic discipline: it bars even flirting with ideologues.
But “prematurity” is a temporal diagnosis. I timed my birth badly but those entering the field today should see the train wreck as a goldmine. My generation’s errors are their opportunities. Silicon-Valley-powered soft-science gives us the means of enforcing Mertonian norms of radical transparency in data collection, sharing and interpretation. We can now run forecasting tournaments around Open Science Collaborations in which clashing schools of thought ante up their predictions on the outcomes of well-designed, large-sample-size studies, earning or losing credibility as a function of rigorously documented track records rather than who can sneak what by which sympathetic editors. Once incentives and norms are reasonably aligned, soft-science should firm up fast. Too bad I cannot bring myself to believe in reincarnation.
