https://www.wsj.com/articles/studies-are-usually-bunk-study-shows-1502660991
An interesting detail went overlooked in the fury over fired Google engineer James Damore’s “diversity memo.” At the end of the document he calls for an end to mandatory “Unconscious Bias training.” Large corporations often force employees into re-education classes, this one a dull, hourlong, 41-slide seminar supported by study after study. Can these studies be trusted? Doubtful. Hands down, the two most dangerous words in the English language today are “studies show.”
The world is inundated with the manipulation of flighty studies to prove some larger point about mankind in the name of behavioral science. Pop psychologists have churned out mountains of books proving some intuitive point that turns out to be wrong. It’s “sciencey,” with a whiff of (false) authenticity.
Malcolm Gladwell is the master. In his 2008 book, “Outlier,” he argues that studies show no one is born better than anyone else. Instead success comes to those who put in 10,000 hours of practice. That does sound right, but maybe Steph Curry shoots hoops for 10,000 hours because he is better than everyone at basketball in the first place. Meanwhile I watch 10,000 hours of TV. Facing criticism, Mr. Gladwell somewhat recanted: “In cognitively demanding fields, there are no naturals.” News alert: Professional sports are cognitively demanding.
When reading people like Mr. Gladwell, you’re probably thinking many of the studies’ conclusions sound right but don’t really reflect your own experience. That’s probably because you’re not a hung-over grad student. Andrew Ferguson of the Weekly Standard studied behavioral economic studies and discovered many are done by grad students observing their peers doing trivial tasks. Then researchers draw hard conclusions from this. Rather than a study of human nature, behavioral science is, Mr. Ferguson observes, “the study of college kids in psych labs.”
Many of the studies quoted in newspaper articles and pop-psychology books are one-offs anyway. In August 2015, the Center for Open Science published a study in which 270 researchers spent four years trying to reproduce 100 leading psychology experiments. They successfully replicated only 39. Yes, I see the irony of a study debunking a study but add to this a Nature survey of 1,576 scientists published last year. “More than 70% of researchers have tried and failed to reproduce another scientist’s experiments,” the survey report concludes. “And more than half have failed to reproduce their own experiments.”.
Bunk medical studies are worrisome, but who really cares about pop behavioral science? It’s easy to write this off as trivial, except millions take these studies and their conclusions seriously. The 2008 book “Nudge,” from academics Richard Thaler and Cass Sunstein, called for “libertarian paternalism” to push people in the right direction. But who decides what’s the right direction? Turns out the answer is Mr. Sunstein. He was hired by the Obama administration in 2009 to run the Office of Information and Regulatory Affairs. Call it psychobabble authoritarianism.
In his best seller “Blink,” Mr. Gladwell finds studies suggesting we are all unconsciously biased sexists, racists, genderists, ableists, and a litany of other “ists”—victimhood’s origin story. Newer research has deflated this theory, but the serious conclusions, and boring training seminars they inevitably lead to, remain. In her first debate against Donald Trump, Hillary Clinton channeled her inner Malcolm Gladwell and declared: “Implicit bias is a problem for everyone, not just police.” Everyone? Speak for yourself. It’s as if she called an entire slice of society deplorable.
Psych labs are being replaced. In the past decade, companies have built vast platforms to probe, test and study humans every time they search, like or snap. Google runs what are called Split A/B tests, dividing users into groups and testing usability and other features to see what works best. In 2014, Facebook caused a bit of a stir after altering 689,000 users’ newsfeeds to see if the company could manipulate their emotions. It could. Good or bad, this is the future of studies.
The world is not binary, but conclusions drawn from studies always are. These studies show whatever someone wants them to. So stay skeptical and remember: Correlation doesn’t equal causation. If only I could find a study that shows this.
