A few years ago, we ended up trading some classroom space in the Physics part of the building to Psychology, which was renovated into lab space for two of their new(ish) hires. This turned out to be a huge boon not only for the department (the lab space we got in the swap is really very nice), but for our majors. Most of the psychology experiments on campus use student volunteers, and pay a small amount to boost participation. Since the new psych labs were right next to the physics student lounge, our majors were taking part in four or five studies each, and racking up the study participation fees.
We had a couple of really big classes at the time, so I used to tease the psychologists that their results were going to be so completely dominated by nerdy white guys as to be utterly useless. While I doubt the specific studies in question had much effect on psychology as a whole, there’s a nice blog post at Ionian Enchantment arguing that the situation in our building is just an extreme example of a very general problem for psychology in general:
The authors of the paper, Canadian psychologists Joseph Henrich, Steven Heine and Ara Norenzayan, argue that most experimental subjects in the behavioral sciences are WEIRD – Western, Educated, Industrialized, Rich, and Democratic – and thus weird – not representative of most human beings. And this, if true, is a very serious problem indeed. Behavioral scientists (anthropologists, psychologists, behavioral economists and so on) are often interested in explaining the brains, minds and behavior of Homo sapiens as a species. (Some scientists, of course, are only interested in understanding specific cultures or what makes us different, but one important goal of the behavioral sciences has long been to explain universal human behavior). As evolutionary psychologists John Tooby and Leda Cosmides have put it, they “seek to characterize the universal, species-typical architecture of [the information-processing mechanisms that generate behavior]”.
But… Henrich and his colleagues review a large body of literature that seems to show that, across several domains, Western undergraduates – the workhorses of the behavioral sciences – are extreme outliers. In other words, if they are correct, most of the data behavioral scientists have used to test hypothesis and to drive theorizing derives from subjects who are possibly the least suited for generalizing about the human race.
Not being a psychologist, I don’t have much to add to this, but it struck me as interesting. The original research articles are linked from the post, as well, if you’d like to track down the details.
Interesting.
This makes it even easier to shrug off the dubious, just-so science of too much evolutionary psychology. Fine with me.
Rent psychology lab space in storefronts next to bail bondsmen and payday loan stores to balance things out.
I’ve always thought that by now psychology departments should have been able to compile the ultimate guide to the undergraduate brain.
At the very least I wish there was a better feedback loop with instructional methods. I find it hard to believe we can’t improve college lectures given how much we (should) have learned about undergrad brains.
As someone who did research in a psychology department (though I’m not trained in psychology), this is a problem that is often overlooked. One thing that forces a bit of diversity is the NIH’s requirement for distribution of subjects in a study funded by them to reflect that of the community (not just the school). Of course, that’s just for race and gender, age, socioeconomic status, etc. are still problems.
They should try looking for another test subject other than undergraduate
Thanks for the plug!
@jentennae: actually, as Henrich et. al. note, evolutionary psychologists (or, more generally, Darwinian-orientated behavioral scientists) are far less guilty of relying on WEIRD subjects. Evolutionary psychologists have long been at the forefront of large cross-cultural research. Another BBS article – Buss et. al., 1989 [large pdf] – is one example.
I’ve known a good few psychology PhD students who themselves joke that their work is only applicable to undergrads – although talk to them a bit more and they do consider it a fairly serious problem. However there are further problems too when trying to widen participation in the studies: if you ask for participants, you end up with some kind of self selection – people who volunteer to an advert for psychological study usually aren’t that representative of the overall human population either! I suspect a large percentage of these people would be the so-called “outliers” too.
Psychology is still to a large extent a pseudoscience: absurdly small and biased test samples, relying on self reporting, frivolous interpretations which arbitrarily pick one of many possible explanations, poor to nonexistent controls, little scientific rigor, little independent verification…
Yes, some solid studies have been done but most of it is worthless junk.
Paul, a lot of psychology is mixing with neuroscience and using objective techniques and basing their theories on actual physiology. Though, I kind of agree with you about a lot of pscyh studies out there, I wouldn’t dismiss psychology as being (to a large extent) a pseudoscience (though pretty much all of social psychology is… we actually joked about the social psychologists across the hall from us about that).
Also, at my college the study population was disproportionately psychology majors, because some of the psych courses had participating in one or more of those experiments as a course requirement.
They should try looking for another test subject other than undergraduate
Who do you recomend? I’m not a psycologist or academic researcher of any type, but undergraduates have some obvious advantages that are not easy to overcome:
1) They are conveniently located. A lot psycology research requires equipment which is non-trivial to move.
2) The are often willing to work really cheap. I was involved in a perception/reaction study in which I stared intently at a computer monitor for about 20 minutes. I was paid with a candy bar.
3) They are relatively homogeneous. This is the problem we are trying to address, of course, but automatically controlling for age and educational level has some advantages.
4) They are often willing to do some fairly weird things. There was a study I heard about which involved running on a treadmill while being towed by a golf cart around campus. I expect an older volunteer would either have been unwilling or required a lot more than the (iirc) $20 they recieved.
Wow, some people have an axe to grind huh? I’m just glad we use scientific processes to examine data sets regardless of how WEIRD the group the is. Also, I’d be interested in the demographics of typical universities, especially top reseach universities.
The degree to which this is a problem entirely depends on what conclusions the psychologist is trying to make. Some results will vary across age/SES/etc, some don’t (sometimes that variance will tell you something about the general mechanism).
Folks who are unfamiliar with what is actually currently being done in Psychology departments might be surprised what types of results are generalizable. Particularly aspects of behavior that have clear links to the underlying biology. Many folks in your local Psychology department likely have training (if not degrees) in areas such as neuroscience.
I am a neuroscience researcher and I have Three things to add.
First, there’s a bunch of labs that specifically study groups other than undergraduates. Some look at the effect of socio-economic status, culture, intelligence, age, or probably about any other factor you can think of. I can’t think of any huge seed change that has trickled back to general psychology, but the data is out there.
As Jules said, there’s a lot of processes that WEIRD people aren’t outliers for, or that WEIRD people seem at least continuous with the general population. If you testing say, control of attention, I wouldn’t be surprised if someone at a university tested better than someone who dropped out of highschool, but they are probably using a very similar underlying mechanism, so if you are just trying to work out that mechanism, your probably fine.
Even in the circumstances where WEIRD people use fundamentally different mechanisms than, say your average person in Mumbai, the data you collect isn’t wrong data, just biased data. While it’s important to try and remove as much bias as possible, it’s also important not to make the perfect the enemy of the good. We can still learn a lot from an imperfect data set, which can be used as a reference point when evaluating more-perfect data sets.
Hmmm…I think the first error here is the idea of “generalizing the human race.” I also am a physicist and don’t have expertise in the area of psychology, but I would think that a simple complexity analysis of the human brain would tell you no to generalizing. Besides, there are some pretty diverse psychological theories out there, so I can’t imagine what a mess it would be if they were actually representing all of society!
My goodness, there was some hating on psychology here.
Psychology is still to a large extent a pseudoscience: absurdly small and biased test samples, relying on self reporting, frivolous interpretations which arbitrarily pick one of many possible explanations, poor to nonexistent controls, little scientific rigor, little independent verification…
If the “still” were taken out of there, I would agree. Psychology has a long tradition of largely being a cult of personalities. While to some degree that is still the case, the personalities are the folks who really turned psychology into a hard science. And their results are constantly being challenged or pushed to the limits, to find the breaking points.
Every single psychology class I have taken now, has focused as much on science and the methods of science, as it has on the specific science we’re talking about. And I have barely begun scratching the surface. Nor is my experience in the least bit unique, this is how psychology is being taught, because that is what psychologists do – science.
And there are a lot of us who are particularly keen on generalizing outside of Western undergrad populations. That is largely why I am focusing on evo-psych, which will (for me) mostly involve cross-cultural work that will focus on Eastern European and Asian populations (I hope and assuming my brain doesn’t explode while learning Russian).
But even where we are seriously limited by biases, there is a lot of solid science being done. While a lot of studies use relatively small sample sizes, they are replicated several times over. It is kind of hard to conduct studies with massive sample sizes, when you have to invest several man hours into each subject. So you use the sample that is feasible and if the results are promising, you replicate it – along with several other investigators.
It is really, really irritating to listen to (or in this case read) people who have little to no clue what is actually happening, expound on the junkiness of the science. The human brain and all the influences on human behavior are exceedingly complicated. Kind of like physics is complicated, or genetics is complicated or cosmology is complicated. Like the science being done in those fields, we break things down as much as is feasible and investigate each bit as best we can. As we reach verifiable, quantifiable conclusions, they become part of the larger picture that is “what we know,” or more accurately is, “what we are pretty sure we know.”
For psychology, it is complicated by the same major problem that medicine has to deal with – we need to treat real human beings with the best tools we have, regardless of what we actually know. We don’t have the luxury of perfecting anything, before we try to help people. There are people who need help, regardless of how well prepared we are.
So we muddle along, because someone who’s anorexic, isn’t likely to live long enough for us to perfect their treatment. Someone who has absolutely no control over their drinking, their snorting, their smoking or their shooting, doesn’t have the luxury of waiting until we’re sure we have it down. Especially as addiction has a hell of a lot of causes. Like cancer, we’re talking about an array of illnesses – not some singular entity. And that is exactly the case for a great many mental illnesses, as we have been learning through cognitive studies and neurological studies.
But we are most certainly not engaged in pseudoscience. If you honestly want to use that word, then you need to figure out what the hell you’re actually talking about beforehand.