Illuminating studies on illumination studies

January 5, 2010 11:19 AM

A series of studies conducted at GE's Hawthorne Works in Cicero, Illinois in the 1920s remain some of the most important in social science, with a lasting impact on the working lives of almost everybody. Before the Hawthorne Experiments, the approach to work was to treat humans like machines, optimizing their movements and time. But the researchers in Cicero discovered that any change in the workplace, even dimming the lights, increased productivity, because the plant workers reacted to being studied. The "Hawthorne Effect" launched a management revolution, suggesting that worker's feelings and attitudes might actually be important. Except, according a new paper by the author of Freakonomics, the results of the Hawthorne studies "proved to be entirely fictional."
posted by blahblahblah (26 comments total)

Oops - should be "the findings of the Hawthorne studies" that are fictional. Evidence of the Hawthorne Effect have been found many times since, but the original studies don't seem to show what was original claimed.
posted by blahblahblah at 11:22 AM on January 5, 2010

worker's feelings and attitudes might actually be important

The only thing important is that the workers don't sue them.
posted by tommasz at 11:25 AM on January 5, 2010 [2 favorites]

Nonsnes. I jstu trnued off teh lights and git this cinnent tuped teice as quicjly.
posted by rusty at 11:33 AM on January 5, 2010 [1 favorite]

The paper does seem to argue, however, that the Hawthorne effect is at work in the original data - just not in the forms that were claimed in the original paper. "There are, however, hints of more subtle manifestations of a Hawthorne effect in the original data."
posted by dd42 at 12:03 PM on January 5, 2010

Wow, the "Freakonomics" guys came out with a darring paper that tries to overturn conventional wisdom in favor of powerful establishment interest? I'm SHOCKED at this revelation!

How are people still taking these idiots seriously?
posted by delmoi at 12:19 PM on January 5, 2010 [9 favorites]

i agree with delmoi,

my answer to your question is this: they present themselves as so smart, people buy into it without thinking. Certainly not critically thinking about sometimes weak data and incomplete analysis. Freakonomics had a few hits and was entertaining, but it sounds far bigger than it is.
posted by oshburghor at 12:35 PM on January 5, 2010

Before there is too much more Levitt-bashing, I should say that his paper on the Hawthorne Effect seems pretty sound. He and his coauthor actually rediscovered the original Hawthorne data, previously thought lost, and subjected it to its first real analysis in 80+ years. Its a reasonably cool thing, and a good paper. I blame Levitt's NYT journalist co-author for the edgier stuff, especially in the sequel, Levitt's academic research is really good, I don't know his politics.
posted by blahblahblah at 12:44 PM on January 5, 2010 [1 favorite]

Yeah, I turned off at the word "Freakonomics." I've gotten really sick of contrarianism as a sort of value in itself.
posted by LMGM at 12:47 PM on January 5, 2010 [1 favorite]

Did you see the comment from "Charles D.Wreg - Cornell University"?

I regret to say that Steve Levitt and John List have not made an original discovery by locating the supposed “Lost” reports of the illumination tests at Hawthorne. they merely found the few microfilm copies that Dr. Richard Franke had the foresight to have made in l977. I discovered them in l957, still held by the original investigator Charles E.Snow who had copies of the reports he had turned over to Hawthorne during the original tests. I described this in my book: Facts and Fallacies of Hawthorne (Garland Publishing, l986). the failure of Garland to promote this book has resulted in people still thinking the records were lost. The original copies at Hawthorne were destroyed on December 19, 1928, All of these records, along with correspondence with Mr. Snow ( before his early death on November 11, l962,) are at the Kheel Center for Labor-Management Documentation and Archives, Martin Catherwood Library , Cornell University as Collection 5167.

I don't know how trustworthy the comment is - why are several of the 1's seemingly-transliterated as l's?
posted by muddgirl at 12:51 PM on January 5, 2010

They're just trying to explain away why they didn't get the red lab jacket.
posted by drezdn at 12:59 PM on January 5, 2010 [3 favorites]

Does this mean I can go back to whipping my staff?
posted by Mister_A at 1:11 PM on January 5, 2010

"why are several of the 1's seemingly-transliterated as l's"

If you learned to type on very old-school typewriters, they frequently did not have a "1" key and the operator was expected to use the lower-case L. This is consistent with the author being an elderly professor.
posted by i_am_joe's_spleen at 1:20 PM on January 5, 2010 [4 favorites]

This was the plot for a recent episode of Better Off Ted, with the side-effect of creating an atmosphere of paranoiac bean-plating over what the small change meant, by the rank-and-file employees.

or what drezdn said.
posted by nomisxid at 1:27 PM on January 5, 2010

If you learned to type on very old-school typewriters, they frequently did not have a "1" key

Yep, my dad's typewriter from the 50's has no "1" key, which was very confusing the first time I tried to use it.
posted by electroboy at 1:45 PM on January 5, 2010

First off, as a former Telephone Company employee, I must point out that it wasn't the GE plant in Cisero, IL, it was the Western Electric plant, where they made telephone equipment for AT&T. They would have LOVED to know that turning down the lights would increase productivity.

I find the whole industrial time and motion studies very interesting. At first, the desire was to standardize things so that identical motions could create more productivity. The Hawthorne Studies were looking at one thing, lighting. Sure they fiddle with the lights, but what other things might have accounted for the changes in productivity? People slow down in warmer weather (when the natural light flooded the factory and the experiment was halted.)

I don't think that the idea is being debunked as much as it is being questioned. It LOOKS like lighting has something to do with productivity, but we didn't study anything else, so it might have been ANYTHING that caused those changes in productivity.

I have to say that working for the phone company is a singular experience. Militaristic, arbitrary and strange in the extreme. It's a place where one must ask for permission to use the restroom. Under such conditions even the smallest of changes could account for a change in some metric.

And for the record, I like lots and lots of light.
posted by Ruthless Bunny at 1:55 PM on January 5, 2010

blahblahblah: "7Before there is too much more Levitt-bashing, I should say that his paper on the Hawthorne Effect seems pretty sound. He and his coauthor actually rediscovered the original Hawthorne data, previously thought lost, and subjected it to its first real analysis in 80+ years. Its a reasonably cool thing, and a good paper. I blame Levitt's NYT journalist co-author for the edgier stuff, especially in the sequel, Levitt's academic research is really good, I don't know his politics."

Reposted for excellence. I think it's really easy to interpret this as 'the Hawthorne Effect isn't real', when its that the Hawthorne Effects do actually occur, did actually occur in the original study (as discovered by the Levitt and List analysis), but were not documented as such by the original authors (they either fabricated their effect - when they didn't need to per se - or they misanalyzed/misrepresented their data). Either way, they happily arrived at a viable conclusion, the Hawthorne Effect, which also fundamentally changed social science. Yay.

Incidentally, in linguistics, this has been called the Observer's Paradox, coined by William Labov, and it becomes a perpetual problem in doing just about any kind of linguistic research, from sociolinguistics to phonetics to ethnography and such.

Now I'm kind of curious if there are other terms for this, in other fields?
posted by iamkimiam at 2:41 PM on January 5, 2010

(these effects don't always have to be a problem in research, if you account for them, if it's the very thing you're studying, or some other reason.)
posted by iamkimiam at 2:43 PM on January 5, 2010

the results of the Hawthorne studies "proved to be entirely fictional."

Not exactly. Levitt & List's abstract says, "existing descriptions of supposedly remarkable data patterns prove to be entirely fictional. There are, however, hints of more subtle manifestations of a Hawthorne effect in the original data."

I'm not surprised actually. I read Rosenthal & Jacobson's Pygmalion in the Classroom a few years ago, the study the Pygmalion effect is named after. It was a bit of an eye-opener. The effect they found was statistically significant, but really tiny, partly because, as they realized after they'd interviewed the teachers involved, they screwed up the experiment. (The bogus results they gave teachers from a test they gave kids was supposed to be the independent variable. But as they realized later, most of the teachers hadn't actually read it.) It held up on replication, but with similarly modest results.

So, yes, it's real. And the idea that other people's expectations can affect your performance, even if you're not actually aware they have those expectations, is something psychology students, who may grow up to design experiments on human beings, need to be aware of. (This is one of the reasons we have double blinds, kids.)

But the Pygmalion study has turned into a kind of academic legend, repeated from one textbook to another with few authors checking the original, and getting better with each repetition. The original study wasn't very impressive at all.

The Hawthorn effect seems to have a similar story.
posted by nangar at 3:28 PM on January 5, 2010

Next you will be finding some psychological research revisionist who will look at Rosenhanz's data (Being Sane in Insane Places) and discover all those subjects really were batshit insane after all.
posted by bukvich at 3:43 PM on January 5, 2010

Now I'm kind of curious if there are other terms for this, in other fields?

In studying human motivation, there is a related external factor called Goal Publicness (Hollenbeck & Klein, 1987). (I use it on myself all the time. Nothing like keeping track of your fitness goals on Facebook to make you get out your running shoes...)
posted by jeanmari at 4:55 PM on January 5, 2010

Future Headline: According a new paper by somebody with a brain, the results of Freakonomics "proved to be entirely fictional."

Except, of course, it won't get picked up by any of the major media.
posted by oneswellfoop at 5:16 PM on January 5, 2010

blahblahblah: “Before there is too much more Levitt-bashing, I should say that his paper on the Hawthorne Effect seems pretty sound. He and his coauthor actually rediscovered the original Hawthorne data, previously thought lost, and subjected it to its first real analysis in 80+ years. Its a reasonably cool thing, and a good paper. I blame Levitt's NYT journalist co-author for the edgier stuff, especially in the sequel, Levitt's academic research is really good, I don't know his politics.”

That's fair enough, and I don't feel it necessary to hate Levitt or List as far as it goes. I haven't read their books, but that's mostly because I wasn't really drawn to a book that felt the need to put "Freako-" in front of another apparently unrelated word, which is probably not a great reason.

But it seems as if they buy into this complete nonsense that is "the scientific field" variously called management or "org theory," as your "management revolution" link has it. The simple fact is that, like many conceptual confusions masquerading as fields of science, org theory is just an elevation of particular often-less-than-useful concepts to the level of being real objects of study. Nowadays you can take just about any noun whatsoever, stick the word "theory" at the end, and get respect and research grants aplenty from idiots in suits somewhere, if you know how to do it. Seriously, fancy made-up words and discoveries of "the x effect" and "the y effect" doesn't mean you're thinking carefully or clearly about a subject. Really, this kind of think makes me almost livid. It's the dominion of those monkeys clever enough to learn obscure words and invent new definitions for them.

Moreover: there is no purely statistical way to study human beings. Yet we're bombarded with this complete misconception daily now: "hey, did you hear about that study they did where they discovered that, when you give people a banana, 99% of the time they'll peel it and eat it?" This is the most useless tripe imaginable. It's the equivalent of trying to learn about the behavior and society of wild goats by studying the color spectra of their hair - maybe minutely useful, but otherwise a complete distraction. Even in medicine, by analogy with which the apparent all-importance of studies in studying social interaction is drawn, no doctor or physiologist would ever dream of coming to any conclusion whatsoever without some consideration of what's actually going on behind the results of a study.

The unfortunate fact is: there is no easy way to study human beings so that you can "apply it to management." In fact, it's pretty clear that to study human beings the framework of management isn't particularly useful at all - that, insofar as she's being a manager, a person isn't particularly good at understanding humans, just by the very nature of the position. In order to understand human beings, a person has to step outside her role as a manager to a wider view. And, in short: you can't break things down into tiny bits like that and hope the tiny bits are productive. You can't say to yourself, "I want to understand cup of coffee theory, so I'm going to just study what goes into a cup of coffee - forget all that chemistry nonsense, that's not for me, I don't want to hear about acids and bases and various compounds. I'm a specialist, and all I want to study is this cup of coffee." In the same way, you can't separate off some arbitrary division of the human sphere artificially and say, "I want to understand how human beings interact, purely with regard to their productivity or insofar as they are members of organizations - nothing else."

There is one study that can cover these things - a long and difficult study, a study which has already existed for millennia and will exist, at least potentially, as long as the human species does. That study is political philosophy, and no attempt to specialize oneself around it has yet succeeded. You've got to actually study human beings, in the fullest sense, to ever actually understand what human beings are like and how they will act. All that other stuff - morality, the good, justice, love, sex, money, and so on and so on - all that stuff enters into it, and it's a messy business in which there really isn't time for silly made-up words like "org theory." But that's what you have to dive into if you really want to understand human beings and the way they act.
posted by koeselitz at 5:42 PM on January 5, 2010

That's an interesting opinion, Koeselitz, but I'm not really sure your semantic anxieties translate to the success of otherwise of particular studies, and I'm not entirely convinced you're particularly familiar with this discipline, or indeed multi-disciplinary approaches in general.

There is tonnes of research that could easily fall under a dozen different faculties or departments - or may be published as the output of department A, and then subsequently reinterpreted, reassessed and ultimately republished by department B. Rigorous methodologies and interesting results will often shine through.

The claim that there is 'one department to rule them all and in the darkness bind them' is really is a bit silly in my opinion; both intellectually limiting and mis-focussed. Rather than saying "we should be studying X through Y," we should say "what can Y's study of X show us about Z" - thankfully, a question regularly asked in most disciplines.
posted by smoke at 6:06 PM on January 5, 2010 [1 favorite]

koeselitz - as someone occasionally identified as an organization theorist (trained as an economic sociologist, researching entrepreneurship and innovation at a business school), I, unsurprisingly, object a bit to your characterization of the field, and think you are selling it very, very short.

First off, org. theory, like so many fields in the social sciences, is extremely multidisciplinary. In fact, the whole point of the revolution in management kicked off by the Hawthorne Experiment and others is that understanding organizations is complex and spans fields - economics, sociology, social psychology, anthropology, and political science, at the very least, and many more sub-fields as well. In fact, much of the questions that used to be political philosophy - Smith and Durkheim, Weber and Marx - are part of the study of organizations and management now. Research is conducted quantatively with big datasets, and qualitatively with field studies. Nobody, absolutely nobody, thinks that to understand organizations or any other field of management is to say, in your phrasing: "I want to understand how human beings interact, purely with regard to their productivity or insofar as they are members of organizations - nothing else."

In fact, nobody simply studies "organizations" or "management" per se, those are complicated concepts, so we look at particular aspects, through particular lenses. Some people study the roots of gender discrimination at work, others study what makes start-ups succeed, or work motivation, or firm strategy, or leadership. And, counter to your argument, many scholars have found fascinating things that are both true and powerful by examining organizations, ranging from the narrow (how people respond to particular types of incentives) to the broad (the sources of innovation and change within a society).

There will never be "one theory to rule them all" or even one approach (some economists' dreams notwithstanding), but the study of organizations and management matters a lot. Organizations are where most of us spend most of our time, and things that increase or decrease happiness, or performance, or inequality, or innovation, even by a small amount, in aggregate, over billions of people and billions of hours, make a huge difference for the world.
posted by blahblahblah at 6:39 PM on January 5, 2010 [2 favorites]

WorldCat says "Facts and Fallacies of Hawthorne" is a dissertation by Charles Wrege and was published by Garland in 1986. Quite a few places have copies. Wrege's archival material is also listed. Wrege contributes to the Journal of Management History.
posted by halonine at 10:07 PM on January 5, 2010

But it seems as if they buy into this complete nonsense that is "the scientific field" variously called management or "org theory,"

Don't forget 'Cybernetics'

Which is a good term because in 20 years we'll all be managed by super-powerful AIs.
posted by delmoi at 6:29 PM on January 8, 2010