Link works? Check. Dupe? No, maybe. Best of Web? ..suure

January 2, 2008 6:14 AM

The Checklist - "If a new drug were as effective at saving lives as Peter Pronovost’s checklist, there would be a nationwide marketing campaign urging doctors to use it" [single page]
posted by Gyan (65 comments total)

New York Times article on the shutdown of the checklist program by the Office for Human Research Protections.
posted by Gyan at 6:20 AM on January 2 [1 favorite]

There was also an extensive explanation of the checklist system and the results it produced in Malcolm Gladwell's "Blink."
posted by availablelight at 6:54 AM on January 2

They didn't shutdown implementation of the checklist - only research on its efficacy. One cannot ethically do research on subjects without their consent - ever.
posted by jmgorman at 6:54 AM on January 2

That's freaking incredible what that program was open to do.
posted by Brandon Blatcher at 6:55 AM on January 2

Awesome, awesome article. Thanks for posting. Those ICU infection results are remarkable.

It's astonishing to me that health care delivery is treated like an art instead of a science. It seems obvious every hospital policy dealing with patient care should be justified through scientific research.
posted by sdodd at 6:59 AM on January 2

Seconding jmgorman. If you track the efficacy of any experimental program, that's research. And ethical research requires patient consent. Solution: use the checklist and don't track its success.

Of course, any new program that requires money and authority would need performance tracking. If you give nurses the authority to call out and shut down doctors for not following every detail of said checklist, you are going to need to show the doctors why/how the program works.
posted by uaudio at 7:07 AM on January 2

>They didn't shutdown implementation of the checklist - only research on its efficacy. One cannot ethically do research on subjects without their consent - ever.

Not true. Our paramedics are participating in some sort of trial of a different type of intravenous fluid in resuscitating patients in cardiac arrest. No consent is possible.

Since the checklists are just a way to ensure all the recommended things that are already standard are actually done, I'm not sure consent is needed. Do you need consent from a patient so that the pharmacist can call the doctor if he's made an obvious mistake in dosing of the medication he's prescribed? Consent to essentially make extra-sure that everthing everyone agrees is supposed to be done is done doesn't seem necessary. Consent to try your best to avoid mistakes is implied.
posted by kevinsp8 at 7:09 AM on January 2 [4 favorites]

Fascinating article. And everyone should read the NYT article too. It's really fucked up that the government would shut this down for no good reason. There is a certain irony though. One could say that the research program was shut down for not following a checklist for medical research.
posted by grouse at 7:10 AM on January 2

Well, they stopped the checklist study, but it doesn't sound like they are actually preventing people from using it. There is no reason an individual doctor can't continue to use the checklist on their own, as long as they don't record the data without patient consent. (as far as I can tell)
posted by delmoi at 7:10 AM on January 2

Since the checklists are just a way to ensure all the recommended things that are already standard are actually done, I'm not sure consent is needed.

Additionally, this looks less like 'research' and more like bog-standard monitoring of mortality rates. If they were introducing a new, untested procedure, then the various caveats are fine. However, this looks more like the sort of obstructionism that you so often see in an attempt to defend a less-than-adequate status quo than any genuine concern about informed patient consent to research.
posted by PeterMcDermott at 7:14 AM on January 2 [2 favorites]

You guys can complain about the minutia of consent all you want, but I'm going to print out a copy of the essay, highlight the key facts, and mail a copy with a cover letter to my minister of health ASAP.
posted by furtive at 7:29 AM on January 2 [1 favorite]

s/consent/research
posted by furtive at 7:30 AM on January 2

One cannot ethically do research on subjects without their consent - ever.

This is not ethically correct, and it is not the standard endorsed by institutional review boards, either. Retrospective observational research is frequently performed with a consent waiver, and prospective studies that don't involve changing the way care is delivered are frequently approved as well.

In this case some ethic wonk probably decided that creating a checklist to implement a standard way of delivering care was a "change" from the old way, requiring a patient's informed consent. Hooray for ethics.

As far as checklists, most of the great docs I've known have had checklists in their heads for all sorts of situations, and have tended to be very methodical about them. I remember one stroke doc I used to call up at night when I was a resident. Right before he started telling me what to do, he would say "What did the EKG show?" It got so I would save the EKG information to the end and tell him right before I thought he would ask. One time, late at night, I saved the EKG for last, told him what it showed, he said "OK," and then proceeded to ask "What did the EKG show - oh, you just told me." I told the story to some of my co-residents and there was a little bit of laughter, but not too much, because we understood that this is part of how you practice medicine if you want to do it right.

I have some checklists of my own - some mental, some written down - but I am very reluctant to show them around or talk about them with other docs. The reason is that they are only part of the process, to be used in places where dotting all the i's and crossing the t's is more important than creative use of good medical judgment. Doctors are proud of having good medical judgment, but perhaps we are not as proud of being able to follow a checklist as we ought to be.
posted by ikkyu2 at 7:30 AM on January 2 [6 favorites]

Having a career in manufacturing working with items such as steering wheels and airbag covers and as a private pilot to find out that the medical industry does not have checklists is astounding. luckily Ive not had to view this first hand but if an operator has a multipage checklist to make a " simple plastic part" health care should have a longer one. Whom ever the Office of Human Research Protections is they sure have not got the memo's from the justice department on protection of privacy data.
On preview; ikkyu2, the mental list will fail at some point or in certain situations. FAA files are full of examples.
posted by mss at 7:42 AM on January 2 [1 favorite]

Great post title, BTW.
posted by mwhybark at 7:46 AM on January 2 [1 favorite]

It's astonishing to me that health care delivery is treated like an art instead of a science.

It's funny though, because much of Gawande's writing focuses on the idea that medicine has to be more art than science and that we patients would do well to better appreciate that instead of thinking of a doctor as an infallible scientist. What might make me very ill, what might kill me, might not have any affect on you at all, so there will always be room for artistry, at least until we have a much better understanding of the interaction of bacteria/ viruses and us.

This article revolves around one of Gawande's other themes: what are we doing our jobs for? I just finished Better and the idea is sticking with me as a programmer. Every field has a bell curve of performance, every field is becoming more specialized. What happens when you take highly-trained, highly-respected workers and tell them, "You need to start following these steps and checking off boxes"? Anyone with enough ego (and a doctor certainly qualifies) is going to be annoyed. But if we're doing our jobs because we care about them, because we're trying to make the world a better place and make things more efficient, shouldn't we be happy to add anything that improves performance?

Jeff Atwood recently caused a stir by expanding on a simple statement, that programmers can be divided into two groups:

20%: leaders, trailblazers, trendsetters
80%: know exactly enough to get their job done, then go home on the weekend and forget about computers.

Commenters on Jeff's blog immediately deduced there was a 4 in 5 chance they were shitty programmers. Jeff suggested the odds were lower: the simple virtue of caring enough about their craft to read a blog put them in the 20%. If so, it's a jam-packed 20%. What do we do about the bell curve? Gawande asks the question directly in Better and medicine is the best lens to look at the question as the bottom performers in medicine might be killing people.
posted by yerfatma at 7:52 AM on January 2

I can think of another reason to resist something like this: A checklist will generate a paper trail. Now, paper trails are usually a good thing - they ensure accountability on whether a process was followed. On the other hand, they can open up a whole world of problems, even when processes are followed.

If there is a checklist, then every time there is a sub-optimal outcome in the ICU there will be a malpractice attorney taking a good hard look at the checklist. A malpractice attorney is going to look at every notation, every marking, everything, no matter how small or insignificant. Are all the markings in the same handwriting? Were they all made with the same pen? The same indentation on the paper? (indicating that the surface on which the paper was resting changed between notations). Anything that might indicate that the process wasn't followed to the absolute letter might leave a door open a crack for a lawsuit.

Without a checklist, however, that closes that particular avenue for legal Monday-morning quarterbacking. In other words, if there wasn't a detailed written process that includes recordkeeping on process compliance, it's a lot harder to prove that the process wasn't followed. And, with malpractice insurance rates going through the roof, that's GOT to be at least a consideration here.
posted by deadmessenger at 8:10 AM on January 2 [1 favorite]

Given the malpractice insurance rates we should probably ditch charts as well. And burn test results after they've been analysed.

Or, we could consider that having fewer sub-optimal outcomes will balance out the extra avenue for malpractice claims.

Or perhaps we can go that one step further and suggest that a process that helps more medical practitioners follow minimum guidelines to help save lives shouldn't be abandoned because some doctors will not follow the checklists and will then be held accountable.
posted by Reto at 8:20 AM on January 2 [3 favorites]

Jeff Atwood recently caused a stir by expanding on a simple statement, that programmers can be divided into two groups

Of course that 20% of leaders, trailblazers, trendsetters etc. is probably packed with its fair share of assholes who destroy projects by making them needlessly complicated. Most of the time someone who gets the job done and doesn't do anything weird is actually preferable.
posted by Artw at 8:35 AM on January 2 [3 favorites]

Reto, I'm not suggesting that checklists (or charts for that matter) aren't a good thing. I think they're a GREAT thing - anything that improves medical outcomes is a good thing. And I'm not suggesting that doctors that don't follow the checklists not be held accountable.

On the other hand, I know a few physicians, and I've heard enough talk about malpractice witch-hunts to know that simply providing the best care possible isn't enough to keep your ass out of a legal sling if things don't go well for the patient. Which, a lot of the time, they won't.

What I said above was intended to explain a possible reason for resistance to the implementation of checklists, rather than to justify that resistance.
posted by deadmessenger at 8:46 AM on January 2

Sorry deadmessenger, that wasn't supposed to be targetted at you personally - it's a real shame that in our society the financial repercussions of improved accountability are a real demotivator to improving processes.

In short, I'm angry because you're absolutely correct.
posted by Reto at 8:51 AM on January 2

I was an industrial engineer in SE MI and am a new, second career nurse at a large University hospital. I'm suffering a bit of culture shock related to observations similar to mss's. From what I can tell, medicine has always operated as a cottage industry - and still does. Cottage industries don't scale well and the pressure to adopt newer operational models and programs is being met with some resistance: docs and nurses are used to a great deal of autonomy and flexibility in making interventions. Most patient safety initiatives (which are usually re-branded Toyota production system/QC programs) impinge on that autonomy.
posted by klarck at 9:04 AM on January 2

On the other hand, I know a few physicians, and I've heard enough talk about malpractice witch-hunts to know that simply providing the best care possible isn't enough to keep your ass out of a legal sling if things don't go well for the patient.

Doctors would love you to believe this, but from what I've seen, it's largely bullshit.

Very large percentages of malpractice cases go against very small percentages of doctors. Now, I understand that some jurisdictions are more vindictive than others, which can be a problem (for either the patient or the doctor), but in general malpractice seems to me to be a relatively minor problem which is blown out of proportion because no doctor seems willing to admit that a few of their colleagues are outright idiots.

Odd, since every other profession (even ones that involve massive training) has a fair number of unqualified idiots working the trade.
posted by Tacos Are Pretty Great at 9:09 AM on January 2 [1 favorite]

this looks more like the sort of obstructionism that you so often see in an attempt to defend a less-than-adequate status quo than any genuine concern about informed patient consent to research.

Absolutely, and I thank ikkyu2 for providing informed analysis. But I also agree with mss: mental lists are no substitute for physical ones. Vital items do not temporarily slip off a sheet of paper.

Thanks for this post; Gawande is one of the names that always makes me glad to be a New Yorker subscriber.
posted by languagehat at 9:27 AM on January 2

klarck has it right. The whole medical industry is based on a 19th-century mentality, with hand-written notes, desultory habits of recording information, and huge variability in quality of outcomes. That a quality-control technique can save more lives than any "miracle drug" is telling. Not all physicians are happy with the state of their art, but the cultural values are deeply entrenched.

I would propose whole-scale reoganization of the US health-care system, but, of course, there is no system to reorganize.
posted by Mental Wimp at 9:43 AM on January 2

The state of medicine as a science or not is also the topic of Lewis Thomas' The Youngest Science, which I think is worth a read.
posted by hattifattener at 9:46 AM on January 2 [1 favorite]

For those without the time to read through the whole thing, here's the money shot...

The results were so dramatic that they weren’t sure whether to believe them: the ten-day line-infection rate went from eleven per cent to zero. So they followed patients for fifteen more months. Only two line infections occurred during the entire period. They calculated that, in this one hospital, the checklist had prevented forty-three infections and eight deaths, and saved two million dollars in costs.

Holy crap!
posted by ZenMasterThis at 9:52 AM on January 2

re: malpractice.

The very fact that this system exists, is widely publicized, that there is solid, measurable evidence of its efficacy (the Michigan results being the most apparent), and that its implementation is so cost-effective should (and, eventually will) mean that any hospital or Healthcare organization which fails to implement it post-haste will soon find itself the target of negligent malpractice cases.

It could be argued that--in theory at least--any patient who dies of ICU-originated infection in a non-list using hospital is a victim of malpractice.
posted by Chrischris at 9:55 AM on January 2 [4 favorites]

Wow that was a fantastic read.

I always hate it that I end up reading mefi-linked new yorker stories before my paper issue even arrives. It's like I'm buying it for the cartoons, or something..
posted by cowbellemoo at 9:56 AM on January 2

The whole medical industry is based on a 19th-century mentality, with hand-written notes, desultory habits of recording information,...

I just left the cardiologist's office. He didn't have a pen or a pencil that I could see. Every bit of data he generated or learned went into his laptop, where he had what looked like a standardized form. He sent a prescription to my pharmacy from the laptop, too. The 19th Century seems to be over.
posted by Kirth Gerson at 9:57 AM on January 2

As a student, I was kind of astonished when I was scolded for bringing along a checklist for review of systems. I understand that you want me to know it, but it's a list of 108 questions; I'm going to forget one at some point.
posted by a robot made out of meat at 9:59 AM on January 2

Kirth Gerson - yes, but you now have cholera.
posted by Artw at 10:00 AM on January 2 [2 favorites]

I just left the cardiologist's office. He didn't have a pen or a pencil that I could see. Every bit of data he generated or learned went into his laptop, where he had what looked like a standardized form. He sent a prescription to my pharmacy from the laptop, too. The 19th Century seems to be over.

Yes, many individual practices and HMOs are upgrading their systems to electronic formats. However, as a researcher, I have found that even these systems are full of errors, inconsistent usage, and free text. Many of them are essentially electronic versions of the paper systems. And yes, when the transaction involves payment, the system is very accurate and efficient. When it involves patient welfare, not so much.
posted by Mental Wimp at 10:03 AM on January 2

I stopped reading after the little girl lived. That's my happy story for the day.
posted by davejay at 10:17 AM on January 2 [1 favorite]

Yes, many individual practices and HMOs are upgrading their systems to electronic formats. However, as a researcher, I have found that even these systems are full of errors, inconsistent usage, and free text. Many of them are essentially electronic versions of the paper systems.

Without the hand-written notes. That's a huge leap forward, right there. What is "free text"?
posted by Kirth Gerson at 10:21 AM on January 2

And yes, when the transaction involves payment, the system is very accurate and efficient. When it involves patient welfare, not so much.

...which is why I was so amazed last year when I got a medical bill for $0.00. Yes, really. No, I'm not joking.

It was even more exciting when the hospital & the insurance company started going back & forth with each other for $0.00. I lost track of how many notices I received, saying that the hospital refused to pay $0, so I might be billed, then the insurer refused to pay $0, so I might be billed, and so on.

It took six months, and finally ended with me writing a check for $0.

Yes, really. No, I am not kidding. Blue-Cross/Blue-Shield of Illinois, and Northwestern University Memorial Hospital. Makes me wonder if they've got squirrels powering the MRI machines.
posted by aramaic at 10:26 AM on January 2 [13 favorites]

Without the hand-written notes. That's a huge leap forward, right there. What is "free text"?

Means 'free text', just like what you're posting here. It can't be easily queried or processed by computers.
posted by delmoi at 10:33 AM on January 2 [1 favorite]

Jeff Atwood recently caused a stir by expanding on a simple statement, that programmers can be divided into two groups:

20%: leaders, trailblazers, trendsetters
80%: know exactly enough to get their job done, then go home on the weekend and forget about computers.

More like 2%, 98%. Programmers aren't like doctors where there is a rigorous screening process, anyone can call themselves a programmer. Interestingly, according to the post the "20%" and "80%" are just labels, they're not actual estimates (as far as I can tell). I think it was meant to evoke the famous "80/20 rule".
posted by delmoi at 10:41 AM on January 2

Means 'free text', just like what you're posting here. It can't be easily queried or processed by computers.

I'm not going to buy your dictionary, man.


DICOM - more non-19-Century stuff.
posted by Kirth Gerson at 10:44 AM on January 2

Since a lot of talk was about the bell curve and doctors, it reminds me of a joke my father told me as he was driving me to the hospital (while my appendix was trying to rupture).


Q: "What do you call the person who graduated last in their class from medical school?"

A: "Doctor."


Checklists are good things.
posted by daq at 10:47 AM on January 2 [1 favorite]

The blind flight practices of the 19th century should have ended at the beginning of the 20th, with Ernest Codman's introduction of records standards, outcome tracking, and morbidity and mortality conferences.

If your hospital's a halfway decent one, its doctors will be held accountable for the outcomes of their work, and such accountability will be based on stringent rubrics and intensive data collection. I have a relative who performs that data collection for a living, actually; she does outcomes management and National Surgical Quality Improvement compliance at the local hospital.

That's why I was surprised to see the Office for Human Research Protections come down so hard on The Checklist. Such guidelines aren't exactly a novelty for the industry. (Though it's true that doctors, especially those who consider themselves artists as much scientists, may not enjoy performing to a mandated sequence. I hope that Pronovost’s work will change that reluctance.)

Now I'm curious as to how the Checklist differs from conventional rubrics, and how the "research" that the OHRP flagged differs from conventional outcome tracking. Hopefully, this was an isolated incident of technical disqualification instead of a change in the government's attitude to data gathering. Patient privacy and consent are crucial, but so is the medical industry's ability to monitor itself.
posted by Iridic at 10:51 AM on January 2 [1 favorite]

More like 2%, 98%. Programmers aren't like doctors where there is a rigorous screening process, anyone can call themselves a programmer. Interestingly, according to the post the "20%" and "80%" are just labels, they're not actual estimates (as far as I can tell). I think it was meant to evoke the famous "80/20 rule".

Is there a reason you have to redefine everything you see? Do you ever make a comment that doesn't include some asinine aside about how everyone is wrong? It's infuriating because you spend all this time nitpicking and completely miss larger points. The idea is that every practice has its stars and it's slackers, regardless of screening process or educational requirements. The questions are:

Do we cut a certain % off the bottom? If so, how much? Why?
Whether we cut anyone out, how do we deal with the Bell Curve among those that remain? No one wants to see a doctor in the 20th percentile. No one wants a programmer in the bottom quintile working on their make or break project.

But we'll always have a bottom 20%. Checklists and processes can go a long way to making even those folks productive, but there's a tradeoff point where people start to feel like cogs in a machine and the tradeoff point comes earlier and earlier the more specialized the practice is.
posted by yerfatma at 10:52 AM on January 2

And don't take a physician's sense of humor for granted. I made the mistake of referring to a doctors patients as her customers. Bad move.
posted by notreally at 11:01 AM on January 2

The biggest problem I see in Medicine is the lack of CRM. CRM? Crew Resource Managment.

What CRM does -- it explicitly rejects the "I am the Captain, we do things my way." argument. Why? Because that attitude kills people. It caused the largest accident that didn't involve a building -- the Tenerife Disaster -- because the KLM Captain -- one of the most senior captains at KLM -- was certain that he had takeoff clearance when he did not, and his flight crew was afraid to speak up to the senior Captain. Because of this deference, over 500 people died.

So, now, we don't have the Captain and First Officer flying -- we do have them, and the Captain gets the left seat, but once they're about to go underway, the role change to Pilot Flying and Pilot Not Flying. The Pilot Flying flies the aircraft. The Pilot Not Flying calls out checklists. Both have to agree before continuing. Thus, if the PNF doesn't agree that you have takeoff clearance, they will hold the throttles back and stand on the breaks until both agree that they do -- and it doesn't matter if the PNF on that leg is the junior member of the crew.

I strongly suspect that if we deployed doctors as teams, with one treating and one calling out, many of the simple mistakes that cause real harm would stop. It's hard to remember all the steps, every time, when you're asked to do a different procedure every ten minutes.

As to familiarity? Captain van Zaten had flown planes off of runways thousands of times. He still died, because he did so without clearance. I don't care how many time you do it. You run the checklist. It's the ad-hoc optimization that make you forget to set flaps correctly, or makes you think you've gotten clearance when you haven't, or makes you forget to start an engine before takeoff, or makes you powerback in the snow and ice. (Yes, all of these have happened.)

Every year we've made air travel safer -- and a big part of it is mandating the use of checklists and CRM. The other big thing is then looking carefully at the failures and implementing procedures -- on those same checklists -- to make sure they're checked for. And still, to this day, a great deal of accidents occur because somebody decided that the checklist wasn't that important.

So, yes, day-to-day *and* rarely used procedures need to be checklisted. Rare ones need it because you don't have practice to guide you. Common ones need them because familiarity brings those ad-hoc "optimizations" that kill people. But having someone with the authority to say "Don't skip that" is just as important. Checklists are useless if doctors decide that they're good enough not to need them -- regardless if that is true or not.

I have some checklists of my own - some mental, some written down - but I am very reluctant to show them around or talk about them with other docs.

A big reason to have everyone working the same checklists is that it makes omissions leap out. It may be that you were going to drape the patient in step four, but Dr. Foo does that in step two. When should Nurse Bar speak up that the step on the checklist has been missed?

Also: Checklists are for procedures, not for diagnosis. Checklists must avoid judgement calls -- at that point, they're not checklists, they're decision trees, and that's a very different beast.
posted by eriko at 11:07 AM on January 2 [17 favorites]

Funny story on consent for research: My wife had both of our children in one of the best teaching hospitals in New York for that particular experience. It's the place they take you if, God forbid, something goes wrong with your birth at another hospital. As it turned out she decided to have an epidural.

When she made that decision a nice young man came in and asked if she would mind participating in a study. It seems there are three drugs routinely used for epidurals, all of which have been used forever at most hospitals (most use all three interchangeably) but this particular hospital had only ever used one of the three. They were now conducting a blind study to determine whether either of the other two produced better results. The young man wanted to know whether we would mind participating. We discussed the issue with our OB who had seen all three used in her many years of practice at many other New York hospitals and she confirmed that there was no reason to think there was any difference between them, so being a good little scientist, my wife agreed.

The nice young man seemed supernaturally happy about this, and low and behold the net result was that the chief of anesthesiology for the department came to administer the epidural (man he was good) and the nice young man came back every hour on the hour to check on my wife and make sure her levels were okay and she was comfortable. All in all it was a great experience, anesthesia-wise.

Afterward I was curious about why the resident had been so happy to get our agreement to participate. I asked him what sample size he needed for valid results. "We want 600," he said. How many do you have? "You guys were 158." Not bad, I thought. How long has the study been going on? "Four years."

This particular hospital delivers over 5,000 babies a year. Not every mother has an epidural and I'm sure not every mother is asked if she wants to participate -- the nice young man can only do so much -- but man that's a tough row to hoe. The moral of the story: people don't want to hear "consent to a study" when they are lying in a hospital bed. They don't want you experimenting on their babies, even if you're not actually experimenting on their babies. They have enough to worry about, I guess. Lucky there are people like my wife who can actually remain sane and rational during the process of giving birth. I know I wouldn't.
posted by The Bellman at 12:26 PM on January 2 [1 favorite]

They don't want you experimenting on their babies, even if you're not actually experimenting on their babies.

Ironic, because if no one participates in these studies, they ARE experimenting on your babies, but in such a way as to never ascertain which anesthetic is best. True of any therapy, actually. Without such studies, physicians are left to guess which therapy is best, the worst kind of experiment for humans to be subjected to. {snark on}Fortunately, though, you don't need to consent to that kind.{snark off}
posted by Mental Wimp at 12:33 PM on January 2

What is "free text"?

Electronic hand-written notes: more legible, just as opaque.
posted by Mental Wimp at 12:36 PM on January 2

Exactly, MW, which was my wife's thinking: "I'm going to benefit from this study when I come back to this hospital for my next kid; why wouldn't I participate?"
posted by The Bellman at 12:40 PM on January 2

@eriko: I strongly suspect that if we deployed doctors as teams, with one treating and one calling out, many of the simple mistakes that cause real harm would stop. It's hard to remember all the steps, every time, when you're asked to do a different procedure every ten minutes.

As someone with a good deal of experience working in a system like this (As a Nuke in the navy), it doesn't really fix any problems. Eventually, someone will make a mistake that won't be caught by the other guy. Then you have two highly-qualified, hard-to-replace people who can't do anything until they get recertified. Then, if you're like my organization was, you add more eyes to it. Then you have three highly-qualified, hard-to-replace people who can't do anything until they get recertified. At some point, you have to realize that there will be failures in your system. And I know how much that sucks because failure in nuclear power means the same thing as failure in medicine: People Die.

Going back to what yerfatma was saying, if you are just looking to cut X% off the bottom, you will never reach your goal because there is always X% more that you can cut off the bottom.
posted by ArgentCorvid at 1:38 PM on January 2 [1 favorite]

Not a bad article, it's covered here as well. The aviation analogy is a useful one, and hints at all the other things that can be done to improve human performance in complex domains. Checklists are well and good, but if your equipment and procedures encourage human error, checklists may be acting as more of a band-aid than a cure.

One really blatant example is a self-administered anaesthesia (PCA) machine that was associated with the deaths of several patients. here's a fairly technical discussion of the design flaws with this specific machine. The good news is that many hospitals now have dedicated human factors staff whose job it is to evaluate medical equipment before making purchases. It's a very exciting time to be working in medical human factors.
posted by anthill at 1:42 PM on January 2

oops - last link should go here. Great summary of some of the work that's being done to help improve heathcare safety and performance at some major Toronto hospitals.
posted by anthill at 1:44 PM on January 2

My trusty pack-for-trip checklist rocks.
posted by bonaldi at 3:06 PM on January 2

The weird thing about the research consent thing is in pediatric cancer, more than 60% of kids are in clinical trials-- where you'd think people would be *least* likely to consent. Whereas in adult cancer, less than 1% of people are in clinical trials.

And guess what? Cure rates for childhood leukemia are now up to 70%, from something like 5% in the early 60's. Whereas for adult cancers, progress has been much, much slower.

I wrote about this here The main author on the piece won the Lasker Award in 2007 (which is seen as sort of a pre-Nobel, as many who win it go on to the Nobel), so I felt very cool to have worked with him.

His point was basically the adult cancer people need to take a hint from what the kids' field has done-- admittedly, it will be harder because it's a much larger field, but if you see every case as a chance to improve research, you won't get stuck with ruts of "that's the way we do it because that's the way we've done it" as much.

What amazes me about the checklist thing is that you can just introduce a new behavioral practice of any kind-- a talk therapy, say or telling people to stand on their heads to cure depression-- without problems from anyone. You can even advertise it. No requirement to prove it safe or effective.

But when someone tries to *study* how to implement evidence-based care, they get in trouble! I'm a fan of regulations usually-- but this is an example of where blindly following regulations completely misses their point in the first place..
posted by Maias at 4:08 PM on January 2 [1 favorite]

As someone with a good deal of experience working in a system like this (As a Nuke in the navy), it doesn't really fix any problems. Eventually, someone will make a mistake that won't be caught by the other guy.

Sure -- there is no 100% solution. However, two sets of eyes can catch a whole bunch of errors. Talk to any commercial pilot, all of them have stories of things that would have caused real issues -- or real deaths -- if the guy in the seat next to them hadn't said anything. A distressing amount of these are summed up in a simple two word question. "Gear down?"

It's not perfect. Mistakes will still be made, people will still die from them. It's impossible to get 100% error free, but if you get a guy to 99%, and then you have two of them, you have a surprisingly reliable system.

The weak point is still personality -- one guy convincing the other one that everything is okay, when it wasn't. But CRM and checklists have *drastically* reduced the error rate made by flight crews, and this has saved money and lives.

I'm completely not surprised that checklists would do the same thing for medical procedures, and I suspect that CRM would improve that.
posted by eriko at 4:23 PM on January 2 [1 favorite]

> What is "free text"?

Two people have answered the question already, but I want to provide a concrete example. You know when you get a multiple-choice test question that also has a "E. Other (Please describe)" free-form answer? That's "free text."

Why is that a problem? In order for computers to process data, they generally need that data to be regular, structured, and normal. "Structured" refers to the relationships between data elements and constraints on the allowable values, while "regular" and "normal" refer to consistency and the elimination of duplication. Computers can't generally understand written English -- they need a lot of rules imposed on your input before they can do anything meaningful with it. (Storing text is easy, but you won't be able to analyze it without comparatively laborious and slow human intervention to impose rules after the fact.)

So it's disappointing to hear a researcher describe hospital systems as "full of free text." The doctor might as well be using old paper records, since the computer won't be able to use the data she's entering anyway.
posted by sdodd at 5:55 PM on January 2

I was surprised that Gawande didn't bring up Crew/Cockpit Resource Management, and glad that eriko did. As it happens there is attention being given to this, e.g. EMCRM (Emergency Medicine Crisis Resource Management), or CRM and its Applications in Medicine. It seems blindingly obvious that nurses should be "co-pilots" in the operating room or anywhere else, and certainly they are often portrayed that way on TV. It shouldn't be a threat to the doctor's expertise, either -- any lieutenant worth his salt knows that he should rely on his sergeant's field experience to supplement his classroom smarts.

Frankly, though, I was just chilled to realize in retrospect all the stuff that could have gone wrong when my parents were in the hospital for major operations the last few years.
posted by dhartung at 7:10 PM on January 2

Hmm. If you don't think free text entry is required then you have an awful lot of faith in whoever is creating the forms.

Me, I think they should skip all this early 20th century form-filling factory line stuff and go straight to Medecine2.0, with tagging!
posted by Artw at 8:16 PM on January 2

I was just thinking about how this reminds me of something I read once. Japanese bullet train drivers have to salute (with their hands) every signal they pass. This essentially works like a one-item checklist—by doing so they are more likely to pay attention to what the signal says.
posted by grouse at 9:10 AM on January 3 [1 favorite]

One caveat: The jury's still out as to how large an effect CRM actually has.
posted by anthill at 12:26 PM on January 3

And one piece of practical advice given to me by a healthcare human professional. If you're in the hospital make sure you (or a friend) asks hospital staff three questions before they administer any treatment to you:

1) who am I?
2) what do you think is wrong with me?
3) what are you planning to do about it?

That should catch a good chunk of possible medication/surgery errors: wrong patient, wrong ailment, wrong treatment.
posted by anthill at 12:30 PM on January 3

The checklist says you'd say that, Mr. Smith. Now let's get you prepped for that lobotomy!
posted by Artw at 1:04 PM on January 3

many individual practices and HMOs are upgrading their systems to electronic formats. However, as a researcher, I have found that even these systems are full of errors, inconsistent usage, and free text.

My primary care office switched to electronic charts a couple years ago. When I expressed excitement at seeing my doctor enter his notes by selecting options on a computer screen, he had a different opinion. He said it was great for accuracy, but terrible for memory. I'm paraphrasing, but his general point was, "If I were writing notes in my own handwriting about you, I'd probably remember you next time you came in, especially after re-reading the chart. Now that I'm clicking radio buttons, you'd be lucky if I remember what we talked about by the time you make it to the lobby."

So it's disappointing to hear a researcher describe hospital systems as "full of free text." The doctor might as well be using old paper records, since the computer won't be able to use the data she's entering anyway.

I agree that free text is way less useful for research purposes, but for the doctor's purposes it can express nuances that nice, regular multiple-choice options can't. Letting the doctor record details and impressions is important so the doctor can use them, not so the computer can.

1) who am I?
2) what do you think is wrong with me?
3) what are you planning to do about it?

When my dad had minor throat surgery last year, he said to the pre-op nurse as they wheeled away to the OR, "Now, make sure the doctor knows to work on the RIGHT knee, not the left!" The poor nurse just about had a heart attack, before we convinced everyone that Dad is an inveterate jokester. Heheh... memories.
posted by vytae at 3:14 PM on January 3 [1 favorite]

Is there a reason you have to redefine everything you see? Do you ever make a comment that doesn't include some asinine aside about how everyone is wrong?

Well, it irritates me when people just make things up and pull numbers out of their asses. At least Attwood made it pretty clear that he was just labeling the groups 80/20

There are two "classes" of programmers in the world of software development: I'm going to call them the 20% and the 80%.

Your post made it seem like that was the actual ratio, when in fact there is nothing to support it, and it's obviously wrong to anyone in the "20%".

Now, if Attwood actually believes the ratio breaks down like that (and he might) he's been spending a lot of time around really good programmers, which is what you would expect for someone working at Microsoft.
posted by delmoi at 11:23 PM on January 3

Interesting choice of labels. And dumb.
posted by Artw at 11:38 PM on January 3

he's been spending a lot of time around really good programmers, which is what you would expect for someone working at Microsoft.

Last I checked he still worked at Vertigo Software. It irritates me when people just make things up and pull them out of their asses.
posted by yerfatma at 6:02 AM on January 4 [1 favorite]