Monday, May 23, 2016

Operative risk and surgeon decision-making

Should it surprise you that surgeons might have differences of opinion about whether or not a patient should have an operation?

It doesn't surprise me, but apparently a lot of people were taken aback by an Annals of Surgery paper published online last week stating just that.

The authors gave 767 surgeons four brief complex clinical scenarios and asked whether they would operate on each patient. The vignettes were purposely designed to not have "correct" answers.

In response to the question would you recommend an operation, the surgeons could choose one of the following responses: very likely, unlikely, neutral, likely, or very likely.

If you were in the emergency department with mesenteric ischemia, would you want a surgeon who responded "neutral"?

Why the authors selected five possible choices is puzzling. In real life when you are faced with a difficult decision in the middle of the night, you don't have five options. You have only two—operation or no operation.

More about that later.

The surgeons' estimation of the risks of each procedure varied widely, and most of them agreed about recommending surgery only for the patient with a small bowel obstruction.

In a Vox story about the paper, a Harvard health policy expert, Dr. Ashish Jha, said the findings were “disturbing."

I would call the findings "expected." These were difficult cases with no right answers.

A second paper in the same journal by the same investigators came up with somewhat different results. It randomized 779 surgeons into two different groups. One group had access to the American College of Surgeons operative risk calculator score and the other did not. For the same four clinical scenarios, surgeons who were given the risk calculation score estimated risks significantly closer to what the calculator’s values were—another non-surprise.

The difference between the estimated risks between the two groups was statistically significant but probably not clinically significant. For example, surgeons who used the risk calculator score estimated operative risk for the small bowel obstruction patient at 13.6% compared to 17.5% for the surgeons who didn't know risk calculator score, p < 0.001. Would the 3.9% difference between the two estimates really change a surgeon's mind about operating? I doubt it.

The effort to quantify risk so precisely may not only be wrong; it could be impossible.

Radiologist Saurabh Jha blogged about this two years ago. He wrote, “Numbers are continuous. Decision-making is dichotomous. One can be at 15.1%, 30.2% or 45.3% risk of sudden cardiac death. But one either receives an implantable cardioverter defibrillator (ICD) or does not. Not a 15.1% ICD.”

Jha concluded, “You can remove the burden of judgment from a physician but then you will no longer have a physician.”

As the authors of the two papers pointed out, among other points to be considered is that the risks of not operating are unknown. The topic has not been studied and probably never will be.

The most important finding of the second paper was that "averaged across the four vignettes, the two groups did not differ in their reported likelihood of recommending an operation, p = 0.76."

Since the first paper portrayed surgeons as wildly erratic at estimating risk, it of course received all the attention.

Monday, May 16, 2016

Deciding whether adverse events are preventable or not

Adverse events and poor outcomes are not always preventable. Deciding whether an adverse event is preventable or not can be difficult.

"To Err is Human: Building a Safer Health System," the original Institute of Medicine report in 1999, stated that  between 44,000 and 98,000 deaths each year were caused by preventable medical errors.

That report was widely cited and spawned a number of studies and reviews claiming that anywhere from 250,000 to 440,000 preventable deaths occur in the United States every year.

I was critical of the 440,000 deaths paper as well as the most recent of these estimates—the one claiming 250,000 deaths due to medical errors per year.

It's not widely known or perhaps simply forgotten, but the 1999 Institute of Medicine report also came under fire. In 2000, two researchers from Dartmouth, Drs. Harold C. Sox Jr, and Steven Woloshin, published a critique called "How many deaths are due to medical error? Getting the number right."

In that paper, the authors pointed out that the IOM was correct about the number of adverse events per hospitalization (2.9-3.7%). However, the IOM's report was based on only two studies which used data from 1984 and 1992 and did not define preventable adverse events or medical errors.

Re-analyses of the two papers judged that 54-69.6% of deaths due to adverse events were preventable, figures that Sox and Woloshin said were subjective and not reliable because the re-analyses were performed by looking only at summaries of patient hospitalizations not the actual records.

As is the case with all subsequent preventable death studies, the IOM report relied on many estimates and extrapolations.

Sox and Woloshin concluded, "It is unfortunate that we do not have a credible estimate of the number of deaths due to medical errors." That statement remains true today.

If you think it's easy to tell a preventable complication from a non-preventable one, read this summary of a case from one of the papers cited in the IOM extrapolation.

A 39-year-old woman employed as an engineering and science technician had a laparoscopic cholecystectomy. Three days later, she developed fever and abdominal pain and was found to have a bile leak and possible infectious peritonitis requiring 4-day hospitalization for observation only. 

Was this complication preventable or not?

Feel free to explain your answer in a comment.

In a few days, I will tell you what the reviewers decided.

Friday, May 6, 2016

When bad research is not critically reported by journalists

Yesterday I posted a critical review of the study "Medical error—the third leading cause of death in the US."

I did not have time to address the media coverage of the paper, but fortunately the website HealthNewsReview did.

Their post started with "Seemingly all the major outlets carried the story, with headlines so alarming that they’d have any conscious hospital patient demanding an immediate discharge."

They called attention to headlines which included the word "now" such as these:

CBS: Medical errors now 3rd leading cause of death in U.S., study suggests
Washington Post: Researchers: Medical errors now third leading cause of death in United States
Nature World News: Medical Errors Now the Third Leading Cause of Mortality in U.S.

They pointed out that the headlines were similar to the title of a press release issued by the PR department at Johns Hopkins and that using "now" in the headline implies that the incidence of deaths caused by medical error has increased.

There is nothing "now" about the BMJ paper. if you read my post from yesterday, you will know that it was based on studies from the first decade of this century.

The HealthNewsReview story quoted my criticism of the extrapolation of the 35 deaths in the studies reviewed by the BMJ paper 's authors into a figure of 250,000 deaths per year nationwide and suggested that those reporting on the paper should have taken a closer look at the calculations.

HealthNewsReview ended with a concern that studies like this might frighten people into avoiding medical care.

It could be worse than that. It may provide fodder for the anti-vaccine movement. Based on the Washington Post story, a completely bizarre analysis of the BMJ paper appeared on Infowars yesterday. Note the sub-headline.
Read it to the end to grasp its full impact and enjoy the irrational comments too.

That concern that the public might take the BMJ paper's claims the wrong way was shared by some of the commenters on the BMJ website. Below are some excerpts from comments critical of not only the paper but the BMJ for publishing it.

There are many things we can do in medicine to improve patient care, but frightening the public by claiming that errors by their own caregivers constitute the third leading killer in the world, is not one of them.

Makary and Daniel’s analysis has misrepresented the true situation in 2 respects. Firstly, their extrapolations from the literature are unrealistic, based on flawed assumptions; secondly the single case study which they cite is unrepresentative of the majority of preventable hospital deaths.

Such an interpretation suggests that the authors believe that all hospital patient deaths are due to preventable errors. Most people who take care of cancer, trauma and cardiac patients know that there are many causes of death unrelated to medical errors.

In the UK, the 2015 in-hospital mortality rate was 1.05%. A recent study of UK hospital deaths estimated that 3.6% were avoidable, giving an overall preventable lethal adverse event rate of 0.04%. This is an order of magnitude lower than the 0.71% estimate calculated by Makary and Daniel. If the true rate is closer to 0.04%, the total number of annual preventable deaths in US hospitals may be 14,166 rather than 251,454.

Shame on the authors and publishers of this piece. This is a sensationalized title that harmfully misrepresents data, and seems to propel a dangerous paradigm against the medical establishment.

And my favorite (goes for all the news media too):

Congrats on the huge clickbait title. The National Enquirer would be proud.

Thursday, May 5, 2016

Hand hygiene follow-up: The CDC may be reading my posts

Two weeks ago I blogged about a hand hygiene study that showed a 6-step alcohol-based hand washing technique significantly reduced hand colony counts compared to a standard 3-step technique.

The 6-step process took about 45 seconds and when coupled with the amount of time a healthcare worker took to remove gloves, go to the dispenser, and let the hands dry, at least a minute will have elapsed.

In a busy emergency department with 10 or more hand hygiene events per hour, personnel might spend 80 to 120 minutes per shift on that activity alone.

Today is World Hand Hygiene Day and none other than the Centers for Disease Control has validated my hypothesis. Here is what the CDC tweeted today:
An anonymous commenter on my original post said, "Now imagine doing this [the 6-step technique] on rounds, with 40 patients to be seen, and a chief, pgy 3, and 2 interns who have to line up at the sink to do this."

Another commenter named Levi wondered if lower hand colony counts resulted in fewer infections and safer patient care. I don't think that has been investigated.

The CDC also reminded us that hand hygiene must occur before and after every patient encounter:
As I understand it, the reason hands must be washed after patient contact is that some people contaminate their hands when they remove their gloves. Wouldn't it be simpler and less time-consuming to teach people the correct way to take off gloves? It's not that hard really. Surgeons, scrub techs, and nurses do it every day

Maybe it would be better if we washed our hands more often, but took less time to do it.

Thanks to @michelaccad for alerting me to the @CDC tweets.

Are there really 250,000 preventable deaths per year in US hospitals?

For the last couple of days, the Twitter medical community has been discussing the latest in a long line of papers attempting to estimate the role of medical error as a cause of death.

This week's entry appeared in the BMJ (full text available here) and was by a surgeon at Johns Hopkins, Dr. Martin Makary, who claims that 251,454 patients die from medical error every year.

Makary's review extrapolated that figure from three papers published before 2009 which had a combined 35 supposedly preventable deaths. That's not a typo—35 deaths in all. One of the papers stated that all 9 deaths in three tertiary care hospitals were preventable. In his BMJ paper, Makary says, "some argue that all iatrogenic deaths are preventable."

I disagree. I have analyzed other papers on this subject and pointed out that certain complications and deaths are not 100% preventable. For example, no study of deep venous thrombosis and pulmonary embolism shows total efficacy of any prevention strategy. And some patients will suffer myocardial infarctions and die even when they are properly treated.

In this month's BMJ Quality and Safety, Dr. Helen Hogan of the Department of Health Service Research and Policy at the London School of Hygiene and Tropical Medicine discusses the problems associated with using preventable deaths as a measure of quality.

Tuesday, April 26, 2016

It's time to discuss surgeon headgear again

I received this email last week:

My state has recently banned the time-honored surgical cap,
Timed-honored cap
in favor of the “bouffant” cap.
"Bouffant" is French for "doofus"
I have been wearing the disposable surgical caps for my entire career, and have one of the lowest infection rates of any surgeon at my hospital.

Not only are the bouffant caps uncomfortable, I have now developed a severe allergy to them, making them unwearable. Now they say I must wear a full head/beard cover type cap in order to satisfy their requirements...REALLY? 

Head and beard cover even more uncomfortable than bouffant
I asked the hospital to show me a study that proves that the use of a bouffant cap has ANY advantage at all. Of course they can't, because no such study exists.

Monday, April 25, 2016

Does talking about burnout cause more burnout?

Almost every day over the last few years, someone has written about physician burnout or depression.

The problems begin in medical school. A recent paper featured drawings that medical students had done depicting faculty as monsters. One student felt so intimidated during a teaching session that she drew a picture of her urinating herself.
The paper equated faculty and residents supervising students to “zombies, vampires, ghosts, and other supernatural figures.” In dealing with the state of the world today, the authors cited a comment by the novelist Stephen King saying that to cope with adversity, people make up horror stories. That sounds pretty serious.

Could the problem be declining student resilience? An article about college students in Psychology Today pointed out that they are less able to deal with seemingly minor affronts. And teachers are reluctant to “give low grades for poor performance, because of the subsequent emotional crises they would have to deal with in their offices.”

This has forced faculty “do more handholding, lower their academic standards, and not challenge students too much.” The article pointed out that college students exhibit more anxiety and depression and take more prescription drugs for these problems than ever before.

It’s not just students.

A systematic review of 54 studies found that 29% of resident physicians were depressed or had depressive symptoms. The number ranged from 20.9% to 43.2%, depending on the method studies used to assess depression.

Investigators from the Mayo Clinic and the AMA found that more than 50% of practicing physicians surveyed exhibit symptoms of burnout, and the problem has worsened since 2011. Depression and suicidal thoughts are also common.

All of these studies have been covered extensively by media such as the Washington Post, NPR, Time Magazine, Forbes, among many others and were widely discussed on Twitter.

Here’s a thought. What about “emotional contagion,” the subject of a 2014 paper published in PNAS? The authors studied 689,000 Facebook users and found that negative emotions can be transferred from one person to another without direct contact. They concluded, “…emotions expressed by others on Facebook influence our own emotions, constituting experimental evidence for massive-scale contagion via social networks.” If emotional contagion has such an impact on Facebook users, maybe all these stories about burnout and depression have an impact on students and doctors too.

There is certainly much to be depressed about in medicine—stress, declining reimbursements, soul-sucking electronic medical records, long hours, arbitrary rules, and much more.

Could it be that the more stories written about burnout and depression, the more burned out and depressed doctors become?