Wednesday, September 2, 2015

Variation is not causation

I made a rookie mistake in statistics of the “correlation is causation” genre by confusing variation for causation in the recent JAMA Surgery paper referred to in my last post. I contacted Dr. Timothy M. Pawlik, the lead author of the Johns Hopkins study, who said the following:

"The model is explaining and attributing variation in readmission and not attributing readmission itself to the different domains. The model suggested that only 2.8% of the variation in readmissions was attributable to surgeons. This is different than saying that only 2.8% were the 'fault' of surgeons. A more accurate interpretation would be that only 2.8% of the variation seen in readmissions was attributable to provider level factors. The majority of the variation in readmission was due to patient factors."

He added that some of the 82.8% variation in readmissions attributable (note: attributable doesn’t mean it’s the patient’s fault) to the patient could be modified by better medically managing patients' comorbidities or not operating on some of these patients.

That readmissions can be explained by a single domain or a single person is simplistic. Dr. Pawlik's clarification confirms my original concern that attributing differences in patient outcomes solely to differences in technical quality of surgeons is probably inaccurate, statistically speaking.

Variation is not causation but variation is still a call to action. Regardless of who is to blame for unfavorable outcomes, surgery is a team sport. The incision is just as important as the community care. In this regard, I am certain that ProPublica and I are on the same side. Let’s work together so that we see the whole story behind the numbers.

Sunday, August 30, 2015

Are surgeons the cause of high postoperative readmission rates?

No, according to a recent paper published online in JAMA Surgery.

The authors concluded, "The majority of the variation in readmission was attributable to patient-related factors (82.8%) while surgical subspecialty accounted for 14.5% of the variability, and individual surgeon-level factors accounted for 2.8%."

The investigators looked at data for over 22,000 surgical patients treated at Johns Hopkins and found the overall rate of readmission within 30 days was 13.2%. After the exclusion of those who performed fewer than 21 operations per year, 56 surgeons made up the study cohort.

Multivariable analysis showed significant non-modifiable patient-related factors associated with readmission were African-American race/ethnicity, more comorbidities, occurrence of postoperative complications, and an extended length of stay.

Variation in readmission by subspecialty ranged from 2.1% after breast, melanoma, or endocrine surgery to 37% following cardiac surgery.

The authors pointed out that this study "echoes growing concerns regarding the use of readmission as a quality metric based on its current methods."

Let's compare it to the controversial ProPublica Surgeon Scorecard.

Both the Surgeon Scorecard and the JAMA Surgery paper used data from the years 2009 through 2013. The scorecard involved only eight high-volume low-risk in-patient procedures while the paper looked at in-patient surgery of all types.

From an article written by the authors of the Surgeon Scorecard: "If a patient was readmitted to any hospital (not just the hospital where the surgery was performed) within 30 days of a surgery for one of the conditions we identified, we counted the case as a complication for the surgeon who performed the initial procedure."

What we learned from the JAMA Surgery paper raises some questions about the the Surgeon Scorecard. On Twitter, I asked for comment from Marshall Allen, the lead author of a white paper [not peer-reviewed] describing the methodology of the Surgeon Scorecard.

Between attacks on my credibility because I choose to use a pseudonym, he said that they did not count most readmissions as complications. It is unclear from the article, the white paper, or its appendices exactly which complications were included. For clarification, we could ask the "surgeon experts" who advised ProPublica, but their names have not been disclosed. They are anonymous, just like me.

According to the white paper, surgeons were blamed for 64,367 (46%) of all complications incorporated into the Surgeon Scorecard. Table 3 of the white paper lists the 20 most frequent complications. The top three, comprising 26,795 complications, were postoperative infection, iatrogenic pulmonary embolism, and infection/inflammatory reaction due to internal joint prosthesis.

Other studies have shown that not all occurrences of those three complications are attributable to a surgeon's misdeed. Among the rest of the top 20 causes of readmission were postoperative pain, fever, and dysphagia (difficulty swallowing)—again possibly not the fault of a surgeon.

So the JAMA Surgery paper says surgeons are responsible for 2.8% of readmissions within 30 days, but ProPublica's self-published white paper says 46% of all readmissions are due to something a surgeon did or did not do.

Who to believe?

Note added at 7:27 a.m. on 9/2/15: See my next post for a clarification about causation and variation. 

The full text of the peer-reviewed JAMA Surgery paper is available here.

Tuesday, August 25, 2015

In 22% of kids with appendicitis, antibiotics do not prevent perforation

Those clambering aboard the "antibiotics for appendicitis" bandwagon should read this interesting paper about appendicitis in children.

A group of emergency physicians from Maimonides Medical Center in Brooklyn, New York found that "Increasing in-hospital time delay from ED presentation to OR appendectomy is associated with increased risk for developing appendicitis perforation in children who present with CT-documented uncomplicated appendicitis."

Children with simple appendicitis who were taken to the operating room longer than 9 hours from the time of ED presentation were much more likely to develop a perforation than those who had surgery in less than 9 hours.

During the four years of the study, 404 consecutive children ≤ 18 years of age had a CT scan diagnosis of acute appendicitis; 156 (38.6%) had evidence of perforation at the time of presentation and were not included in the final analysis.

Thursday, August 20, 2015

A medical student in Cuba is looking for advice

Someone writes: I am trying to help a friend's brother who is not a US citizen and currently a medical student in Cuba, and I came across your very informative web site. The brother most likely is going to be able to come to the United States in the fall.

My friend is wondering if he should complete the last year of medical school there in Cuba or come here and continue on. It seems like there is no benefit from completing med school in Cuba, given the difficulty to be licensed in the U.S. And the difficulty in getting a residency position.

Does any of the course work from his studies in Cuba transfer over to U.S? Is it likely that he'd have to get a bachelor's degree here before ever going to a U.S. Med school? My friend says that he has an outstanding record in the Cuban medical school, speaks excellent English, does well on tests, etc. Any advice you could give?

As far as I know, no medical students from Cuba have transferred to a med school in the United States recently or possibly ever. Regarding your questions, I can only give you my best guesses.

I doubt very much that a course from the Cuban medical school would be accepted here in the US. US med schools that accept a few transfers from Caribbean schools like Ross or St. George's usually take those students at the beginning of the third year of medical school.

A few schools are doing combined BS/MD degrees in five or six years, but I don't know of a single US school that would take a student directly out of high school into a 4-year program.

Tuesday, August 18, 2015

Male docs are more often involved in medicolegal actions than female docs

"Male doctors are more likely to have experienced medico-legal actions compared to female doctors. This finding is robust internationally, across outcomes of varying severity, and over time," concluded a recent meta-analysis.

The study, published online in BMC Medicine, said men were 2.45 (95 % CI, 2.05–2.93) times more likely to have been the subject of legal proceedings.

Legal action was defined as disciplinary action by a medical regulatory body, malpractice experience, complaints received by a medical regulatory or healthcare complaints body, a criminal case, or when a paper on the topic did not specify one of the above.

Data from 32 published papers were pooled and analyzed. At first glance, the methods seem reasonable, and the conclusion may even be correct.

But to their credit, the authors mention that the paper has some limitations which, in my opinion, probably invalidate the results.

Thursday, August 13, 2015

A “shallow water blackout” is a silent killer

In Jacksonville, Florida, a 50-year-old woman was found at the bottom of her backyard swimming pool. She was an experienced scuba diver who “often stayed at the bottom of the 9-foot deep end without oxygen to increase [her] lung capacity for future dives.”

Despite receiving CPR from her son, she could not be revived.

The Associated Press story about this tragic incident did not explain why a swimmer with her background drowned.

It appears to be a classic case of “shallow water blackout.” This phenomenon occurs when people hyperventilate before diving.

An increasing level of carbon dioxide (CO2) is what triggers the urge to breathe. Hyperventilating causes hypocapnia, a reduced amount of CO2 in the blood. If a swimmer uses up enough oxygen to pass out before the CO2 trigger point for breathing is reached, drowning will occur without notice. Victims are usually found at the bottom of the pool.

Here’s what it looks like in a diagram from Wikipedia:

A physician who lost her son to this little-known phenomenon started a website to heighten awareness of the problem. The site contains more information and stories about other drownings caused by shallow water blackouts.

Here is a video of a woman swimming laps of a pool underwater. Advance to the 0:50 point and watch what happens as she begins to slow down. [Addendum 8/13/15 12:50 pm: Warning. The video is graphic. It shows the unconscious swimmer being pulled from the water.]

A shallow water blackout may have been responsible for the death of Natalia Molchanova, the world’s foremost freediver, who went missing a few days ago.

Hyperventilating prior to diving is not recommended. Tell your friends.

Wednesday, August 12, 2015

Why in-hospital deaths are not a good quality measure

You may be tired of hearing about the Surgeon Scorecard—the surgeon rating system that was recently released by an organization called ProPublica. Like many others, I have pointed out some flaws in it. You can read my previous posts here and here.

I had decided to stop commenting about it because enough is enough, but a recent paper in the BMJ raises a question about one of the criteria ProPublica used to formulate its ratings.

ProPublica defined complications 1) as any patient readmission within 30 days and 2) "any patient deaths during the initial surgical stay."

The authors of the BMJ paper randomly selected 100 records of patients who died at each of 34 hospitals in the United Kingdom. The 3400 records were reviewed by experts to determine whether a death could have been avoided if the quality of care had been better.

The number of patient records in which a death was at least 50% likely to have been avoidable was 123 or 3.6%.

There was a very weak association between the number of preventable deaths and the overall number of deaths occurring at each hospital. By two measures of overall hospital deaths, the hospital standardized mortality ratio and the summary hospital level mortality indicator, the correlation coefficient between avoidable deaths and all deaths was 0.3, not statistically significant.

From the paper: "The absence of even a moderately strong association is a reflection of the small proportion of deaths (3.6%) judged likely to be avoidable and of the relatively small variation in avoidable death proportions between trusts [hospitals]. This confirms what others have demonstrated theoretically—that is, no matter how large the study the signal (avoidable deaths) to noise (all deaths) ratio means that detection of significant differences between trusts is unlikely."

The Surgeon Scorecard was derived from administrative data. No individual analysis of patient deaths was undertaken. According to a ProPublica article discussing some key questions about their methodology, "As for deaths, we took a conservative approach and only included those that occurred in the hospital within the initial stay."

Maybe that wasn't such a conservative approach after all.

And maybe we need to rethink that 2013 paper claiming that medical error caused up to 440,000 deaths per year.