Same patient mortality rates for experienced and new surgeons, study shows

Because surgical training was radically changed in recent years–including a reduction of six to 12 months of training time–and other factors, the research team said further study will be needed to ensure the findings generalize.

Researchers involved in this study are Dr. Rachel R. Kelz, associate professor of surgery; Dr. Jeffrey Silber, professor of pediatrics, anesthesiology and critical care; Paul Rosenbaum, professor of statistics; and Sam Pimentel, a doctoral student in statistics–all of the University of Pennsylvania.

The study recently was published on the website of the Journal of the American Statistical Association, a publication of the nation’s oldest and largest organization of professional statisticians.

For the study, the research team accessed a Medicare database from several states covering a period of two years to create two matched groups of surgical patients. The data consisted of 130,106 patients who were operated on in 498 hospitals. The two groups created through the matching process consisted of 6,260 surgical patients each and were similar in 2.9 million categories. For example, each group had the same number of patients who received each surgical procedure (i.e., knee replacement, appendectomy). Additionally, among patients who received a particular procedure, the numbers of patients who had a particular pre-existing condition (i.e., diabetes, past stroke, and high blood pressure) also were as close as possible across the two groups. “Matching” patients in this way makes it easier to compare the clinical outcomes of the two groups.

Following its analysis of the data for each patient group, the researchers found that the mortality rate for patients of experienced surgeons was 3.59% (225 deaths out of 6,260 surgeries), while the mortality rate of new surgeons was 3.71% (232 deaths out of 6,260 surgeries).

“It is reassuring that new surgeons were able to achieve similar mortality rates to experienced surgeons when caring for similar patients. However, mortality is a relatively rare event that may not expose the benefits of experience,” said Dr. Kelz. “Therefore, future studies focused on additional outcomes are needed to ensure that new surgeon training and transition to independent practice are appropriately structured to meet the surgical needs of the public.”

Among the factors that require further analysis are the following:

• First, because new surgeons trained under the shortened training model only recently have entered the nation’s health care system, further analysis will determine if the findings will generalize to patient outcomes of experienced and new surgeons following the reduction in the training time.
• Also, the research team compared outcomes across multiple surgical specialties to demonstrate the feasibility of the new statistical methodology. More patients are needed to examine clinically relevant subsets of patients and to examine additional important outcomes such as post-operative complications and health care costs.
• Last, the pilot data used for the analysis was limited and thus narrows the generalizability of the study’s findings. The researchers have applied for funding to run a broader analysis to see if the patient outcomes of new and experienced surgeons do, in fact, generalize.

“Mandates changing professional education often are implemented without a method to investigate the downstream effects on the population health. This new methodology can be used in a larger study to examine the important question of whether new and experienced surgeons achieve equivalent outcomes following the mandated changes in the training paradigm. Additionally, the methodological framework we establish can be used to monitor the effects of subsequent changes to the health care workforce on patient outcomes across a variety of practice settings and clinical conditions,” wrote the researchers.

The new statistical methodology used in this study–called “large, sparse optimal matching”–was developed because patients typically are assigned to surgeons in a nonrandom manner. As a result, there may be systematic differences between the patients of recent surgical graduates and experienced surgeons, making measurement of surgeon effects difficult. For instance, within the pilot data, newly trained surgeons tended to operate on a different mix of patients than experienced surgeons. The patients of new surgeons more often were admitted via the emergency room and had higher risk factors on average. Other differences include the severity of a patient’s illness and risk of mortality across patients.

In the study, the statistical matching method accounted for these discrepancies and allowed researchers to make fairer comparisons between similar groups of patients. Once the initial discrepancies were removed, researchers no longer observed any significant mortality difference between the two groups.

The new techniques developed for this study have tremendous promise to enable large-scale comparisons across a variety of health care settings and populations with appropriate control for differences in patient cohorts that was not previously possible.

“The new methodology will make it much easier to create good matches from large medical data sets,” said Pimentel. “Large, sparse optimal matching will be useful in contexts outside of clinical medicine, too, including education research and public health, and I’m excited to see what other scientific questions it can help answer.”

Story Source:

The above story is based on materials provided by American Statistical Association. Note: Materials may be edited for content and length.

Journal Reference:

1. Samuel D. Pimentel, Rachel R. Kelz, Jeffrey H. Silber, Paul R. Rosenbaum. Large, Sparse Optimal Matching with Refined Covariate Balance in an Observational Study of the Health Outcomes Produced by New Surgeons. Journal of the American Statistical Association, 2015; 0 DOI: 10.1080/01621459.2014.997879