BROWSE CASE STUDIES Case Studies > Extended-age Out-of-Sample Validation of Risk Stratification Index 3.0 Models Using Commercial All-payer Claim

Extended-age Out-of-Sample Validation of Risk Stratification Index 3.0 Models Using Commercial All-payer Claim

December 2022


Consumers, Researchers, Population Health


Utilization, Health, Medicare


Daniel I. Sessler, M.D., Michael Cudahy Professor and Chair

We previously reported a broad suite of individualized Risk Stratification Index 3.0 models for various meaningful outcomes in patients admitted to a hospital for medical or surgical reasons. The models used ICD-10 trajectories, and were restricted to information available at hospital admission, including coding history in the previous year. The models were developed and validated in Medicare patients, mostly ≥65 years old. We sought to determine how well our models predict utilization outcomes and adverse events in younger and healthier populations.

Our analysis was based on All Payer Claims for surgical and medical hospital admissions from Utah and Oregon. Endpoints included unplanned hospital admissions, in-hospital mortality, acute kidney injury, sepsis, pneumonia, respiratory failure, and a composite of major cardiac complications. We prospectively applied previously developed Risk Stratification Index 3.0 models to the younger and healthier 2017 Utah and Oregon state populations and compared the results to our previous out-of-sample Medicare validation analysis.

In the Utah dataset, there were 55,109 All Payer Claims admissions across 40,710 patients. In the Oregon dataset, there were 21,213 admissions from 16,951 patients. Model performance on the two state datasets was similar or better than in Medicare patients, with an average area under the curve of 0.83 (range 0.71 to 0.91). Model calibration was reasonable with an R2 of 0.93 (range 0.84, 0.97) for Utah and 0.85 (0.71, 0.91) for Oregon. The mean sensitivity for the highest 5% risk population was 28% (range 17, 44) for Utah and 37 (20, 56) for Oregon.

Predictive analytical modeling based on administrative claims history provides individualized risk profiles at hospital admission that may help guide patient management. Similar predictive performance in Medicare and in younger and healthier populations indicates that Risk Stratification Index 3.0 models are valid across a broad range of adult hospital admissions.