Preventing Hospital Readmissions: Healthcare Providers’ Perspectives on “Impactibility” Beyond EHR 30-Day Readmission Risk Prediction

Predictive models based on electronic health records (EHRs) are used to identify patients at high risk for 30-day hospital readmission. However, these models’ ability to accurately detect who could benefit from inclusion in prevention interventions, also termed “perceived impactibility”, has yet to be realized.

Objective

We aimed to explore healthcare providers’ perspectives of patient characteristics associated with decisions about which patients should be referred to readmission prevention programs (RPPs) beyond the EHR preadmission readmission detection model (PREADM).

Design

This cross-sectional study employed a multi-source mixed-method design, combining EHR data with nurses’ and physicians’ self-reported surveys from 15 internal medicine units in three general hospitals in Israel between May 2016 and June 2017, using a mini-Delphi approach.

Participants

Nurses and physicians were asked to provide information about patients 65 years or older who were hospitalized at least one night.

Main Measures

We performed a decision-tree analysis to identify characteristics for consideration when deciding whether a patient should be included in an RPP.

Key Results

We collected 817 questionnaires on 435 patients. PREADM score and RPP inclusion were congruent in 65% of patients, whereas 19% had a high PREADM score but were not referred to an RPP, and 16% had a low-medium PREADM score but were referred to an RPP. The decision-tree analysis identified five patient characteristics that were statistically associated with RPP referral: high PREADM score, eligibility for a nursing home, having a condition not under control, need for social-services support, and need for special equipment at home.

Conclusions

Our study provides empirical evidence for the partial congruence between classifications of a high PREADM score and perceived impactibility. Findings emphasize the need for additional research to understand the extent to which combining EHR data with provider insights leads to better selection of patients for RPP inclusion.

Electronic supplementary material

The online version of this article (10.1007/s11606-020-05739-9) contains supplementary material, which is available to authorized users.

KEY WORDS: readmission prevention, impactibility, high-risk classification, electronic health records

INTRODUCTION

Healthcare organizations strive to reduce readmission rates through targeted readmission prevention programs (RPPs), including discharge planning, coupled with post-discharge follow-up, which have reported promising results. 1 – 3 Comprehensive care approaches that incorporate patient and caregiver engagement, education, and well-being promotion, as well as complexity management and promotion of continuity of care, have been identified as hallmarks of high-quality transitional care. 4 However, RPPs are resource-intensive, and given the need to efficiently allocate resources, identifying which patients would benefit most from RPPs is of growing interest. 5 To maximize efficiency, big data from electronic health records (EHRs) are increasingly used to identify patients at high-risk for 30-day hospital readmission for inclusion in RPPs 6 ; however, whether these models can accurately detect patients potentially benefiting from RPP inclusion, also termed “perceived impactibility”, has yet to be realized. 7

Patients who are perceived as “impactible” 7 – 9 or “care-sensitive” 10 have clinical and social characteristics that make their situation “modifiable”, “addressable”, or “intervenable” and are therefore candidates for RPPs. Impactibility indicates characteristics that may be key contributors to patients’ risk and to suitability for inclusion in RPPs, yet may not necessarily be detected by automated prediction scores. 9 For example, deteriorating health status increases the chance for readmission, but this does not imply that the deterioration is preventable, and therefore is not “intervenable” in terms of readmission prevention. 7 To better align advances in automated prediction models and clinical care, practitioners might use data on individual and social factors to improve high-risk-case identification in terms of prediction accuracy and impactibility. 9

To study the appropriate selection of high-risk patients to RPPs, we used a large EHR data warehouse from the largest integrated payer-provider healthcare organization in Israel, Clalit Health Services (CHS). CHS has developed and implemented the Preadmission Readmission Detection Model (PREADM), 11 which enables early identification of patients at high risk for 30-day hospital readmission to internal medicine units. The PREADM score guides physicians and nurses in prioritizing patients for inclusion in RPPs, tailored to meet their ongoing care needs. However, anecdotal evidence since the introduction of the program in 2012 has shown that inclusion in RPPs is determined not only by PREADM score but also by healthcare provider perspectives on patients’ clinical, functional, cognitive, and social characteristics. Moreover, some patients who are not detected as high risk for 30-day readmission according to their PREADM score are referred to RPPs based on healthcare providers’ perceptions. These considerations, however, are not uniform, and the specific characteristics that deem patients impactible are unclear.

The objective of this study was to explore healthcare providers’ perspectives about patients’ personal and clinical characteristics associated with the decision whether to refer to an RPP (operationalized as “perceived impactibility”), beyond the classification of the PREADM score.

METHODS

Settings

In Israel, all citizens receive healthcare coverage from one of four integrated payer-provider healthcare organizations. CHS, which serves over half of Israel’s population, operates eight general hospitals and over 1400 primary and specialty clinics nationwide. A broad range of RPPs are performed in CHS hospitals and primary care clinics, including employing transitional care nurses in hospitals who assess the patients’ physical, mental, and social conditions; developing tailored interventions in collaboration with patients and their healthcare providers in the hospital and primary care clinic settings; and conducting routine assessments of patients’ condition throughout hospitalization. This study was performed at three medium-to-large general CHS hospitals.

Study Design and Survey Development

This study employed a cross-sectional evaluation of data from a larger cohort study examining hospitals’ and primary care providers’ perspectives of patients’ social, functional, and cognitive characteristics and subsequent readmission risk (forthcoming). As part of this study, we developed a survey to understand the perspectives of healthcare providers based on risk factors for readmission and characteristics of patients included in RPPs. A three-round mini-Delphi method was used to design and develop the survey for healthcare provider perspectives on patient characteristics that might be associated with RPP referral decision-making. In the mini-Delphi method, field experts discuss a pool of items for potential inclusion in a survey and provide reasons and justifications including them. 12 – 15

The mini-Delphi method included the following: (1) three in-person workshops of 30 participants (10/group); (2) a face-to-face panel of two staff nurses and five nursing and medical managers reviewing the 30-item survey in a second round; and (3) a review of the 20 highest-agreement items. The final round resulted in the deletion of seven items deemed irrelevant or repetitive. All healthcare-provider participants were experts in RPPs and represented nursing or medical sectors from hospital and primary care settings, including four chief physicians and six head nurses from internal medicine units in various CHS hospitals, six nursing managers at different organizational levels (district or sub-district), eight primary care clinic nursing managers, and six transitional care nurses. This study was approved by the CHS Ethics Committee.

Study Population

Between May 2016 and June 2017, medical and nursing teams from 15 internal medicine units at three general CHS hospitals were surveyed. Nurses and internal medicine physicians (in charge of direct patient care) were asked for information about patients 65 years or older who were hospitalized in internal medicine wards at least one night. Data were collected by a trained research assistant, twice a week (during weekdays), and then typed and stored in a secure CHS database. Each day about 70–100 patients at each hospital from all internal medicine wards met study-inclusion criteria. Patients who met the inclusion criteria were selected sequentially from the randomized list in each ward until roughly 25% were included in the study. Physicians and nurses both indicated whether patients were or should have been referred to an RPP. Responses on the 13-item survey were collected from nurses only, as they were determined to be the key informants who were most knowledgeable about patient characteristics (according to the focus group discussions).

Dependent Variable

A patient’s referral (or intended referral) to an RPP was the dependent variable (from the survey) and was coded as 1 (= yes) if either the nurse or physician indicated that the patient was or should be referred to an RPP. Otherwise, the variable was coded 0 (= no) if both physician and nurse indicated a non-RPP referral.

Independent Variable

For each patient, the assigned nurse was asked to indicate whether the patient had a specific clinical, functional, cognitive, and/or social characteristic (“yes” = 1) or not (“no” = 0): for example, “Does the patient need support of social services?” or “Does the patient have low language proficiency?” See Appendix 1 for the final survey.

We obtained EHR data from the centralized CHS data warehouse, which stores complete demographic and administrative data for each CHS member, along with complete inpatient records. We extracted data on each patient’s PREADM score, which considers six chronic conditions (congestive heart failure [CHF], chronic renal failure, chronic obstructive pulmonary disease [COPD], malignancy, arrhythmia, disability), previous use of health services (number of hospital admissions, past-year primary care and specialist visits; days from last hospitalization), body mass index, and an indicator of the hospital’s catchment area. 11 A high PREADM score was defined as a score of 40 or higher, reflecting the risk status of 55% of the hospitalized population and a readmission rate of 30%. 11 Additionally, we extracted separate data on age and sex.

Statistical Analysis

Bivariate analyses, using Pearson’s χ 2 tests, were performed to assess differences in characteristics (according to the 13-item survey) between patients who were and were not referred (or were eligible for referral) to an RPP. To ultimately identify characteristics that should be considered in decisions for inclusion in RPPs, we performed a decision-tree analysis, one of the most widely used data-mining algorithms in clinical decision support systems 16 (with a conservative cutoff of < 0.1 for inclusion of variables), applying Classification And Regression Tree (CART) using the recursive partitioning and regression trees (rpart) package in R. We also conducted sensitivity analysis by developing a decision-tree model for patients with responses from both nurses’ and physicians’ surveys. All analyses were performed using R version 3.2.2.

RESULTS

We collected 817 questionnaires on 435 patients. Sixty-two patients died during their hospitalization; therefore, the final cohort included 373 patients with 704 healthcare provider surveys (373 from nurses, 331 from physicians who were only rating whether the patient should have been referred to an RPP).

About a quarter (28.7%) of the patients were 8 5+ years old, and 56% were female. The share of those with chronic diseases ranged from 22.8% with COPD to 41.3% with a disability. Just under two thirds (59.8%) had a high PREADM score (i.e. high-risk of readmission). For 212 (56.8%) patients, the physician or nurse indicated or recommended referral to an RPP. For 42 patients (11.3%), information on RPP referral was not provided by physicians. The characteristics of these 42 patients were statistically significantly similar to those of the 331 patients for whom both nurse and physician assessments were available, in terms of demographics and co-morbidity (Appendix 2 ). Our sample was similar to the overall sample of admitted patients in terms of age and comorbidities. 17

Table ​ Table1 1 shows that those referred to an RPP had higher rates of a high PREADM score (72.2% vs. 43.5%, P < 0.001), of CHF (35.4% vs. 23.6%, P = 0.019), and of disability (47.2% vs. 33.5%, P = 0.011). Only two of 13 patient characteristics identified by healthcare providers were not significantly related to the likelihood of an RPP referral (P > 0.1) (i.e. living alone and low language proficiency); therefore, these characteristics were excluded from the decision-tree analysis.

Table 1

Characteristics of Patients Referred and Not Referred to Readmission Prevention Program, Taken from the Survey of Hospital Teams

Abbreviations: RPP = readmission prevention program; EHR = electronic health records; PREADM = preadmission readmission detection model; CHF = congestive heart failure; CRF = chronic renal failure; COPD = chronic obstructive pulmonary disease

a Subjective assessment of the therapist as to whether the patient was stable during hospitalization

b Oxygen, cpap—continuous positive airways procure

c Informal caregiver: family member, close relation or formal paid caregiver

Table ​ Table2 2 demonstrates that having a high PREADM score is only partially congruent with high perceived impactibility. Of the total study population, 41% had a high PREADM score and were also referred to an RPP (high perceived impactibility), 19% had a high PREADM score but were not referred to an RPP (low-medium perceived impactibility), 16% had a low-medium PREADM score but were referred to an RPP (high perceived impactibility), and 24% had a low-medium PREADM score and were not referred to an RPP (low-medium perceived impactibility). Thus, PREADM score and RPP inclusion were congruent in 65% of patients.

Table 2

Congruence Between Preadmission Readmission Detection Model Risk Score and Perceived Impactibility for Readmission Prevention Program

Abbreviations: RPP = readmission prevention program; PREADM = preadmission readmission detection model

The decision-tree analysis identified five patient characteristics that were statistically significantly associated with RPP referral and subdivided all patients into six segments (nodes). The chance of being referred to an RPP varied from 17 to 75% (Fig. 1 ). The first question in the tree (highest importance), also called the root node, was “Does the patient have a high PREADM score?” In classification trees, positive or “yes” responses branch to the right. If “yes” (69% probability, 223 patients), the second question was “Is the patient eligible to be referred to a nursing home?” Patients who met these criteria were classified as suitable for RPP referral (terminal node 1, 75% probability, 150 patients). For patients with a high PREADM score but who were ineligible for nursing-home referral, the third question was “Are any of the patient’s diseases not under control?” (i.e. the patient’s chronic condition is not stable or above threshold levels). If “yes”, patients were classified as suitable for RPP referral (terminal node 2, 68% probability, 40 patients); if “no”, patients were classified as unsuitable for referral to RPPs (terminal node 3, 42% probability, 33 patients).

Decision-tree analysis for classification of referral patients for readmission prevention programs. RPP = readmission prevention program * Blue-shaded terminal nodes indicate that the tree classified patients as suitable for readmission prevention programs, and percentages reflect the probability of patients assessed for referral to readmission prevention programs. Terminal node numbering (1–9) is included in parentheses.

Among the 150 patients who had a low-medium PREADM score (question 1), the root node branched left; the next question (highest importance) was “Does the patient need support of social services?” If “yes”, patients were classified as suitable for RPP referral (terminal node 4, 67), with a 58% probability. Finally, for patients having a low-medium PREADM score and no need of social-services support, the third question was “Does the patient need special equipment at home?” If “yes”, patients were classified as suitable for referral to a RPP (terminal node 5, 11), with a 73% probability; if “no”, patients were classified as unsuitable for RPP referral (terminal node 6, 72), with a 17% probability.

Sensitivity analysis for 331 patients with responses from both nurses and physicians about RPP referral yielded similar results.

DISCUSSION

Our study provides empirical evidence of the partial mismatch between classification of patients at high risk for hospital readmission and perceived impactibility. PREADM score and RPP inclusion were congruent in 65% of patients, whereas 19% had a high PREADM score but were not referred to an RPP and 16% had a low-medium PREADM score but were referred to an RPP. The decision-tree analysis identified five patient characteristics that were statistically significantly related to the reason for patient RPP referral: high PREADM score, eligibility for nursing-home referral, having a condition not under control, need for social-services support, and need for special equipment at home.

The finding that about two thirds of patients were both high risk and perceived as impactible is congruent with other recent findings. 8 The remaining third were almost evenly classified between those who were high risk but perceived as not impactible and those who were perceived as impactible but not high risk. This supports the notion that for decisions about RPP referrals, identifying individuals at high risk for readmission is insufficient; it is also important to understand who would benefit from inclusion in RPPs. 7 – 10 The decision-tree analysis showed that the most important driver of the RPP inclusion decision was the PREADM score, justifying nurses’ and physicians’ initial use of the PREADM tool to select appropriate individuals for RPP inclusion. 18 , 19

This study also shows that patients with low-medium PREADM scores might need to be referred to an RPP. This finding expands the common focus of readmission-risk identification. While it is common practice to concentrate on identifying high-risk patients, our findings point to the value of considering RPP inclusion according to clinicians’ assessments even where predictive models fail to accurately identify high-risk patients. 7 Our results show that individuals with low-medium PREADM scores who needed social-services support or special equipment at home were highly likely to be referred to an RPP. Thus, healthcare provider assessments may be beneficial to incorporate into the current RPP decision-making process to consider inclusion of patients with low-medium PREADM scores.

The current study provides insight into the characteristics that hospital nurses and physicians consider when selecting patients as potential participants in RPPs beyond an automated prediction score. For example, non-nursing-home-eligible patients with controlled disease, despite having a high PREADM score, were unlikely to be referred to an RPP. Such a decision may be due to the low likelihood of those patients to benefit from existing care-management-based approaches to readmission prevention.

The decision-tree supervised learning methodology used in this study is rarely used in the readmission literature but may offer clinicians a practical, user-friendly output for addressing patient perceived impactibility. Decisions for RPP inclusion could be guided by two two-step approaches: first, among high-risk patients, nursing-home eligibility is determined, and then the degree to which the disease is under control is ascertained. Second, among non-high-risk patients, those who need social support are included in RPP, as are those who need special equipment at home.

Ultimately, linked EHR data and information from nurses and physicians can enable structured decision-making for care-management patient selection. Based on these characteristics, the future goal is to develop an on-site, user-friendly decision tool for choosing individuals perceived as impactible to refer to RPPs. Nonetheless, development of on-site user-friendly decision tool requires additional research to first establish the evidence for the decision criteria on which such a tool can lead to standardized RPP selection.

Strengths and Limitations

The main strength of our study was our multi-source and mixed-method design, which combined EHR data with self-reported surveys of nurses and physicians, along with a mini-Delphi approach. Mixed-method studies employing both quantitative and qualitative approaches allow for better understanding of research problems and complex phenomena than either approach alone 20 while reducing the threat to construct validity. 21 Thus, this combination enabled a comprehensive understanding of readmission prevention. Additionally, the multi-source component of our study reduced same-source bias, thus increasing our confidence in our conclusions. 22

An additional strength was the use of the decision-tree model. A decision tree is generally intuitive, and, rather than showing the factors that together compose the likelihood of an outcome (as in regression analysis), it presents the main characteristics that can be followed linearly to guide decisions such as those presented here. 16

Despite the current study’s strengths, some limitations should be noted. First, there is no gold standard for the RPP measure. Referral to an RPP aimed at readmission reduction, such as patient discharge counselling and follow-up, or involvement of a transitional care nurse, is commonly performed according to physicians’ or nurses’ recommendation. Accordingly, we used information from physicians or nurses, thus reflecting an all-inclusive approach encompassing both nursing and medical considerations. For the purposes of this study, provider referral for RPP implies perceived impactibility. However, to truly address the issue of impactibility, we need to know whether those selected for intervention are in fact “impactible” according to a set of criterions or standards. Future studies should develop such standards to assess the distinctive contribution of providers’ perceived impactibility.

Second, in the decision-tree analyses, the third branches included relatively small sample sizes, which can increase the likelihood of type II errors skewing the results within these sub-populations. Larger sample studies are needed to increase the power of future studies. Future studies should also consider expanding the target population to those younger than 65. Nonetheless, as much of the focus of current healthcare delivery systems is on better streamlining care for the growing, complex population of older adults, findings from this study are pertinent to other healthcare systems and populations.

Finally, although disentangling the effect of accurate identification and effectiveness of readmission prevention interventions in terms of readmission reduction rates is beyond the scope of this study, it is important to study perceived impactibility in the context of actual readmission reduction and improved clinical outcomes.

CONCLUSIONS

Our study provides empirical evidence for the partial congruence between classifications of a high PREADM score and perceived impactibility. The findings emphasize the need for additional research to understand the extent to which combining EHR data with provider insights leads to better selection of patients for inclusion in RPPs.