A study of the use of machine learning to predict the risk of dying within six months, combined with behavioural nudges, led to increased serious illness conversations between oncologists and cancer patients. “To our knowledge, this is one of the first randomized clinical trials testing the effect of real-time machine learning predictions and behavioural nudges on clinical care,” write the authors of the article (Manz CR et al. JAMA Oncol 2020).
Serious illness conversations are structured conversations between clinicians, patients, and families around prognosis, treatment goals, and end-of-life preferences (Fricker J. CancerWorld, 2020). While early discussions have been associated with less-aggressive care for cancer patients near the end of life, including increased hospice utilisation, and reduced chemotherapy and hospitalisation rates, most patients with cancer die without documented discussions about their goals and treatment preferences.
The study by Christopher Manz, from Dana-Farber Cancer Institute (Boston) and colleagues, set out to determine effects of integrating machine-learning mortality predictions with behavioural nudges on motivating serious illness conversations between clinicians and patients.
Overall, 78 oncology physicians from nine medical oncology clinics in the University of Pennsylvania Health System (who had all received training in serious illness conversations) were randomly assigned to control or intervention groups between June 17 and November 1 2019. The intervention consisted of three main components:
- Component 1: Oncology clinicians received weekly emails with performance feedback on their serious illness conversations (indicating the number they had performed in the prior 4 weeks) and peer comparisons.
- Component 2: With the email, oncology clinicians received a link to a secure dashboard where they could review up to six high-risk patients scheduled for the following week estimated to have high risk of mortality (≥10% predicted risk of 180-day mortality). Estimations used a validated machine learning algorithm that had been demonstrated to accurately predict 180-day mortality risk (Parikh RB et al. JAMA Netw. Open 2019).
- Component 3: Oncology clinicians received a text message (which they could opt out of) on the morning of appointments with high-risk patients, reminding them to consider having a serious illness conversation.
The clinics included eight subspeciality clinics (breast, central nervous system, gastrointestinal, genitourinary, lymphoma, melanoma, myeloma, and thoracic or head and neck) and one general oncology clinic. The study involved 14,607 patients who had 26,059 physician encounters. It used a ‘stepped wedge cluster’ randomised trial design, involving random and sequential crossover of clusters from control to intervention groups, until all clusters were exposed. Mortality predictions were updated weekly, with the result that patients assigned to the ‘high-risk’ group could change between encounters.
Results for all encounters showed that serious illness conversations were conducted for 1.3% of patients in the control group versus 4.6% in the intervention group (adjusted difference in percentage points 3.3; 95%CI 2.3−4.5; P<0.001). Among the 4,124 high-risk patient encounters, serious illness conversationss were conducted for 3.6% of patients in the control group versus 15.2% in the intervention group (adjusted difference in percentage points 11; 95%CI 8.2−12.15; P<0.001).
“Behavioral nudges combined with machine learning mortality predictions can positively influence clinician behavior and may be applied more broadly to improve care near the end of life,” conclude the authors, adding that longer-term studies are needed to assess whether algorithm-based interventions result in improved clinical outcomes for patients.
A key innovation of the study, they write, was integrating principles of behavioural economics into the machine-learning-based intervention, including peer comparisons, performance feedback, and opt-out reminders, which have been used successfully in other clinical settings to change clinician behaviour.
Although the intervention effect was strongest for high-risk patients flagged by the algorithm, the study still detected an increase in serious illness conversations among patients not deemed to be at high risk of dying within six months. “This off-target effect may have occurred because the clinicians who received the intervention became accustomed to actively considering mortality risk in their clinical practice, resulting in an increase in SICs [serious illness conversations] for patients who may have a poor prognosis but were not flagged by the algorithm as high risk,” write the authors.
Commenting on the study to Cancer World, Stephanie Harman, from Stanford Health Care, in Stanford, California, says, “What’s novel about this study is introducing behavioural nudges into the routine workflow to increase the number of SICs. Nudges have previously been shown to be effective in health care quality improvements.”
Strengths of the study, she adds, include input from primary frontline oncologists, and the general oncology population (not just high risk patients), underscoring the effectiveness of large-scale implementation. “But the algorithm may not be replicable in other health-care systems if they do not have the same level of biomedical informatics support, expertise, or cannot afford the approach,” she says.
Illustration: Maddalena Carrai
