Research paperA prognostic index (PI) as a moderator of outcomes in the treatment of depression: A proof of concept combining multiple variables to inform risk-stratified stepped care models
Introduction
There are a wide variety of treatments for depression. These treatments differ in how intense they are in terms of investment and resources required from patients and providers but the average superiority of higher-intensity treatments relative to lower-intensity ones appears to be small. For example, the combination of antidepressants and psychotherapy appears to be superior to either treatment as a monotherapy but that difference is small (Khan et al., 2012). Similarly, the superiority of antidepressants relative to placebos appears limited (Kirsch and Sapirstein, 1998) and the superiority of evidence-based psychotherapy relative to treatment as usual appears to be small as well (Flückiger et al., 2014, Wampold et al., 2011). Despite the availability and approximate equivalence of many treatments for depression, current models for the delivery of care are inadequate. At one end of the spectrum, many patients receive higher intensity interventions than they require to experience symptom relief (Lorenzo-Luaces et al., 2015, Lovell and Richards, 2000). Conversely, many do not receive the level of care that they might require to experience symptom relief (Kocsis et al., 2008, Lecrubier, 2007).
It is difficult to match patients to an appropriate level of care because depression is extremely heterogeneous in presentation and prognosis (Lorenzo-Luaces, 2015, Parker, 2005). We describe an approach, conceptually motivated by research on risk-stratified care in breast cancer (Akay et al., 2012, Chen et al., 2004; Chen et al., 2005; Huang et al., 2006), for combining variables to create a prognostic index (PI) that can be used when selecting between treatments that differ in intensity. PIs can be thought of as predictive of patients’ symptom status in a future time frame and they can also be used to determine the intensity level of care a patient should receive (see Delgadillo et al. (2016) and Garber et al. (2016)).
In the treatment of breast cancer after neoadjuvant chemotherapy, mastectomies are known to be a more aggressive treatment than breast-conservation therapy (BCT), which preserves a part of the breast tissue for cosmetic purposes. While these treatments clearly differ in intensity, on average there are only small (1–9%) differences in 5–10 year recurrence rates between the two treatments (Morris et al., 1997, van Dongen et al., 2000). Although this average difference is small, it is possible that BCT might be indicated for patients with less aggressive cancers whereas mastectomies, the more aggressive or higher intensity treatment, are better-suited for patients with more aggressive illnesses. This possibility was explored by Huang et al. (2006) who analyzed treatment differences according to a previously-developed PI (Chen et al., 2005). Their PI consisted of a score, ranging from 0 to 4, which indicated the number of risk factors for a cancer recurrence (see Chen et al., 2004). For patients with a low predicted risk of recurrence, recurrence rates were low and did not differ between the treatments (12% for BCT, 9% for mastectomy). For patients with a score of 3–4, however, recurrence rates were significantly higher for those treated with BCT (61%) than for those treated with mastectomy (19%). This pattern of results were subsequently replicated by Akay et al. (2012) who reported no differences between BCT and mastectomy for patients who were at a low risk of recurrence but a large difference in favor of mastectomy (6% vs. 32%) for patients with a high risk score. We propose that a similar phenomenon occurs in the treatment of depression and other disorders. Specifically, comparisons of treatments that appear to differ in intensity (e.g., combination treatment vs. psychotherapy alone or long term therapy vs. brief interventions) may produce small differences because many patients in treatment trials can be expected to benefit from minor interventions and it is only patients with an overall poorer prognosis that will evidence more benefit from the higher-intensity treatment.
The idea of using baseline variables to predict response to depression treatments is not new. For example, the efficacy of antidepressants relative to placebos seems to be limited to patients with more severe depressions (Barbui et al., 2011, Fournier et al., 2010, Khan et al., 2002, Kirsch et al., 2008). Similar findings have been reported for psychotherapy (Driessen et al., 2010). Likewise, the efficacy of combination treatment relative to antidepressants alone or psychotherapy alone seems to be limited to patients with more severe depression (Hollon et al., 2014, Thase et al., 1997). Some stepped models of care for depression make use of findings such as these. In stepped models of care, most patients are started on lower-intensity treatment options before entering more intense care. Illness severity, however, can be used as an indicator to bypass lower intensity treatments for more intensive ones. For example, the NICE guidelines in the United Kingdom (National Institute for Clinical Excellence, 2004) do not endorse the use of combined treatment as the initial treatment strategy in mild depression and instead suggest further assessment, a low-intensity intervention, or a monotherapy.
Research suggests that a multitude of variables, other than symptom severity, account for treatment response (see Kessler et al. (2016a)). For example, in a large dataset pooling comparisons of antidepressants vs. placebos, Nelson et al. (2013) reported that there were no statistically significant differences in outcomes for patients whose depression was non-chronic. However, a large difference (d=0.70) emerged for the subset of patients who had chronic and severe depression. Similarly, Thase et al. (1997) reported that, for individuals with mild depression, combining an antidepressant and cognitive-behavioral therapy (CBT) or interpersonal psychotherapy (IPT) was not superior to psychotherapy alone. However, for patients with severe and recurrent depression, combination treatment yielded superior recovery rates (60% vs. 19%; see also Hollon et al., 2014). Kessler et al. (2016a) conducted a review of patient self-reported variables that have been replicated at least once as predictors or moderators of treatment outcomes. According to these authors, predictors of depression can broadly be grouped into demographics (e.g., age, unemployment), features of depression (e.g., severity, prior episodes), co-morbidities (e.g., anxiety, sleep problems), stress history (e.g., childhood maltreatment), personality features (e.g., high negative affect), and other features (e.g., impairment). Broadly speaking, it appears as if the variables that predict overall depression treatment response are variables that predict the persistence and severity of depression. These are usually variables that relate to underlying vulnerabilities to depression and social-interpersonal functioning. However, an issue that obscures the interpretation of the existing literature is that most studies only explore single variables (DeRubeis et al., 2014a; Kessler et al., 2016b).
Kraemer (2013) asserted that “if there are multiple [moderators] related to the same underlying construct, these … should be combined in order both to increase the reliability of the measurement of that construct and to avoid problems associated with multicollinearity in combining them.” (p. 1969) DeRubeis et al. (2014b) argued for the existence of one such construct when they discussed patient response profiles. According to these authors, patients differ in the extent to which they can benefit from the active effects of treatments and processes. Patients who are likely to improve much irrespective of interventions, as is characteristic of samples of patients with depression, are unlikely to reveal specific intervention effects. Preliminary evidence supports the use of combined moderator variables like PIs in guiding treatment selection in mental health treatment (see Cloitre et al., 2016; Delgadillo et al., 2016). In the context of a clinical trial in which there were no differences in overall outcome between treatment as usual (TAU), stepped care starting with brief therapy (BT) or stepped care starting with CBT, we hypothesized that prognostic status would moderate the treatment differences. Our study is meant as a proof of concept in the treatment of depression. We hypothesize that among patients who, based on pre-treatment characteristics, are predicted to do well, few if any differences in outcome will emerge between the treatments we studied. However, among patients with a poorer prognosis, the more intensive CBT should outperform TAU and BT.
Section snippets
Methods
The aim of the trial from which these data were drawn was to compare TAU to each of two stepped care regimens. One of the regimens began with a low-intensity treatment (i.e., BT) and the other began with a high-intensity treatment (i.e., CBT; Van Straten et al., 2006).
The trial was designed so as to mimic conditions found in routine care settings. Patients were sampled from a representative subsample of 7 of the 47 regional mental health care centers (MHCs) that provide mental health care in
2.1 Outcomes and missing data
Patients were interviewed at baseline and then every 3 months, irrespective of the timing of treatment initiation and termination. The primary outcome for the current analyses was recovery, defined by the absence of MDD status, at the 18–24 month follow-up. This final follow-up interview occurred at least 18 months after enrollment in the study. The first 59 patients who entered the study were followed for 24 months; subsequent enrollees were followed for 21 months (n=105) or 18 months (n=256).
2.2 Analytic approach
Analyses were conducted using the R programming language (R Core Team, 2014). A total of 23 variables were available for analysis. These included demographics, clinical variables, personality traits as assessed by the NEO Five-Factor Inventory (Costa and MacCrae, 1992), and subscales of the Symptom Checklist 90 (Derogatis, 1983; Table 1). In choosing which variables to explore as predictors of treatment outcomes, we cross-referenced a recent review by Kessler et al. (2016a) on predictors of
Results
Descriptive statistics at baseline are presented in Table 1. There were no statistically significant differences in the rate of MDD recovery at the 18–24 month follow-up between TAU (68.4%) and either of the stepped care conditions (BT=75.4%, OR=1.33, 95% CI=0.84 –2.07, B =0.28, SE=0.23, χ2=1.47, p=0.23; CBT=74.5%, OR=1.38, 95% CI=0.91 –2.11, B =0.32, SE=0.22, χ2=2.25, p=0.13). Five of the 13 potential predictor variables submitted to the LASSO procedure were retained. Being unemployed, having
Discussion
We described a procedure that yields a prognostic index that can be used in determining which patients are most likely to benefit more from a high-intensity intervention, relative to a lower intensity one. We tested this procedure in the context of a randomized trial comparing TAU, BT, and CBT. Despite the trial being adequately powered to detect treatment differences and the fact that the treatments differed in intensity, on average, there were only small, nonsignificant differences in
References (69)
- et al.
Enduring effects of Preventive Cognitive Therapy in adults remitted from recurrent depression: A 10 year follow-up of a randomized controlled trial
J. Affect. Disord.
(2015) - et al.
The effects of psychotherapies for major depression in adults on remission, recovery and improvement: a meta-analysis
J. Affect. Disord.
(2014) - et al.
Different people respond differently to therapy: a demonstration using patient profiling and risk stratification
Behav. Res. Ther.
(2016) - et al.
Enduring effects of evidence-based psychotherapies in acute depression and anxiety disorders versus treatment as usual at follow-up—a longitudinal meta-analysis
Clin. Psychol. Rev.
(2014) - et al.
Comparison of risk of local-regional recurrence after mastectomy or breast conservation therapy for patients treated with neoadjuvant chemotherapy and radiation stratified according to a prognostic index score
Int. J. Radiat. Oncol. Biol. Phys.
(2006) - et al.
Acute phase cognitive therapy for recurrent major depressive disorder: who drops out and how much do patient skills influence response?
Behav. Res. Ther.
(2013) - et al.
Chronic forms of major depression are still undertreated in the 21st century: systematic assessment of 801 patients presenting for treatment
J. Affect. Disord.
(2008) - et al.
Evidence-based treatments for depression and anxiety versus treatment-as-usual: A meta-analysis of direct comparisons
Clin. Psychol. Rev.
(2011) - et al.
Evaluation of the MD Anderson Prognostic Index for local-regional recurrence after breast conserving therapy in patients receiving neoadjuvant chemotherapy
Ann. Surg. Oncol.
(2012) Diagnostic and Statistical Manual Of Mental Disorders
(2000)
Empirically derived decision trees for the treatment of late-life depression
Am. J. Psychiatry
Efficacy of antidepressants and benzodiazepines in minor depression: systematic review and meta-analysis
Br. J. Psychiatry
Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis
PLoS Med.
Are the parts as good as the whole? A meta-analysis of component treatment studies
J. Consult. Clin. Psychol.
Breast conservation after neoadjuvant chemotherapy
Cancer
Breast conservation after neoadjuvant chemotherapy: the MD Anderson cancer center experience
J. Clin. Oncol.
Patient characteristics as a moderator of post-traumatic stress disorder treatment outcome: combining symptom burden and strengths
Br. J. Psychiatry Open
Revised NEO Personality Inventory (NEO PI-R) and NEO Five-factor Inventory (NEO FFI): Professional Manual
Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies
J. Consult. Clin. Psychol.
Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies
Psychol. Med.
Self-guided psychological treatment for depressive symptoms: a meta-analysis
PLoS One
Symptom Checklist-90-R Administration, Scoring and Procedures Manual II
Understanding processes of change: how some patients reveal more than others – and some groups of therapists less – about what matters in psychotherapy
Psychother. Res.
The Personalized advantage index: translating research on prediction into individualized treatment recommendations. A demonstration
PLoS One
Temporal profiles of the course of depression during treatment: predictors of pathways toward recovery in the elderly
Arch. Gen. Psychiatry
Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression
J. Consult. Clin. Psychol.
Does pretreatment severity moderate the efficacy of psychological treatment of adult outpatient depression? A meta-analysis
J. Consult. Clin. Psychol.
Differential efficacy of cognitive behavioral therapy and psychodynamic therapy for major depression: a study of prescriptive factors
Psychol. Med.
Initial severity and differential treatment outcome in the National Institute of Mental Health Treatment of Depression Collaborative Research Program
J. Consult. Clin. Psychol.
Antidepressant drug effects and depression severity
JAMA
Prevention of depression in at-risk adolescents: moderators of long-term response
Prev. Sci.
Detecting anxiety and depression in general medical settings
BMJ
Improved variable selection algorithm using a LASSO-type penalty, with an application to assessing Hepatitis B infection relevant factors in community residents
PLoS ONE
Regression Modeling Strategies: with Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis
Cited by (57)
Artificial intelligence, machine learning and mental health
2023, Encyclopedia of Mental Health, Third Edition: Volume 1-3Perspectives on the course of major depression: Trajectories of remission and relapse with CBT or no treatment
2022, Journal of Behavioral and Cognitive TherapyCitation Excerpt :Further, prediction is made more difficult by findings that even the frequently assessed variable of depression severity may be too general to capture important individual symptom dimensions for predicting treatment outcome (Fried & Nesse, 2015). Yet, there is renewed hope for predictive algorithms that utilize many individual variables and sophisticated analytic models to generate patient profiles for prediction (Kessler et al., 2017; Lorenzo-Luaces et al., 2017; Webb et al., 2020). The goal behind such predictive models is to be able to provide personalized medicine for initial depression treatment selection (e.g., DeRubeis et al., 2014).
Data-Informed Advances and Technology Augmentation
2022, Cognitive and Behavioral PracticeCitation Excerpt :Research on pretherapy recommendations is focused on the development of empirically based decision rules for the selection of psychological interventions derived from statistical and/or machine learning algorithms using archived or prospective data at the beginning of therapy (Chekroud et al., 2021; Cohen et al., 2021; Cohen & DeRubeis, 2018). The goal is to choose an algorithm that allows the prediction of the optimal treatment package (e.g., cognitive-behavioral vs. interpersonal therapy), module (e.g., motivation enhancement vs. cognitive appraisal), processes or strategies (e.g., problem-solving vs. motivation-oriented) with respect to each patient’s set of intake characteristics (e.g., Deisenhofer et al., 2018; Fisher et al., 2019; Gómez Penedo et al. 2022; Huibers et al., 2015; Lorenzo-Luaces et al., 2017; Rubel et al., 2020; Webb et al., 2019). Furthermore, first studies have begun to cross-validate treatment selection models using independent holdout or patient per therapist data (e.g., Delgadillo et al., 2020; Schwartz et al. 2021), data from independent studies (van Bronswijk et al., 2021), or to test these models prospectively (Delgadillo et al., 2020; Lutz, Deisenhofer, et al., 2021).
Methods of Clinical Psychology
2022, Comprehensive Clinical Psychology, Second EditionThe 9-year clinical course of depressive and anxiety disorders: New NESDA findings
2021, Journal of Affective DisordersCitation Excerpt :Longitudinal research is essential to validate diagnostic classifications and tailor treatment plans for optimal effectiveness and efficiency for patients with affective disorders (Gillis et al., 1995; Kendell and Jablensky, 2003; Kraepelin, 1921; Lorenzo-Luaces et al., 2017; McGorry et al., 2016; Penninx et al., 2011).
A personalized index to inform selection of a trauma-focused or non-trauma-focused treatment for PTSD
2021, Behaviour Research and TherapyCitation Excerpt :We conducted linear regressions predicting outcomes at follow-up with variables representing treatment condition, the PI (controlling for baseline symptoms by its inclusion in the model from which the PI is derived), and the interaction between the PI and treatment condition. If the test associated with the coefficient for the interaction between the PI and treatment is statistically significant, it is an indication that outcomes vary between the treatments across different values of the PI (Lorenzo-Luaces et al., 2017). We used the Johnson-Neyman technique (Johnson & Neyman, 1936) to determine the cut-point on the PI at which the difference between PE and PCT became significant, using a modified version of the MODPROBE macro (Hayes & Matthes, 2009).