Research paperIdentifying relapse prevention elements during psychological treatment of depression: Development of an observer-based rating instrument
Introduction
Psychotherapy research on the effective prevention of relapse and recurrence after acute depression treatments has mostly investigated cognitive behavioral interventions, both alone and in combination with antidepressants (Bockting et al., 2005, Vittengl et al., 2007, Jarrett et al., 2013). However, after a terminated acute phase of cognitive behavioral therapy (CBT), relapse/recurrence rates were found to be 29% in the first year and 54% in the second year (Vittengl et al., 2007). Even with continued/maintained psychotherapy or pharmacotherapy, relapse and recurrence rates are still found to be high (Vittengl et al., 2007). Considering generally high relapse rates in depression, it is essential to identify potentially underlying processes of relapse prevention, followed by an investigation of their effectiveness and related mechanisms of action. Although treatment manuals recommend a variety of relapse-prevention elements, available measures assessing therapists’ treatment adherence and competence only marginally target relapse-preventive aspects beyond core cognitive treatment elements (e.g., CT techniques).
Three types of psychological interventions target the prevention of recurrence of depressive symptoms (Bockting et al., 2015): 1. Interventions during acute treatment aiming to maintain reduced symptoms also beyond termination. 2. Continuation and/or 3. Maintenance treatment, both provided after terminating acute treatment. Whilst continuation treatments are provided to currently remitted patients or to patients that previously responded to treatment, maintenance treatments are given during recovery defined as remission lasting longer than six months (Frank et al., 1991, British Psychological Society, 2010). All three types of interventions intend to prevent patients from experiencing relapse (return of depressive symptoms before full remission has been achieved) or recurrence (appearance of another new episode of depression after full remission) (Frank et al., 1991).
Established treatment manuals for cognitive and interpersonal therapies (Ellis and Dryden, 1997, Hollon et al., 2002, Beck et al., 2010, Schramm and Berger, 2010, Hautzinger, 2013) include specific recommendations for relapse prevention techniques being mostly assigned to the final phase of therapy. More specifically, elements to rehearse in this treatment phase include: early detection of depressive symptoms, anticipating critical situations and adequate coping skills, maintaining antidepressant activities, reinforcing helpful cognitions, planning the future, sensitizing the patient to potential relapses, and preparing transition from therapy to time after therapy termination. Although a variety of relapse prevention strategies seem to be recommended and commonly used in clinical practice, instruments measuring the quantity and quality of specific relapse-prevention elements have not yet been developed. More specifically, available scales assessing adherence and competence during psychotherapy do not fully cover the adequate implementation of relapse-prevention techniques beyond core treatment elements (e.g., central cognitions in CT).
In psychotherapy research, treatment integrity is defined by adherence and competence (Waltz et al., 1993). Adherence is defined as the degree a therapist provides interventions as determined in the treatment manual, whereas competence is defined as the extent to which a therapist implements such techniques in a skillful manner. A variety of reliable and validated instruments assessing therapist's adherence and competence during psychotherapy are available.
The most influential adherence scale is the Collaborative Study Psychotherapy Rating Scale (CSPRS; Hollon et al., 1988), developed to assess treatment integrity in CBT, interpersonal therapy, and psychiatric clinical management, including also particular CBT techniques (e.g., recognizing cognitive errors) observed in-session (Keefe et al., 2016). Referring specifically to relapse prevention after terminated acute treatment in individual setting, only one scale is available: The Cognitive-Behavioral Maintenance Therapy—Adherence Scale (CBMT-AS) was developed in the context of cognitive-behavioral maintenance therapy to assess therapists’ adherence to the manual of relapse-prevention therapy for recurrent depressive disorder (Weck et al., 2011b). Besides global evaluations (“management of time”) and cognitive-behavioral contents (“Encouragement of self-monitoring”), also one “relapse prevention” item is included. However, this single item was not used in further analyses because it was considered to be insufficiently assessable during therapy sessions (Weck et al., 2011b). The most influential competence scale is the validated and frequently applied Cognitive Therapy Scale (CTS; Weck et al., 2010). It is an observer-based rating instrument containing items on general competencies (e.g., use of feedback and summaries), specific competencies (e.g., focus on central cognitions), and one global item (overall rating of competence). Evidence for associations of competence and adherence measures with clinical outcomes in cognitive treatments for depression is equivocal. The meta-analysis of Webb et al. (2010) found no significant associations, which the authors attributed to the variety of applied methods (rating instrument, number of rated sessions, etc.), or to little influence of therapists’ adherence and competence on patients’ symptom change.
The mentioned instruments target relapse-preventive elements only to a marginal degree, i.e., items such as ‘Encouragement of self-monitoring’ or ‘relapse prevention’ (e.g., CBMT-AS; Weck et al., 2011b), and these instruments focus rather on the application of certain behaviors during treatment than preparing for their application after discontinuing treatment. Thus, there is a need for an instrument assessing specific relapse-preventive elements initiated by the therapist which focus on the patient's implementation of strategies after termination of treatment, such as anticipating critical situations or sensitizing the patient to potential relapses and recurrences in the long-term.
Aim of the present study was to create a rating instrument to systematically assess relapse-preventive elements during CBT for depression. For this, we developed the KERI-D (Kodierbogen zur Erfassung Rückfallprophylaktischer Interventionen bei Depression/Coding System to Assess Interventions of Relapse Prevention in Depression), an observer-based rating instrument to be applied in videotaped psychotherapy sessions during acute and continued/maintained depression treatment. The KERI-D assesses relapse-preventive elements that include both quantitative and qualitative aspects of the related psychotherapeutic process. The main objectives were to 1) assess the reliability of the KERI-D on an item level, 2) determine whether the KERI-D is able to identify relapse-preventive interventions occurring during psychotherapy, 3) explore the factorial structure of content items, 4) investigate associations with clinical outcome data, and 5) assess content validity of the KERI-D as measured by expert ratings.
Section snippets
Developmental steps
The KERI-D was developed by means of the following iterative steps: First, literature was screened for information on relapse-preventive strategies recommended in manuals for clinical practitioners (e.g., Jarrett, 1989; Hollon et al., 2002). Existing treatment integrity scales were screened (e.g., Evans et al., 1984; Hollon et al., 1988; Barber and Crits-Christoph, 1996; Weck et al., 2011b). In addition, experienced psychotherapists were interviewed on how they define relapse prevention and how
Reliability of the KERI-D
Table 1 shows the median ICC across all three pairs for each pair of raters. The median ICCs ranged from 0 to .95, with a mean of .69 and a median of .80. The items varied substantially in terms of inter-rater reliability, particularly for the items that occurred infrequently (see Table 3). Table 2 shows retest reliabilities for each rater and the median score over all three raters. Median ICCs ranged from 0 to 1, with a mean of .85 and a median of .93. We did not determine a median score for
Discussion
Aim of this study was to identify a comprehensive collection of explicit relapse-preventive elements during psychological treatment of depression and, on this basis, to develop an observer-based rating instrument called the KERI-D (Kodierbogen zur Erfassung Rückfallprophylaktischer Interventionen bei Depression/Coding System to Assess Interventions of Relapse Prevention in Depression). 17 of 20 items showed moderate to good inter-rater reliability, as well as moderate to very good stabilities
Conclusion
To our knowledge, this is the first instrument that measures specific relapse-prevention elements in psychotherapy by observer ratings. Considering the high relapse rates of formerly depressed patients after a successful acute treatment, an effective relapse prevention is important to maintain gains of therapy. The development of the KERI-D is a first step in defining specific relapse-prevention elements during psychotherapy against depression by providing an instrument that assesses the
Acknowledgements
We thank Mila Urschbach, Lea Bischof, Sira Löpfe, Patrick Wetzel and Lucy Lange for their support in developing and optimizing the rating scheme as well as coding the videotaped therapy sessions, and Elisa Haller for assistance and for proofreading the manuscript. We thank the therapists and patients participating in the study for allowing us to use the videotaped psychotherapy sessions. We also thank the scientific and clinical experts being involved in the I-CVI rating.
Role of funding source
Funding for this study was provided by the University of Zurich and the University of Bern; neither institution had any further role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.
Katja Machmutow received her B.S. degree in Psychology from the University of Leipzig, Germany, in 2010. She received her M.S. degree in Clinical Psychology and Health Psychology at the University of Zurich, Switzerland, in 2012. She is currently a Ph.D. Candidate of Prof. Dr. Birgit Watzke in the Department of Clinical Psychology and Psychotherapy Research at the University of Zurich, Switzerland. Her research interests are in the areas of relapse prevention in depressive disorders, and
References (41)
- et al.
A lifetime approach to major depressive disorder: the contributions of psychological interventions in preventing relapse and recurrence
Clin. Psychol. Rev.
(2015) - et al.
Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM)
Resuscitation
(2010) - et al.
Self-reported interpersonal problems and impact messages as perceived by significant others are differentially associated with the process and outcome of depression therapy
J. Couns. Psychol. Pagin-. Specified
(2017) - et al.
Development of a therapist adherence/competence rating scale for supportive-expressive dynamic psychotherapy: a preliminary report
Psychother. Res.
(1996) - et al.
Kognitive Therapie der Depression
(2010) - et al.
Manual for the Beck Depression Inventory-II
(1996) - et al.
Preventing relapse/recurrence in recurrent depression with cognitive therapy: a randomized controlled trial
J. Consult. Clin. Psychol.
(2005) - British Psychological Society, 2010. Depression: The Treatment and Management of Depression in Adults (Updated...
Einführung in die Test- und Fragebogenkonstruktion
(2010)- DGPPN, B.Ä.K., AWMF, K.B.V., BPtK, AkdÄ, DAGSHG, BApK, DGPM, DEGAM, DGRW, DGPs, 2009. (Editors) for the Guideline Group...
The Practice of Rational Emotive Behavior Therapy
Raters' manual for the Collaborative Study Psychotherapy Rating Scale - Form 6
Balanced incomplete block designs for inter-rater reliability studies
Appl. Psychol. Meas.
Der berner patienten- und therapeutenstundenbogen 2000: ein instrument zur erfassung von therapieprozessen. [The Bern Post Session Report 2000, patient and therapist versions: measuring psychotherapeutic processes.]
Z. für Klin. Psychol. und Psychother.: Forsch. und Prax.
Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Remission, recovery, relapse, and recurrence
Arch. Gen. Psychiatry
Kognitive Verhaltenstherapie bei Depressionen
Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: correspondence between therapists and observers
Adm. Policy Ment. Health
Development of a System for Rating Therapies for Depression: Differentiating Cognitive Therapy, Interpersonal Psychotherapy, and Clinical Management Pharmacotherapy
Treatment and prevention of depression
Psychol. Sci. Public Interest
The Therapist Action Scale. Instruments for the assessment of activities during dynamic psychotherapy
Psychother. Theory Res. Pract.
Cited by (3)
How Reliable Are Therapeutic Competence Ratings? Results of a Systematic Review and Meta-Analysis
2020, Cognitive Therapy and ResearchStrengthening competence of therapists-in-training in the treatment of health anxiety (hypochondriasis): Validation of the Assessment of Core CBT Skills (ACCS)
2019, Clinical Psychology and Psychotherapy
Katja Machmutow received her B.S. degree in Psychology from the University of Leipzig, Germany, in 2010. She received her M.S. degree in Clinical Psychology and Health Psychology at the University of Zurich, Switzerland, in 2012. She is currently a Ph.D. Candidate of Prof. Dr. Birgit Watzke in the Department of Clinical Psychology and Psychotherapy Research at the University of Zurich, Switzerland. Her research interests are in the areas of relapse prevention in depressive disorders, and continuation and maintenance treatments for persistent depressive disorder.
Martin grosse Holtforth received his Bachelor- and Master's-level training in psychology at the FU Berlin, Germany, in 1995. He received his Ph.D. in Clinical Psychology (summa cum laude) at the University of Bern, Switzerland, in 1999. He received his habilitation (Psychology) at the University of Bern, Switzerland, in 2006. He is currently an associated professor and lecturer in Clinical Psychology and Psychotherapy at the University of Bern, Switzerland, and head of research at the University hospital of Bern, Switzerland. His research interests are in the areas of psychotherapy in mood disorders, psychotherapeutic processes and therapeutic alliance, motivation and emotion regulation.
Tobias Krieger received his Bachelor- and Master's-level training in psychology at the University of Bern, Switzerland, in 2008. He received his Ph.D. in Psychology at the University of Zurich, Switzerland, in 2013. He is currently a clinical and scientific assistant at the Department of Psychology the University of Bern, Switzerland. His research interests are in the areas of psychotherapeutic outcome and process research, internet-based interventions, self-compassion, and psychopathology of depression.
Birgit Watzke received her Bachelor- and Master's-level training in psychology at the University of Bielefeld, Germany, in 1998. She received her Ph.D. in Psychology at the University of Hamburg, Germany, in 2002. She received her habilitation and venia legendi in Clinical Psychology and Medical Psychology in 2010. She is currently a professor in Clinical Psychology and Psychotherapy Research at the University of Zurich, Switzerland. Her research interests are in the areas of efficacy, effectiveness and processes in psychotherapy, stepped care programs, access and aftercare treatments in psychotherapy, and treatment of people with chronic forms of affective disorders.