Methodological issues in psychological treatment research : applications to gambling research and therapist effects

Abstract

Over the last couple of decades evidence-based psychotherapies have flourished, and there are now therapies that are well-established for a wide range of problems. At the same time the mental-health burden is still enormous, and challenges to the dissemination of treatments are substantial. Despite the considerable gains in knowledge that have been made, many issues remain unsolved, and there are many reasons to be skeptical of the current quality of the evidence. The aim of this thesis was to explore methodological challenges that impact the evaluation of psychological treatments in general but also gambling treatment trials specifically. In the first part, I present a broader overview of some of the contemporary issues that concern the scientific investigation of psychotherapies. Two issues are then explored in more detail, 1) the broader issue of therapist effects in longitudinal studies, and 2) the more specific issue of analyzing semicontinuous data as a treatment outcome. After expanding on these issues, the findings are then used in two clinical gambling studies.

In Study I, we investigated the consequences of ignoring therapist effects in longitudinal data. We derived what factors impact the type I errors, and performed an extensive simulation study. The empirical simulation results validated the analytical results and showed that even when 5% of the variance in slopes is at the therapist level, the type I errors can be substantially inflated. When analyzing data from longitudinal studies, investigators should account for the possibility that therapists might have different overall slopes. In an LMM, this can be accounted for by including a random slope at the therapist level. In order to help investigators plan multilevel longitudinal studies, an R package (powerlmm) was developed.

In Study II, we investigated the challenges of estimating treatment effects in gambling studies using gambling expenditure as an outcome. Gambling outcomes are typically very skewed and can include a large number of zeros. Investigators typically try to analyze such data mostly by log transforming the outcome, or continue with a standard analysis based on normally distributed residuals. In this paper, we propose that a marginalized two-part model can be a more attractive option. We compared the performance of the proposed two-part model to the typical methods used by investigators. The performance of these models were compared using real data and via different Monte Carlo simulation scenarios. The choice of an appropriate estimand for treatment effects was also discussed, and we argue that gambling researchers should primarily be concerned with the overall reduction in gambling losses.

In Study III, we applied and extended the work in Study II to investigate how concordant gamblers and their concerned significant others (CSOs) were in their reports of gambling losses. The sample consisted of problem gamblers and their CSOs participating in a trial comparing individual CBT versus behavioral couples therapy. A total of 133 dyads were included, and we used their baseline reports of gambling losses using the timeline followback covering the last 30 days. Overall we found that there was a fair level of agreement, ICC = .57, 95% CI [.48, .64]. There were some evidence that partner CSOs had a higher level of agreement compared to parent CSOs, ICCdiff = .20, 95% CI [.03, .39].

In Study IV, we applied the results from Study I, II, and III to investigate the effects of an internet-delivered program aimed at the CSOs of treatment refusing problem gamblers. In total, 100 CSOs of treatment-refusing problem gamblers were randomized to either ten weeks of ICBT or a waitlist control. At posttest the intervention group reported an improvement on the CSO’s emotional consequences (d = -0.90, 95% CI [-1.47, -0.33]), relationship satisfaction (d = 0.41, 95% [0.05, 0.76]), anxiety (d = -0.45, 95% [-0.81, -0.09]), depression (d = -0.49, 95% [-0.82, -0.16]). Any effects on the CSO’s reports on gambling losses and on treatment-seeking were inconclusive. Problem gamblers are hard to influence via their CSO proxies; however, the intervention had a clinically meaningful effect on the CSO’s coping as measured by their emotional consequences, anxiety, depression, and relationship satisfaction. Several methodological issues are discussed in relation to this RCT. For transparency and for better pooling of data, we also published the raw data, including all measured outcomes together with the R scripts used to analyze the trial. The data and scripts can be downloaded from https://osf.io/awtg7.

Psychotherapy researchers face significant challenges, and there is a great need for high-quality psychotherapy trials, a better appreciation of the methodological issues, and more transparent reporting practices. Hopefully, improvements to psychotherapy research will follow, and that these improvements will improve clinical practice and reduce the mental health burden in general.