2009 5th Annual Scientific Meeting – Improving Methodology of RCTs in Post-traumatic Stress Disorder – Session Summary
Improving Methodology of Randomized Clinical Trials in Post-traumatic Stress Disorder
A Plenary Symposium of the International Society for CNS Drug Trials Methodology (ISCTM),
Arlington, VA, 4 March 2009
Chairmen: Andrew C. Leon, Ph.D., Weill Cornell Medical College, New York; Lori L. Davis, M.D., Veterans Administration Medical Center, Alabama
Speakers: Drs. Leon and Davis, plus Thomas Neylan, M.D., University of California; Timothy O’Leary, Veterans Affairs, Washington, D.C.
Summary
The basis of this session originated in the Institute of Medicine (IOM) review of randomized controlled trials (RCT) for interventions for posttraumatic stress disorder (PTSD) which concluded that 1) the evidence was inadequate regarding the effectiveness of most pharmacologic and psychosocial interventions for PTSD, 2) most studies that were reviewed had methodological limitations, some of which compromised the credibility of the results and 3) well-designed studies are needed to define efficacious treatments for PTSD.
Timothy O’Leary, MD, PhD, the Director of VA Clinical Science Research and Development (VA CSR&D) gave a short introduction entitled “National Priorities for Clinical Trial Research in PTSD.” The 2008/2009 research funding initiatives include the VA Cooperative Clinical Trial Award (CCTA) Program to support smaller scale clinical trials, resourcing with Coordinating Center expertise, the VA Central DMC to monitor smaller scale clinical trials, the DOD (USAMRMC) 150M budget for PTSD and TBI research, including PTSD/TBI Clinical Trials Coordinating Center, PTSD Research Consortium, intramural and extramural programs, a joint collaboration with the NIMH, VA CSR&D funding opportunity entitled “Clinical Pharmacotherapy for PTSD: Single and Collaborative Studies“ (R34), and additional solicitations planned by the VA in coordination with other federal funding agencies. The challenges for the field were discussed and include the concern that preclinical investigations in PTSD are not leading to new therapeutic modalities or development of PTSD-specific molecular entities, the use of pharmacologic interventions optimized for other conditions may not be optimal for PTSD, and comorbid conditions need to be addressed more effectively.
Lori Davis, M.D., Professor of Psychiatry at the University of Alabama School of Medicine and Chief of Research Service at the Tuscaloosa VA, presented a comprehensive review of PTSD and the issues related to pharmacological treatments and randomized controlled trials in PTSD. The lifetime prevalence of PTSD is 7% of the US population (3-5% men, 10% women), 19% of Vietnam veterans, 13% Iraqi war veterans, and 32% of the treatment seeking Iraqi war veterans. PTSD is chronic and may persist for decades or a lifetime. PTSD is commonly comorbid with other Axis I (>50%) disorders. Despite its high prevalence and chronicity, there are only two FDA-approved medications: sertraline and paroxetine.
Dr. Davis reviewed the nosology of PTSD, pointing out that in addition to exposure to a life-threatening event involving intense fear, helplessness, or horror (criterion A), the current DSM-IV criteria involved three clusters of symptoms: re-experiencing symptoms (criterion B), avoidance and emotional numbing symptoms (criterion C), and hyperarousal symptoms (criterion D). The symptoms must be present for at least one month duration (acute < 3 mo; chronic > 3 mo) and cause significant social dysfunction, occupational dysfunction, or significant personal distress. DSM-IV uses a conventional 3-Factor Model, whereas more recent research suggests that consideration of the King 4-Factor Model that separates avoidance symptoms from numbing symptoms. Symptoms of dissociation, shame and guilt continue to be insufficiently accounted for in the diagnostic criteria.
Dr. Davis discussed the etiologic Bio-Psycho-Social Model of PTSD, that include biological disturbances (monoamines, amino acid neurotransmitters, neurohormones, neuroanatomical pathways), psychological disturbances (in sense of self and integrity, trust vs. suspiciousness, guilt and shame, and difficulties with relationships) and behavioral disturbances (avoidance, checking for safety, and social and occupational difficulties).
The pharmacologic treatment for PTSD has included antidepressants (SSRI, TCA, mirtazapine, nefazodone, venlafaxine, MAOIs), alpha-adrenergic blockers, atypical neuroleptics, anticonvulsants, and benzodiazepines. The Institute of Medicine Treatment of PTSD: An Assessment of the Evidence concluded that the evidence for all classes of drugs was inadequate to determine efficacy for PTSD. However, one of the PTSD expert panelists Thomas Mellman, MD did not concur and wrote the dissenting comment that the evidence is suggestive but not sufficient to conclude efficacy of SSRIs in general population with PTSD and the evidence is suggestive that SSRIs are not effective in populations consisting of predominantly male veterans with chronic PTSD. In addition, Dr. Mellman wrote that the evidence is suggestive but not sufficient to conclude the efficacy of second generation antipsychotic medications as adjunctive treatment for PTSD. Dr. Davis discussed the methodologic issues raised by the IOM Review of RCTs in PTSD that include challenges to internal validity, standardization of treatment and outcome measures, attrition, multiplicity, credible control group, investigator independence, need to study special veteran populations , applicability to VA and veteran populations, inconsistent definitions of recovery and remission, and lack of studies to define early intervention, length of treatment, and length of follow-up.
Dr. Davis reviewed the limited literature regarding analysis of individual PTSD item response to SSRI and also venlafaxine. In these pooled analyses, anger is consistently an early response item (week 1) with other items being delayed and nightmares or dreams typically not responding by week 12. (Sertraline: week 6 psychological distress at cues/triggers, anhedonia, detachment, numbness, hypervigilance; week 10 avoidance of activities, foreshortened future. Venlafaxine: week 2 physiological reactivity, week 4 intrusive recollections, psych distress at exposure to cues, week 6-12 avoidance of thoughts/feelings or conversations, diminished interest, detachment or estrangement, restricted range of affect, sense of foreshortened future, difficulty concentrating, hypervigilance, exaggerated startle response). These data suggest that there may be some symptoms of PTSD that are more biologically driven (anger and impulsivity) and others that may be behaviorally driven (avoidance) and respond differently to different treatments based on mechanism of action.
Dr. Davis concluded with a list of dilemmas faced when designing an RCT for the treatment of PTSD, which include study population issues such as civilian vs. veterans, gender balance, number of lifetime traumatic events, type of traumatic index event, chronicity of exposure to trauma, subtypes of PTSD, chronicity of PTSD illness, comorbidities, prior treatment resistant or refractory, and genetic polymorphisms or markers. The alternatives for intervention include novel antidepressants, modulators of norepinephrine, atypical neuroleptics, CRF antagonists, modulators of GABA and glutamate, combination psychopharmacology, combination of medications with psychotherapy. Design issues include monotherapy vs. placebo, combination strategies, and augmentation designs. The targeted outcomes include the full PTS score, PTSD symptom clusters (subdimensions), response and remission, and quality of life and reintegration. Also consideration of frequency of assessments, length of trial, adherence to protocol and treatment, and retention is important in the planning for RCTs in PTSD.
Dr. Davis concluded with the points that PTSD is common, chronic, and complex. To date, we are limited by the small number of RCTs, small sample sizes, and modest effect sizes. Innovative treatments are needed and sound study designs are essential.
Andrew C. Leon, Ph.D. of Weill Cornell Medical College, New York, NY described “Strategies to Examine Moderators of Treatment for PTSD”. The talk covered four general areas: 1) Roles of moderators in RCTs 2) A contrast between two designs for RCTs with moderators 3) A consideration of hypotheses that each design addresses 3) A comparison of the required sample size for each design
A moderator of treatment is a baseline characteristic of subject that is associated (positively or negatively) with response. (see Kraemer et al., Mediators and Moderators of Treatment Effects in RCTs. Arch Gen Psych, 2002 ) For example if there is no active-placebo separation for males, but active is superior to placebo for females, gender would be a moderator of treatment.
Dr. Leon described an ongoing study “Vocational Rehabilitation For PTSD Interventions” (PI: LL Davis) where two interventions are being compared: Individual Placement and Support vs. Standard Vocational Rehabilitation Program. There are four hypothesized moderators of treatment. The between intervention effect size is hypothesized to be smaller for subjects with any of the following: 1) inadequate transportation 2) inadequate housing 3) little to no financial reserve/means 4) family care burden
Two approaches to examining moderators in RCTs were discussed:
1) Design RCT to examine moderator by treatment interaction with hypothesis testing
2) Exploratory hypotheses in RCT protocol that examine pre-specified, hypothesized moderators of treatment.
Kraemer and colleagues (2002) urge that exploratory analyses of a moderator focus on magnitude of effect and not on p-values. The magnitude of effect can be quantified with Cohen’s d, Number needed to treat (NNT), number needed to harm (NNH), AUC (as defined in Kraemer & Kupfer, Biological Psych , 2006). Exploratory results are tentative, they are not for treatment decision making. If exploratory analyses support moderating effect, 2 subsequent designs might be considered: 1) 2 x 2 factorial RCT 2) Main effect design
The 2 x 2 factorial RCT would randomize subjects to either Active or Control and the randomization would be stratified by the moderator. For example, in examining a genotype as a moderator, one might test to see if there is a differential treatment effect for SNP+ & SNP- . H0: ActiveSNP+- ControlSNP+ = ActiveSNP– ControlSNP- . The significance test involves a Treatment by SNP interaction.
The sample size required to detect clinically meaningful differences with an interaction is fourfold that of main effect. This has been shown for ANOVA (Fleiss, 1986. Design & Analysis of Clinical Experiments) and for both 2 level mixed-effects models (Leon & Heo, CSDA, 2008 and 2 level mixed-effects models (Heo & Leon, J Biopharm Stat, 2009). Therefore the cost of 1 RCT testing an Interaction will be at lest the cost of 2 or 3 RCTs for main effects.
The Main Effects strategy was then considered. Initially include exploratory hypotheses in an RCT protocol. If the results suggest moderator is associated with enhanced response (as identified in Exploratory), and use in design of subsequent RCT. In contrast, if moderator is associated with decreased response, use to select (or develop) a novel intervention and test in subsequent RCT. Finally, attempt to replicate RCT results in subsequent trial. For example, one might examine whether there is treatment effect for SNP- . H0: ActiveSNP- = ControlSNP-. The significance test involves the main effect of treatment. The generalizability of such a design is limited to SNP- . Importantly, because the sample size is 25% that of an 2×2 design testing an interaction, there is an opportunity to replicate.
In a similar fashion, one could test a novel intervention for those expected to have decreased response. This would be done by limiting recruitment to those with characteristics of decreased response (as identified in exploratory). Randomize subjects either to a Novel Active or the Control. For example it would examine the question, “Is there treatment effect for SNP+ ? and test H0: ActiveSNP+ = ControlSNP+ . The significance test would involve the Main effect of Treatment. Once again the generalizability is limited, in this case to SNP+. The Main effect N is 25% that of interaction and this provides an opportunity to replicate.
Dr. Leon summarized by stating that moderators are baseline characteristics of subjects. Initially exploratory analyses of RCTs can examine moderators. The exploratory results can guide design of subsequent RCT, particularly the inclusion and exclusion criteria. Most importantly, the sample size required to detect an effect size for an interaction is fourfold that for main effect of the same magnitude. Consider recruiting only S’s with target characteristic (e.g., genotype), but this limits generalizability. Develop and evaluate novel treatments for those with characteristic associated with decreased or adverse response. A limitation of the approach that was described is that the examples involved the simplest case, in which the cell sizes are equivalent and the dependent variable is continuous. If the design differs from that the same general principles apply, but the details will vary somewhat.
Alina Suris, Ph.D., ABPP, an Associate Professor of Psychiatry from the University of Texas Southwestern Medical Center, Dallas, TX presentation entitled “Psychotherapy Methodological Issues: Choice of Comparison Group” reviewed the literature on psychotherapy comparison groups. Gold (1954) developed placebo controlled studies in the US to establish that the benefits observed with medications were due to their chemical properties rather than to patients’ expectations, hopes, or other psychological processes. Rosenthal and Frank (1956) suggested that placebo designs be applied to psychotherapy research as well, to rule out factors that are incidental to the active “ingredients” specified by the treatment protocol. Unlike placebo vs. active medication trials, psychotherapy trials cannot use placebo controlled designs in which all factors except the therapy’s active ingredient are held constant—challenge to control all non-specific factors. She compared Psychotherapy vs. Medication RCTs and pointed out that in psychotherapy trials patient and providers are not blinded to treatment (“solution”= blinded raters), the impact of patient provider interactions on outcome (provider skill/ impacts drop out) is a factor, the provider & patient adherence to the treatment protocol is important, the psychotherapy is a manualized treatment, and the timing of outcome/symptom measures are usually at baseline and endpoint.
Dr. Suris discussed the traditional psychotherapy designs that include: randomized wait list (currently being challenged as a credible control), nonspecific comparison, dismantling, additive, parametric designs, and direct comparisons of two treatments. She highlighted that the randomized Wait List Control design have been held by proponents to use as a way to assess an untreated comparison group at the same intervals as the treatment group. Wait list controls for many threats to internal validity (maturation, testing, instrumentation, regression, selection, etc.), but it can be viewed by the participant detrimental. While it controls for the passage of time, the disadvantages of wait list control include the concern that it does not control for increased attention, expectation of therapeutic intervention, psychological consequences of legitimized sick role (Klerman, 1986).
Dr. Suris reviewed the common ingredients of psychotherapy and non-specific treatment comparison, which include contact with therapist, attention to a problem, suggestions for change, and the generation of faith, hope, and expectancy for improvement. Treatment effect is impacted by type of design and how well conditions are equated (e.g., more active comparison group and greater control for non-specific factors = smaller effect size). It is not appropriate to compare effect sizes from a study that used wait list with study that used non-specific design.
Dr. Suris concluded that medication and psychotherapy RCT for PTSD vary on multiple factors (comparison group choice, blinding, manualization, fidelity, interactions of provider & patients, adherence to protocol, timing of symptom and outcome measurement), psychotherapy RCT have no true placebo control comparison, comparison groups differ depending on questions being addressed, and effect sizes vary depending on comparison group and how well groups equated.
Thomas C. Neylan, M.D. for the University of California San Francisco presented the final segment entitled “Methodological Challenges for Secondary Prevention Trials in PTSD.” He outlined the challenges related to designing clinical trials aimed for the secondary prevention of PTSD in acutely traumatized victims. The first part of his talk focused on prospective studies that have helped to target those who are at the highest risk for developing PTSD who may be the target group for novel interventions. He subsequently reviewed the current status of secondary prevention trials and challenges with recruitment, enrollment, and retention. Dr. Neylan concluded his presentation on several novel approaches for conducting such trials and the potential for clinician researchers (e.g. anesthesiologists, emergency medicine physicians) outside of the mental health field to help accelerate the development of this field.
According to Dr. Neylan, the goal of secondary prevention is to prevent the development of PTSD and other consequences of traumatic stress exposure. This may involve a prophylactic intervention in high risk individuals (e.g. “Battlemind” training), a blanket approach to everyone exposed to a traumatic event (“Re-Set” training-DoD), or an intervention to a targeted subgroup at high risk for PTSD. There are problems with nontargeted intervention that include a question of when to intervene?, i.e. immediate (0-48 hours) versus acute (0-4 weeks). Interventions such as debriefing and psychological first aid may be intrusive and not feasible. We may need to focus on education and reducing stigma for seeking care after symptoms present. Identifying high risk subgroups requires prospective studies of high risk individuals and most longitudinal studies involve subjects recruited post exposure. These designs have practical and economic advantages but may suffers from selection bias and retrospective biases. Military populations and first responders (police and firefighters) are good samples to start with despite problems with generalizability. We know that risk factors for PTSD include severity of trauma (ie, threat, duration, injury, loss), prior traumatization, gender, ethnicity, prior mood and/or anxiety disorders, family history of mood or anxiety disorders, education level, and peritraumatic dissociation, distress, and panic, increased heart rate variability, low cortisol, and reduced hippocampal volume.
Dr. Neylan gave a brief review of Secondary Prevention Trials which include cognitive behavioral therapy for Acute PTSD, propranolol medication, and SSRIs. Studies involving self help booklet were not helpful and debriefing is not well supported. The problems encountered by Secondary Treatment Trials include Recruitment (Must be embedded in emergency room setting, the research is often must subordinate to immediate critical care interventions, and the study may have iatrogenic effects), retention, and generalizability. Novel Approaches are needed and Dr. Neylan suggests that we take a long view of who will implement successful interventions and the likelihood of taking place in mental health treatment settings dependent on technical complexity of intervention. He concluded that if mental health personnel are involved, ideally it would be in settings with integrated care and studies should likely will involve emergency care staff or settings outside of traditional health care. Novel Approaches with Treatment Provided by Mental Health for PTSD are currently focusing on telephone psychotherapy (e.g. CBT), web interaction, cyber CBT. The novel interventions led by non mental health personnel likely to involve pharmacologic agents, psychoeducation, or effective referral strategies, may focus on non-stigmatized primary outcomes (e.g. sleep, pain, speed of physical recovery), and may involve anesthesiologists (e.g. beta blocker studies), Critical Care and Burn Specialists (e.g. steroid use), Trauma Surgeons (hypnotics, anti-adrenergic agents, etc.), and Primary Care providers (antidepressants, or any of the above). Dr. Neylan concluded that access to effective treatment for the secondary prevention of PTSD will vary as a function of the need for technical expertise in mental health (e.g., the less dependent on mental health, the greater likelihood for wider distribution). Specialists outside of mental health may be ideally suited to lead pharmacologic treatment trials for preventing PTSD.