The primary goal in designing a randomized controlled clinical trial (RCT) is to minimize bias in the estimate of treatment effect. Randomized group assignment, double-blinded assessments, and control or comparison groups reduce the risk of bias. The design must also provide sufficient statistical power to detect a clinically meaningful treatment effect and maintain a nominal level of type I error. An attempt to integrate neurocognitive science into an RCT poses additional challenges. Two particularly relevant aspects of such a design often receive insufficient attention in an RCT. Multiple outcomes inflate type I error, and an unreliable assessment process introduces bias and reduces statistical power. Here we describe how both unreliability and multiple outcomes can increase the study costs and duration and reduce the feasibility of the study. The objective of this article is to consider strategies that overcome the problems of unreliability and multiplicity.

The 2008 Institute of Medicine review of interventions research for posttraumatic stress disorder (PTSD) concluded that new, well-designed studies are needed to evaluate the efficacy of treatments for PTSD. The Department of Veterans Affairs (VA), the Department of Defense, and the National Institute of Mental Health convened a meeting on research methodology and the VA issued recommendations for design and analysis of randomized controlled clinical trials (RCTs) for PTSD. The rationale that formed the basis for several of the components of the recommendations is discussed here. Fundamental goals of RCT design are described. Strategies in design and analysis that contribute to the goals of an RCT and thereby enhance the likelihood of signal detection are considered.

Pilot studies represent a fundamental phase of the research process. The purpose of conducting a pilot study is to examine the feasibility of an approach that is intended to be used in a larger scale study. The roles and limitations of pilot studies are described here using a clinical trial as an example. A pilot study can be used to evaluate the feasibility of recruitment, randomization, retention, assessment procedures, new methods, and implementation of the novel intervention. A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot. Contrary to tradition, a pilot study does not provide a meaningful effect size estimate for planning subsequent studies due to the imprecision inherent in data from small samples. Feasibility results do not necessarily generalize beyond the inclusion and exclusion criteria of the pilot design. A pilot study is a requisite initial step in exploring a novel intervention or an innovative application of an intervention. Pilot results can inform feasibility and identify modifications needed in the design of a larger, ensuing hypothesis testing study. Investigators should be forthright in stating these objectives of a pilot study. Grant reviewers and other stakeholders should expect no more.

The Agency for Healthcare Research and Quality, part of the US Department of Health and Human Services, has issued several Requests for Applications to conduct comparative effectiveness research (CER). Many of the applications will involve randomized controlled clinical trials that include an active comparator. The inclusion of an active comparator has implications for clinical trial design.

Despite a common misperception, a clinical trial result of no significant difference between active treatment groups does not imply equivalence or noninferiority. A noninferiority trial, on the other hand, can directly test whether one active treatment group is noninferior to the other. For example, noninferiority of an inexpensive generic could be tested in comparison with a novel, more costly intervention. Although seldom used in psychiatry, noninferiority clinical trials could play a fundamental role in CER. Features of noninferiority and the nearly ubiquitous superiority designs are contrasted. The noninferiority margin is defined and its application and interpretation are discussed.

Evidence of noninferiority can only come from well-designed and conducted noninferiority CER. Sample sizes needed in noninferiority trials and in superiority trials that include an active comparator are substantially larger than those needed in trials that can utilize a placebo control in their scientific design. As a result, trials with active comparators are more costly, require longer recruitment duration, and expose more participants to the risks of an experiment than do trials in which the only comparator is placebo.

The National Institute of Mental Health Strategic Plan calls for the development of personalized treatment strategies for mental disorders. In an effort to achieve that goal, several investigators have conducted exploratory analyses of randomized controlled clinical trial (RCT) data to examine the association between baseline subject characteristics, the putative moderators, and the magnitude of treatment effect sizes. Exploratory analyses are used to generate hypotheses, not to confirm them. For that reason, independent replication is needed. Here, 2 general approaches to designing confirmatory RCTs are described that build on the results of exploratory analyses. These approaches address distinct questions. For example, a 2 × 2 factorial design provides an empirical test of the question, “Is there a greater treatment effect for those with the single-nucleotide polymorphism than for those without that polymorphism?” and the hypothesis test involves a moderator-by-treatment interaction. In contrast, a main effects strategy evaluates the intervention in subgroups and involves separate hypothesis-testing studies of treatment for subjects with the genotypes hypothesized to have enhanced and adverse response. These designs require widely disparate sample sizes to detect a given effect size. The former could need as many as 4-fold the number of subjects. As such, the choice of design impacts the research costs, clinical trial duration, and number of subjects exposed to risk of an experiment, as well as the generalizability of results. When resources are abundant, the 2 × 2 design is the preferable approach for identifying personalized interventions because it directly tests the differential treatment effect, but its demand on research funds is extraordinary.

On September 18, 2007, a collaborative session between the International Society for CNS Clinical Trials and Methodology and the International Society for CNS Drug Development was held in Brussels, Belgium. Both groups, with membership from industry, academia, and governmental and nongovernmental agencies, have been formed to address scientific, clinical, regulatory, and methodological challenges in the development of central nervous system therapeutic agents. The focus of this joint session was the apparent diminution of drug-placebo differences in recent multicenter trials of antipsychotic medications for schizophrenia. To characterize the nature of the problem, some presenters reported data from several recent trials that indicated higher rates of placebo response and lower rates of drug response (even to previously established, comparator drugs), when compared with earlier trials. As a means to identify the possible causes of the problem, discussions covered a range of methodological factors such as participant characteristics, trial designs, site characteristics, clinical setting (inpatient vs outpatient), inclusion/exclusion criteria, and diagnostic specificity. Finally, possible solutions were discussed, such as improving precision of participant selection criteria, improving assessment instruments and/or assessment methodology to increase reliability of outcome measures, innovative methods to encourage greater subject adherence and investigator involvement, improved rater training and accountability metrics at clinical sites to increase quality assurance, and advanced methods of pharmacokinetic/pharmacodynamic modeling to optimize dosing prior to initiating large phase 3 trials. The session closed with a roundtable discussion and recommendations for data sharing to further explore potential causes and viable solutions to be applied in future trials.

The International Society for CNS Clinical Trials and Methodology (ISCTM) held its 4th Annual Autumn Conference in Toronto, Ontario, October 6-7, 2008. The purpose of the present report is to provide an overview of one of the sessions at the conference which focused on the designs and methodologies to be applied in clinical trials of new treatments for Alzheimer's disease (AD) with purported "disease-modifying" effects. The session began with a discussion of how neuroimaging has been applied in multiple sclerosis clinical trials (another condition for which disease modification claims have been achieved). The next two lectures provided a pharmaceutical industry perspective on some of the specific challenges and possible solutions for designing trials to measure disease progression and/or modification. The final lecture provided an academic viewpoint and the closing discussion included additional academic and regulatory perspectives on trial designs, methodologies, and statistical issues relevant to the disease modification concept.

Based on maximum likelihood estimates obtained from mixed-effects linear models, closed-form power functions are derived to detect two-way and three-way interactions that involve longitudinal course of outcome over time in clinical trials. Sample size estimates are shown to decrease with increasing within-subject correlations. It is further shown that when clinical trial designs are balanced in group sizes, the sample size required to detect an effect size for a three-way interaction is exactly fourfold that required to detect the same effect size of a two-way interaction. Furthermore, this fourfold relationship virtually holds for unbalanced allocations of subjects if one factor is balanced in the three-way interaction model. Simulations are presented that verify the sample size estimates for two-way and three-way interactions.