4th Annual Scientific Meeting – Dinner Debate

ISCTM
4th Annual Scientific Meeting
25-27 February 2008
Washington, DC

Summary of Dinner Debate, Monday, February 25th, 2008: Views on the Evolving Relationship Between Journal Editors and Authors: Is a Two-Tiered Approach for Industry and Academic Authors Beneficial?

I. Will Carpenter: Position for a Two-tiered Review System for Publication of Research Manuscripts:

Dr. William Carpenter began his comments by clarifying that the debate issue was whether journals should have a two-tier system of review in order to provide special scrutiny to manuscripts submitted from the for-profit sector.  In taking up the position for a two-tiered system, Dr. Carpenter stated that his comments were based on the assumption that they applied most directly to reports of studies of marketed drugs.  He stated that these points were made on the following assumption: that they related to common [not ubiquitous] issues with industry sponsored studies which were substantially less common [not absent] in academic studies with sponsorship from non-profit entities.

Points relevant to the debate, as summarized by Dr. Carpenter, are listed below:

1. A key problem for editors concerns manuscripts that can influence clinical practice and/or have financial implications for the study sponsor.

2. The field needs to face squarely the implications of systematic bias in manuscripts from industry relating to marketed drugs.  How else to explain the “our drug beats your drug” publication bias.  An explanation offered at the debate discussion was that it was natural for authors to focus on the things that interest them most.  To an editor, this is just a way of explaining how the bias works.  Company A is not likely to be most interested in reporting how Company B’s drug is superior.

3. Any author may be subject to bias based on prestige, ego, or the hope for fame and fortune.  But the fact that anyone could be biased does not negate the obligation to address an area of systematic bias that contributes to the erosion of public trust.  Consider some fundamental differences between a pharmaceutical company’s sponsored post-marketing clinical trial and an NIH sponsored post-marketing clinical trial:

In one case the sponsor knows the desired outcome

• • In one case the sponsor may control release of the report
• • In one case the hypotheses, analytic strategy, and primary end points have been recorded in advance and peer reviewed
• • In one case the sponsor is likely to limit access to the data, while in the other the sponsor may encourage or require public release of the data
• • In one case commercial third parties may be hired to prepare manuscripts and organize reviews.
• • In one case an academic leader may be offered authorship on manuscripts where their role in the study is minimal or absent.
• • In one case the authors may worry about personal implications of publishing a report which hurts the market value of their employers’ product; in the other case the authors may worry about adverse consequences of not publishing study results in future grant applications.

4. Erosion of public trust in pharmaceutical and academic science is a critical issue.  Damage to the prestige of our scientific field is partly based on widely held views that data mining, selective reporting, bias in interpreting results, and conduct of clinical trials for marketing rather than scientific or clinical reasons are frequent.

5. There are no absolutes in this area, and there is potential for witting or unwitting bias in any study and with any investigator.  But the public [and perhaps most reasonable people] would believe that prestige and ego of authors without dollars is trumped by prestige, ego and dollars.  The financial conflicts are always there for industry studies, and only sometimes for studies sponsored by public sources.

6. There should be a high standard in review of all manuscripts, but any reader of journals will appreciate the inadequacy of the review process.  How else to explain the numerous publications from industry that favor the sponsor’s drug and run counter to reports from public science?

7. A “second tier” approach can be aggressive, such as JAMA’s requirement for independent academic statistical analysis of data being reported.  However, a more modest approach which might increase sensitivity of authors, editors and reviewers [and maybe sponsors] might evolve along the following lines:

• • Clearly identify whether the sponsor or the authors have a financial stake in the study results.
• • State whether the hypothesis, analytic strategy and end points are predefined in a public access format.
• • State whether authors have sole responsibility for the report, or whether the study sponsor has the authority to control release of the report.
• • State plan for making the data publicly available or reasons why not.

The answers would not serve to disqualify a manuscript, but would serve to focus attention on key issues.  It would apply to academic authors who receive large funds from the sponsor, or have intellectual property rights related to the study.

Dr. Carpenter concluded his remarks by stating that, if a plan evolved along these lines, it would apply to all manuscript submissions.  It would, in fact, be a one-tier system with statements that would alert editors and reviewers to potential issues that might require special attention in the review process.

[In comments forwarded following the debate, Dr. Carpenter noted that this topic had received significant attention in a JAMA editorial entitled: Impugning the Integrity of Medical Science—The Adverse Effects of Industry Influence.  The DeAngelis and Fontanarosa editorial contained specific recommendations related to the debate issue.  The same April 16, 2008 issue of JAMA contained two articles illustrating how commercial interest may be influencing the reporting of clinical trial data.] 

II. Alan Breier: Position against a Two-tiered Review System for Publication of Research Manuscripts:

Alan Breier opened his comments by highlighting the fact that public confidence in medical data has been shaken.  He cited the withdrawal of Vioxx as a recent example of the challenges of properly presenting the full picture with regards to medical information, and underscored that the blame for the public’s sense of misrepresentation of data and risk was shared between Industry, the FDA and academia.  Following these opening comments, Dr. Breier clearly stated that he was against a two-tiered publication standard for academic and industry authors, acknowledging that there were powerful potential conflicts of interest for members of both industry and academia (the former relating to the business of selling pharmaceuticals, the latter relating to issues such as the need to support academic promotion and grant funding).  He underscored the point that managing conflict of interest was critical for all authors, regardless of professional affiliation.

Besides the most obvious area of concern, that of conflict of interest, Dr. Breier listed several other issues.  They included the selective reporting of data (“cherry picking”), the failure to disclose results of negative studies (“file drawer phenomenon”), fraud or data fabrication, and ghost authoring.  He then went on to describe four key elements to address with regards to conflict of interest, including the need for transparency, the requirement for codes of conduct and formal publication policies, compliance measures and, lastly, the need for cultural change.

Dr. Breier then briefly discussed some additional considerations for enhancing publication requirements.  These included providing additional complementary materials with any submitted manuscript, including study protocol, protocol amendments, and the statistical analytic plan; ensuring registration of all trials on clintrials.gov; providing access to study data and analytic programs with each submitted manuscript; disclosing journal reviewers’ comments; and external auditing for compliance.

Dr. Breier ended his presentation with an overview of Eli Lilly’s publication policy.

III. John Kane: Summary Comments

In comments following Drs. Carpenter and Breier’s viewpoints, Dr. John Kane discussed his view on the evolving relationship between journal editors and authors

Dr. Kane commented that the advancement of science depends upon the collection, analysis and publication of data in an objective, complete and unbiased fashion. An important reevaluation of potential bias has occurred in recent years; however, the attention has largely focused on conflicts of interest and bias in relation to the publication (or lack there of) of industry-sponsored clinical trial data.  There is considerable evidence, Dr. Kane continued, that publication bias or selective reporting of clinical trial results has occurred, though it is not always clear whether and when this results from failure on the part of the authors and/or sponsors to submit reports, from recommendations by reviewers and decisions by editors not to publish, or both.

Additionally, the scope and potential impact of financial conflicts of interest in biomedical research have received considerable attention.  Dr. Kane stated that there was no doubt that our failure to acknowledge and manage these risks has enormous implications for scientific progress, clinical practice, patient outcomes and healthcare costs.

Dr. Kane commented that both Drs. Carpenter and Breier made important points in discussing this issue. Dr. Carpenter emphasized the high stakes involved in influencing prescriber practice when bias influences the design, conduct and reporting of trials, and contrasted the difference between public and private sponsorship in a number of ways.  However, the contrasts, in Dr. Kane’s opinion, minimized the bias and conflicts of interest that occur everyday in academia, though these may be less obvious and less easily identified.  For example, though there may be more absolute dollars riding on industry sponsored results, the motivating “rewards” (fame, promotion, influence and renumeration) to academic investigators cannot be underestimated in both their scope and impact on behavior (not just in their own work, but in their review of competitors grants and papers).  In many ways, this is far more insidious because it is far harder to detect.  There is no FDA policing data collection, data analyses and the “marketing” of results in academia.  Dr. Kane stated that he was aware of many non-industry supported and NIH supported research that did not get published or adequately published, adding that though it is currently very difficult for industry sponsors to withhold data, this has only recently become an issue (still largely untested) for NIH sponsored studies.

Dr. Kane then asked, are academic investigators more rigorous in recording and publishing a priori hypotheses, end points, data analytic plans etc? Unfortunately not, he concluded. If anything, the drug approval process requires far more advance documentation. Dr. Kane agreed with Dr. Breier’s suggestion that both industry and academic authors offer the study protocol with every clinical trial submitted for publication, adding that all trials should be registered in clinicaltrials.gov and quality review should be built into the research enterprise system-wide.

Dr. Kane emphasized that numerous reports (e.g. Bekelman et al 2003; Perlis et al 2005; Heres et al 2006; Turner et al 2008) have focused on industry publication bias and potential industry–academia conflicts of interest.  He stated that these data make a compelling argument for better management of these potential risks.

Dr. Kane noted that Dr. Breier had emphasized what Lilly is doing to address these challenges.  Clearly progress is being made, but more remains to be done across the biomedical research enterprise.  It would be valuable if similar reviews (to those cited above) could examine relevant risks in non-industry-supported research.  If one could overcome the impediments to accurately analyzing this issue (which is not easy, he added), it might be more embarrassing than we care to imagine.

The debate that fostered theses remarks asked if a two tiered approach for reviewing and publishing papers from industry and academia would be appropriate.  One could argue, Dr. Kane continued, that we already have a two tiered approach in that industry sponsored studies undergo a different type of scrutiny because their sponsorship is obvious.  Journals need to address the concern that negative results are less likely to be submitted for publication or published than positive results in both industry and academia.

Dr. Kane ended his comments by stating that it was not convincing to him whether a two tier publication system would solve the problems summarized above.

The following references were provided by Dr. Kane in support of his comments:

Bekelman JE. Li Y. Gross CP. Scope and impact of financial conflicts of interest in

biomedical research: a systematic review JAMA. 289(4):454-65, 2003

Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S. Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry. 2006 Feb;163(2):185-94.

Perlis RH, Perlis CS. Wu Y. Hwang C. Joseph M. Nierenberg AA. Industry sponsorship and financial conflict of interest in the reporting of clinical trials in psychiatry. American Journal of Psychiatry. 162(10):1957-60, 2005

Turner EH. Matthews AM. Linardatos E. Tell RA. Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine. 358(3):252-60, 2008