ResearchPad - peer-review https://www.researchpad.co Default RSS Feed en-us © 2020 Newgen KnowledgeWorks <![CDATA[Neural networks for open and closed Literature-based Discovery]]> https://www.researchpad.co/article/elastic_article_14696 Literature-based Discovery (LBD) aims to discover new knowledge automatically from large collections of literature. Scientific literature is growing at an exponential rate, making it difficult for researchers to stay current in their discipline and easy to miss knowledge necessary to advance their research. LBD can facilitate hypothesis testing and generation and thus accelerate scientific progress. Neural networks have demonstrated improved performance on LBD-related tasks but are yet to be applied to it. We propose four graph-based, neural network methods to perform open and closed LBD. We compared our methods with those used by the state-of-the-art LION LBD system on the same evaluations to replicate recently published findings in cancer biology. We also applied them to a time-sliced dataset of human-curated peer-reviewed biological interactions. These evaluations and the metrics they employ represent performance on real-world knowledge advances and are thus robust indicators of approach efficacy. In the first experiments, our best methods performed 2-4 times better than the baselines in closed discovery and 2-3 times better in open discovery. In the second, our best methods performed almost 2 times better than the baselines in open discovery. These results are strong indications that neural LBD is potentially a very effective approach for generating new scientific discoveries from existing literature. The code for our models and other information can be found at: https://github.com/cambridgeltl/nn_for_LBD.

]]>
<![CDATA[What makes an effective grants peer reviewer? An exploratory study of the necessary skills]]> https://www.researchpad.co/article/elastic_article_13869 This exploratory mixed methods study describes skills required to be an effective peer reviewer as a member of review panels conducted for federal agencies that fund research, and examines how reviewer experience and the use of technology within such panels impacts reviewer skill development. Two specific review panel formats are considered: in-person face-to-face and virtual video conference. Data were collected through interviews with seven program officers and five expert peer review panelists, and surveys from 51 respondents. Results include the skills reviewers’ consider necessary for effective review panel participation, their assessment of the relative importance of these skills, how they are learned, and how review format affects skill development and improvement. Results are discussed relative to the peer review literature and with consideration of the importance of professional skills needed by successful scientists and peer reviewers.

]]>
<![CDATA[Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial]]> https://www.researchpad.co/article/elastic_article_12549 To evaluate the impact of an editorial intervention to improve completeness of reporting of reports of randomised trials.DesignRandomised controlled trial (RCT).SettingBMJ Open’s quality improvement programme.Participants24 manuscripts describing RCTs.InterventionsWe used an R Shiny application to randomise manuscripts (1:1 allocation ratio, blocks of 4) to the intervention (n=12) or control (n=12) group. The intervention was performed by a researcher with expertise in the content of the Consolidated Standards of Reporting Trials (CONSORT) and consisted of an evaluation of completeness of reporting of eight core CONSORT items using the submitted checklist to locate information, and the production of a report containing specific requests for authors based on the reporting issues found, provided alongside the peer review reports. The control group underwent the usual peer review.OutcomesThe primary outcome is the number of adequately reported items (0–8 scale) in the revised manuscript after the first round of peer review. The main analysis was intention-to-treat (n=24), and we imputed the scores of lost to follow-up manuscripts (rejected after peer review and not resubmitted). The secondary outcome is the proportion of manuscripts where each item was adequately reported. Two blinded reviewers assessed the outcomes independently and in duplicate and solved disagreements by consensus. We also recorded the amount of time to perform the intervention.ResultsManuscripts in the intervention group (mean: 7.01; SD: 1.47) were more completely reported than those in the control group (mean: 5.68; SD: 1.43) (mean difference 1.43, 95% CI 0.31 to 2.58). We observed the main differences in items 6a (outcomes), 9 (allocation concealment mechanism), 11a (blinding) and 17a (outcomes and estimation). The mean time to perform the intervention was 87 (SD 42) min.ConclusionsWe demonstrated the benefit of involving a reporting guideline expert in the editorial process. Improving the completeness of RCTs is essential to enhance their usability.Trial registration numberNCT03751878. ]]> <![CDATA[Publishing in the time of COVID-19]]> https://www.researchpad.co/article/N07072ecf-2583-4c65-8563-382d6b1f3bca

eLife is making changes to its policies on peer review in response to the impact of COVID-19 on the scientific community.

]]>
<![CDATA[Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial]]> https://www.researchpad.co/article/Na12afed9-c22f-4808-b7be-bd8ca2011389

Introduction

Transparent and accurate reporting is essential for readers to adequately interpret the results of a study. Journals can play a vital role in improving the reporting of published randomised controlled trials (RCTs). We describe an RCT to evaluate our hypothesis that asking peer reviewers to check whether the most important and poorly reported CONsolidated Standards of Reporting Trials (CONSORT) items are adequately reported will result in higher adherence to CONSORT guidelines in published RCTs.

Methods and analysis

Manuscripts presenting the primary results of RCTs submitted to participating journals will be randomised to either the intervention group (peer reviewers will receive a reminder and short explanation of the 10 most important and poorly reported CONSORT items; they will be asked to check if these items are reported in the submitted manuscript) or a control group (usual journal practice). The primary outcome will be the mean proportion of the 10 items that are adequately reported in the published articles. Peer reviewers and manuscript authors will not be informed of the study hypothesis, design or intervention. Outcomes will be assessed in duplicate from published articles by two data extractors (at least one blinded to the intervention). We will enrol eligible manuscripts until a minimum of 83 articles per group (166 in total) are published.

Ethics and dissemination

This pragmatic RCT was approved by the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R62779/RE001). If this intervention is effective, it could be implemented by all medical journals without requiring large additional resources at journal level. Findings will be disseminated through presentations in relevant conferences and peer-reviewed publications. This trial is registered on the Open Science Framework (https://osf.io/c4hn8).

]]>
<![CDATA[Minocycline for the treatment of mental health and neurological conditions: study protocol of a systematic review and meta-analysis]]> https://www.researchpad.co/article/N69496870-2fb0-4976-b66a-287629361afd

Introduction

Due to the anti-inflammatory, antioxidant and anti-apoptotic properties of minocycline, clinical trials have evaluated the potential of this drug to treat several psychiatric and neurological disorders, including major depressive disorder, schizophrenia, bipolar disorder, stroke and amyotrophic lateral sclerosis. This protocol proposes a systematic review (and potential meta-analysis) that aims to identify and critically evaluate randomised controlled trials of minocycline for treating psychiatric and neurological disorders.

Methods and analysis

PubMed, Embase, Cochrane Central Register of Controlled Clinical Trials, PsycINFO and Cumulative Index to Nursing and Allied Health Literature (CINAHL) will be used to identify randomised controlled trials that used minocycline to treat psychiatric and neurological disorders. Double-blind, randomised, controlled, clinical trials of participants aged 18 years or older and written in English will be included in the review. Data will be extracted by two independent reviewers. Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines will be followed and the Cochrane Collaboration’s ‘Risk of Bias’ tool will be used to assess the risk of bias in all studies included in the systematic review. The Grading of Recommendations, Assessment, Development and Evaluation system will be used to access the overall quality of the level of evidence of the studies. If sufficient evidence is identified, a meta-analysis will be conducted using the standardised mean difference approach and reported with 95% CIs. Heterogeneity of evidence will be evaluated using the I2 model.

Ethics and dissemination

This systematic review will evaluate only published data; therefore, ethical approval is not required. The systematic review will be published in a peer-reviewed journal and presented at relevant research conferences.

Trial registration number

CRD42020153292.

]]>
<![CDATA[Quantifying gender disparity in physician authorship among commentary articles in three high-impact medical journals: an observational study]]> https://www.researchpad.co/article/N0357e062-01ca-43ac-b343-bcceb7a83a4c

Background

Scholarship plays a direct role in career advancement, promotion and authoritative recognition, and women physicians remain under-represented as authors of original research articles.

Objective

We sought to determine if women physician authors are similarly under-represented in commentary articles within high-impact journals.

Design/Setting/Participants

In this observational study, we abstracted and analysed author information (gender and degree) and authorship position from commentary articles published in three high-impact journals between 1 January 2014 and 16 October 2018.

Primary outcome measure

Authorship rate of commentary articles over a 5-year period by gender, degree, authorship position and journal.

Secondary outcome measures

To compare the proportion of men and women physician authorship of commentaries relative to the proportion of men and women physician faculty within academic medicine; and to examine the gender concordance among the last and first authors in articles with more than one author.

Results

Of the 2087 articles during the study period, 48% were men physician first authors compared with 17% women physician first authors (p<0.0001). Of the 1477 articles with more than one author, similar distributions were found with regard to last authors: 55% were men physicians compared with only 12% women physicians (p<0.0001). The proportion of women physician first authors increased over time; however, the proportion of women physician last authors remained stagnant. Women coauthored with women in the first and last authorship positions in 9% of articles. In contrast, women coauthored with men in the first and last author positions, respectively, in 55% of articles.

Conclusions

Women physician authors remain under-represented in commentary articles compared with men physician authors in the first and last author positions. Women also coauthored commentaries with other women in far fewer numbers.

]]>
<![CDATA[On the value of preprints: An early career researcher perspective]]> https://www.researchpad.co/article/5c784facd5eed0c4840072e6

Peer-reviewed journal publication is the main means for academic researchers in the life sciences to create a permanent public record of their work. These publications are also the de facto currency for career progress, with a strong link between journal brand recognition and perceived value. The current peer-review process can lead to long delays between submission and publication, with cycles of rejection, revision, and resubmission causing redundant peer review. This situation creates unique challenges for early career researchers (ECRs), who rely heavily on timely publication of their work to gain recognition for their efforts. Today, ECRs face a changing academic landscape, including the increased interdisciplinarity of life sciences research, expansion of the researcher population, and consequent shifts in employer and funding demands. The publication of preprints, publicly available scientific manuscripts posted on dedicated preprint servers prior to journal-managed peer review, can play a key role in addressing these ECR challenges. Preprinting benefits include rapid dissemination of academic work, open access, establishing priority or concurrence, receiving feedback, and facilitating collaborations. Although there is a growing appreciation for and adoption of preprints, a minority of all articles in life sciences and medicine are preprinted. The current low rate of preprint submissions in life sciences and ECR concerns regarding preprinting need to be addressed. We provide a perspective from an interdisciplinary group of ECRs on the value of preprints and advocate their wide adoption to advance knowledge and facilitate career development.

]]>
<![CDATA[Reliable novelty: New should not trump true]]> https://www.researchpad.co/article/5c6c75c4d5eed0c4843d014a

Although a case can be made for rewarding scientists for risky, novel science rather than for incremental, reliable science, novelty without reliability ceases to be science. The currently available evidence suggests that the most prestigious journals are no better at detecting unreliable science than other journals. In fact, some of the most convincing studies show a negative correlation, with the most prestigious journals publishing the least reliable science. With the credibility of science increasingly under siege, how much longer can we afford to reward novelty at the expense of reliability? Here, I argue for replacing the legacy journals with a modern information infrastructure that is governed by scholars. This infrastructure would allow renewed focus on scientific reliability, with improved sort, filter, and discovery functionalities, at massive cost savings. If these savings were invested in additional infrastructure for research data and scientific code and/or software, scientific reliability would receive additional support, and funding woes—for, e.g., biological databases—would be a concern of the past.

]]>
<![CDATA[A proposal for the future of scientific publishing in the life sciences]]> https://www.researchpad.co/article/5c6c75c2d5eed0c4843d013c

Science advances through rich, scholarly discussion. More than ever before, digital tools allow us to take that dialogue online. To chart a new future for open publishing, we must consider alternatives to the core features of the legacy print publishing system, such as an access paywall and editorial selection before publication. Although journals have their strengths, the traditional approach of selecting articles before publication (“curate first, publish second”) forces a focus on “getting into the right journals,” which can delay dissemination of scientific work, create opportunity costs for pushing science forward, and promote undesirable behaviors among scientists and the institutions that evaluate them. We believe that a “publish first, curate second” approach with the following features would be a strong alternative: authors decide when and what to publish; peer review reports are published, either anonymously or with attribution; and curation occurs after publication, incorporating community feedback and expert judgment to select articles for target audiences and to evaluate whether scientific work has stood the test of time. These proposed changes could optimize publishing practices for the digital age, emphasizing transparency, peer-mediated improvement, and post-publication appraisal of scientific articles.

]]>
<![CDATA[Long-term outcomes of an educational intervention to reduce antibiotic prescribing for childhood upper respiratory tract infections in rural China: Follow-up of a cluster-randomised controlled trial]]> https://www.researchpad.co/article/5c63396dd5eed0c484ae66e1

Background

Inappropriate antibiotic prescribing causes widespread serious health problems. To reduce prescribing of antibiotics in Chinese primary care to children with upper respiratory tract infections (URTIs), we developed an intervention comprising clinical guidelines, monthly prescribing review meetings, doctor–patient communication skills training, and education materials for caregivers. We previously evaluated our intervention using an unblinded cluster-randomised controlled trial (cRCT) in 25 primary care facilities across two rural counties. When our trial ended at the 6-month follow-up period, we found that the intervention had reduced antibiotic prescribing for childhood URTIs by 29 percentage points (pp) (95% CI −42 to −16).

Methods and findings

In this long-term follow-up study, we collected our trial outcomes from the one county (14 facilities and 1:1 cluster randomisation ratio) that had electronic records available 12 months after the trial ended, at the 18-month follow-up period. Our primary outcome was the antibiotic prescription rate (APR)—the percentage of outpatient prescriptions containing any antibiotic(s) for children aged 2 to 14 years who had a primary diagnosis of a URTI and had no other illness requiring antibiotics. We also conducted 15 in-depth interviews to understand how interventions were sustained.

In intervention facilities, the APR was 84% (1,171 out of 1,400) at baseline, 37% (515 out of 1,380) at 6 months, and 54% (2,748 out of 5,084) at 18 months, and in control facilities, it was 76% (1,063 out of 1,400), 77% (1,084 out of 1,400), and 75% (2,772 out of 3,685), respectively. After adjusting for patient and prescribing doctor covariates, compared to the baseline intervention-control difference, the difference at 6 months represented a 6-month intervention-arm reduction in the APR of −49 pp (95% CI −63 to −35; P < 0.0001), and compared to the baseline difference, the difference at 18 months represented an 18-month intervention-arm reduction in the APR of −36 pp (95% CI −55 to −17; P < 0.0001). Compared to the 6-month intervention-control difference, the difference at 18 months represented no change in the APR: 13 pp (95% CI −7 to 33; P = 0.21). Factors reported to sustain reductions in antibiotic prescribing included doctors’ improved knowledge and communication skills and focused prescription review meetings, whereas lack of supervision and monitoring may be associated with relapse. Key limitations were not including all clusters from the trial and not collecting returned visits or sepsis cases.

Conclusions

Our intervention was associated with sustained and substantial reductions in antibiotic prescribing at the end of the intervention period and 12 months later. Our intervention may be adapted to similar resource-poor settings.

Trial registration

ISRCTN registry ISRCTN14340536.

]]>
<![CDATA[Structure and belonging: Pathways to success for underrepresented minority and women PhD students in STEM fields]]> https://www.researchpad.co/article/5c3fa5ccd5eed0c484ca8b7b

The advancement of underrepresented minority and women PhD students to elite postdoctoral and faculty positions in the STEM fields continues to lag that of majority males, despite decades of efforts to mitigate bias and increase opportunities for students from diverse backgrounds. In 2015, the National Science Foundation Alliance for Graduate Education and the Professoriate (NSF AGEP) California Alliance (Berkeley, Caltech, Stanford, UCLA) conducted a wide-ranging survey of graduate students across the mathematical, physical, engineering, and computer sciences in order to identify levers to improve the success of PhD students, and, in time, improve diversity in STEM leadership positions, especially the professoriate. The survey data were interpreted via path analysis, a method that identifies significant relationships, both direct and indirect, among various factors and outcomes of interest. We investigated two important outcomes: publication rates, which largely determine a new PhD student’s competitiveness in the academic marketplace, and subjective well-being. Women and minority students who perceived that they were well-prepared for their graduate courses and accepted by their colleagues (faculty and fellow students), and who experienced well-articulated and structured PhD programs, were most likely to publish at rates comparable to their male majority peers. Women PhD students experienced significantly higher levels of distress than their male peers, both majority and minority, while both women and minority student distress levels were mitigated by clearly-articulated expectations, perceiving that they were well-prepared for graduate level courses, and feeling accepted by their colleagues. It is unclear whether higher levels of distress in women students is related directly to their experiences in their STEM PhD programs. The findings suggest that mitigating factors that negatively affect diversity should not, in principle, require the investment of large resources, but rather requires attention to the local culture and structure of individual STEM PhD programs.

]]>
<![CDATA[Why sampling ratio matters: Logistic regression and studies of habitat use]]> https://www.researchpad.co/article/5b60362d463d7e4090b7ce1d

Logistic regression (LR) models are among the most frequently used statistical tools in ecology. With LR one can infer if a species’ habitat use is related to environmental factors and estimate the probability of species occurrence based on the values of these factors. However, studies often use inadequate sampling with regards to the arbitrarily chosen ratio between occupied and unoccupied (or available) locations, and this has a profound effect on the inference and predictive power of LR models. To demonstrate the effect of various sampling strategies/efforts on the quality of LR models, we used a unique census dataset containing all the used roosting cavities of the tree-dwelling bat Nyctalus leisleri and all cavities where the species was absent. We compared models constructed from randomly selected data subsets with varying ratios of occupied and unoccupied cavities (1:1, 1:5, 1:10) with a full dataset model (ratio 1:31). These comparisons revealed that the power of LR models was low when the sampling did not reflect the population ratio of occupied and unoccupied cavities. The use of weights improved the subsampled models. Thus, this study warns against inadequate data sampling and highly encourages a randomized sampling procedure to estimate the true ratio of occupied:unoccupied locations, which can then be used to optimize a manageable sampling effort and apply weights to improve the LR model. Such an approach may provide robust and reliable models suitable for both inference and prediction.

]]>
<![CDATA[Training and Experience of Peer Reviewers: Authors' Reply]]> https://www.researchpad.co/article/5989db18ab0ee8fa60bcd89d ]]> <![CDATA[Toward an Understanding of the Environmental and Public Health Impacts of Unconventional Natural Gas Development: A Categorical Assessment of the Peer-Reviewed Scientific Literature, 2009-2015]]> https://www.researchpad.co/article/5989db38ab0ee8fa60bd3ca8

The body of science evaluating the potential impacts of unconventional natural gas development (UNGD) has grown significantly in recent years, although many data gaps remain. Still, a broad empirical understanding of the impacts is beginning to emerge amidst a swell of research. The present categorical assessment provides an overview of the peer-reviewed scientific literature from 2009–2015 as it relates to the potential impacts of UNGD on public health, water quality, and air quality. We have categorized all available original research during this time period in an attempt to understand the weight and direction of the scientific literature. Our results indicate that at least 685 papers have been published in peer-reviewed scientific journals that are relevant to assessing the impacts of UNGD. 84% of public health studies contain findings that indicate public health hazards, elevated risks, or adverse health outcomes; 69% of water quality studies contain findings that indicate potential, positive association, or actual incidence of water contamination; and 87% of air quality studies contain findings that indicate elevated air pollutant emissions and/or atmospheric concentrations. This paper demonstrates that the weight of the findings in the scientific literature indicates hazards and elevated risks to human health as well as possible adverse health outcomes associated with UNGD. There are limitations to this type of assessment and it is only intended to provide a snapshot of the scientific knowledge based on the available literature. However, this work can be used to identify themes that lie in or across studies, to prioritize future research, and to provide an empirical foundation for policy decisions.

]]>
<![CDATA[The Relationship of Previous Training and Experience of Journal Peer Reviewers to Subsequent Review Quality]]> https://www.researchpad.co/article/5989d9d3ab0ee8fa60b64d79

Background

Peer review is considered crucial to the selection and publication of quality science, but very little is known about the previous experiences and training that might identify high-quality peer reviewers. The reviewer selection processes of most journals, and thus the qualifications of their reviewers, are ill defined. More objective selection of peer reviewers might improve the journal peer review process and thus the quality of published science.

Methods and Findings

306 experienced reviewers (71% of all those associated with a specialty journal) completed a survey of past training and experiences postulated to improve peer review skills. Reviewers performed 2,856 reviews of 1,484 separate manuscripts during a four-year study period, all prospectively rated on a standardized quality scale by editors. Multivariable analysis revealed that most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training). Being on an editorial board and doing formal grant (study section) review were each predictors for only one of our two comparisons. However, the predictive power of all variables was weak.

Conclusions

Our study confirms that there are no easily identifiable types of formal training or experience that predict reviewer performance. Skill in scientific peer review may be as ill defined and hard to impart as is “common sense.” Without a better understanding of those skills, it seems unlikely journals and editors will be successful in systematically improving their selection of reviewers. This inability to predict performance makes it imperative that all but the smallest journals implement routine review ratings systems to routinely monitor the quality of their reviews (and thus the quality of the science they publish).

]]>
<![CDATA[How Criterion Scores Predict the Overall Impact Score and Funding Outcomes for National Institutes of Health Peer-Reviewed Applications]]> https://www.researchpad.co/article/5989d9fcab0ee8fa60b72836

Understanding the factors associated with successful funding outcomes of research project grant (R01) applications is critical for the biomedical research community. R01 applications are evaluated through the National Institutes of Health (NIH) peer review system, where peer reviewers are asked to evaluate and assign scores to five research criteria when assessing an application’s scientific and technical merit. This study examined the relationship of the five research criterion scores to the Overall Impact score and the likelihood of being funded for over 123,700 competing R01 applications for fiscal years 2010 through 2013. The relationships of other application and applicant characteristics, including demographics, to scoring and funding outcomes were studied as well. The analyses showed that the Approach and, to a lesser extent, the Significance criterion scores were the main predictors of an R01 application’s Overall Impact score and its likelihood of being funded. Applicants might consider these findings when submitting future R01 applications to NIH.

]]>
<![CDATA[How Do General Practitioners Conceptualise Advance Care Planning in Their Practice? A Qualitative Study]]> https://www.researchpad.co/article/5989daf1ab0ee8fa60bc150a

Objectives

To explore how GPs conceptualise advance care planning (ACP), based on their experiences with ACP in their practice.

Methods

Five focus groups were held with 36 GPs. Discussions were analysed using a constant comparative method.

Results

Four overarching themes in the conceptualisations of ACP were discerned: (1) the organisation of professional care required to meet patients’ needs, (2) the process of preparing for death and discussing palliative care options, (3) the discussion of care goals and treatment decisions, (4) the completion of advance directives. Within these themes, ACP was both conceptualised in terms of content of ACP and/or in terms of tasks for the GP. A specific task that was mentioned throughout the discussion of the four different themes was (5) the task of actively initiating ACP by the GP versus passively waiting for patients’ initiation.

Conclusions

This study illustrates that GPs have varying conceptualisations of ACP, of which some are more limited to specific aspects of ACP. A shared conceptualisation and agreement on the purpose and goals of ACP is needed to ensure successful implementation, as well as a systematic integration of ACP in routine practice that could lead to a better uptake of all the important elements of ACP.

]]>
<![CDATA[Ten simple rules to consider regarding preprint submission]]> https://www.researchpad.co/article/5989db5cab0ee8fa60be03b4 ]]> <![CDATA[Agreements and Discrepancies between FDA Reports and Journal Papers on Biologic Agents Approved for Rheumatoid Arthritis: A Meta-Research Project]]> https://www.researchpad.co/article/5989dadeab0ee8fa60bbac41

Background

Sponsors that seek to commercialize new drugs apply to the Food and Drug Administration (FDA) which independently analyzes the raw data and reports the results on its website.

Objectives

This study sought to determine if there are differences between the FDA assessments and journal reports on biologic agents developed for the treatment of rheumatoid arthritis.

Methods

Available data on FDA-approved drugs were extracted from the website, and a systematic literature search was conducted to identify matching studies in peer-reviewed medical journals. Outcome measures were the American College of Rheumatology response criteria ACR20 (efficacy) and withdrawal due to adverse events (safety). As effect size odds ratios were estimated for each active trial arm vs. control arm (i.e. for both sources: FDA and journal report), followed by calculation of the ratios of the FDA and journal report odds ratios. A ratio of odds ratios not equal to 1 was categorized as a discrepancy.

Results

FDA reports were available for 8 of 9 FDA-approved biologic agents for rheumatoid arthritis; all identified trials (34) except one were published in peer-reviewed journals. Overall, discrepancies were noted for 20 of the 33 evaluated trials. Differences in the apparent benefit reporting were found in 39% (24/61) pairwise comparisons and in 11 cases these were statistically significant; the FDA report showed greater benefit than the journal publication in 15 comparisons and lesser benefit in 9. Differences in the reported harms were found in 51% (28/55) pairwise comparisons and were statistically significant in 5. The “signal” in FDA reports showed a less harmful effect than the journal publication in 17 comparisons whereas a more harmful effect in 11. The differences were attributed to differences in analytic approach, patient inclusion, rounding effect, and counting discrepancies. However, no differences were categorized as critical.

Conclusion

There was no empirical evidence to suggest biased estimates between the two sources. Increased and detailed transparency in publications would improve the understanding and credibility of published results. Further, the FDA report was found to be a useful source when data are missing in the published report (i.e. reporting bias).

]]>