Training for and methods used in authoring

Training

Cochrane runs regular workshops and provides training online for all authors. Online tutorials are available to take authors through each stage of intervention review development.

Please click here to see the full availability of these.

For Diagnostic Test Assay review training, please contact the DTA Working Group. 

Cochrane Vascular editiorial base is also happy to provide training as required by authors.  Please contact us for advice at any time.

Further details of the methods employed in completing a Cochrane Systematic review are given below.  Please also see our resources page.  

Search strategies

The Cochrane Vascular Information Specialist aims to identify all relevant RCTs regardless of language or publication status (published, unpublished, in press or in progress). The Information Specialist will search the Cochrane Vascular Specialised Register, Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, and CINAHL. The Information Specialist will also search the WHO International Clinical Trials Registry Platform and Clinicaltrials.gov. Additional databases such as LILACS or PEDro may be searched depending on the topic of the review.

Additional searches

Authors are encouraged to search specialised journals, conference proceedings, the reference lists in trial reports and other published reviews relevant to their topic area, and the Science Citation Index to identify articles that have cited the studies included in the review. Authors are also encouraged to contact pharmaceutical companies to obtain data on unpublished studies.

Study selection

Cochrane Vascular asks that authors restrict their reviews to randomised controlled trials. For some interventions, only quasi-randomised or controlled clinical trials may be available. Selection of studies for inclusion in reviews should be done independently by more than one review author. Disagreements should be resolved by discussion and where possible, by a third author.

Chapter 4: Searching for and selecting studies

Assessment of methodological quality

Cochrane Vascular recommends that assessment of trial quality should done independently by more than one review author. Disagreements should be resolved by discussion and where necessary, by a third author. Authors of reviews are referred to Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions 5.1.0 (updated March 2011) for more detailed information on Cochrane's risk of bias tool. Quality assessment checklists are provided by Cochrane Vascular.

Data collection

Cochrane Vascular recommends that data extraction should be done independently by more than one review author. Disagreements should be resolved by discussion and where necessary, by a third author. Further information should be sought from the authors of published trials. Unpublished data should always be identified as such and its source acknowledged.

Chapter 5: Collecting data

Analysis

Data should be extracted from studies and entered into Revview Manager by at least two authors.  

Chapter 9: Summarizing study characteristics and preparing for synthesis

Chapter 10: Analysing data and undertaking meta-analyses

Prior to publication all reviews are examined by a statistician for suitability of statistical techniques applied to the data. The Cochrane Vascular statistician has reccommended the following references for authors as they help clarify some common issues:

Absence of evidence is not evidence of absence. Altman and Bland. BMJ 1995; 311:485

Interpreting and reporting clinical trials with results of borderline significance. Hackshaw and Kirkwood. BMJ 2011; 343:d3340

Cochrane Vascular recommends calculation of the relative risks (risk ratios) because of their ease of interpretation. However, in some circumstances there may be good reasons for preferring a different statistic, e.g. the Peto odds ratio appears to perform best when data are very sparse.

Summary statistics: continuous outcomes (Handbook Section 9.2.3)
Cochrane Vascular recommends calculation of the mean difference (MD) if the outcomes are measured in a standard way between trials. Standard mean difference (SMD) should be used only when necessary, i.e. when outcomes are conceptually the same but measured in different ways (e.g. pain scores using different scales).

All continuous outcome data require a mean and standard deviation for each group from each trial. Often reports present the standard error of the mean (SEM) rather than the standard deviation. This is much smaller, and will, if used in a meta-analysis, give far too much weight to that study. If one study appears to have a much smaller standard deviation than the others, it may be that the SEM has been erroneously reported as the standard deviation. SEM can be converted to standard deviation by multiplying it by the square root of the sample size. If standard deviations are not reported, it may be possible to calculate them from statistics given in the paper: the Handbook (section 16.1.2) presents methods for doing this.

Fixed-effect model versus random-effects analysis: The choice between fixed and random effects is not straightforward and there is considerable debate among statisticians about when each method is preferable. There is inevitably an element of subjectivity in the choice, but it is important that whatever decision is made can be justified.

Fixed-effect analysis may be used if all of the trials in the analysis are sufficiently similar that it is a reasonable assumption that the underlying effect size is the same for all of them. If this assumption is not reasonable, use random-effects analysis to obtain an overall summary, or do not combine the trials. If hetereogeneity has been detected, it may be inapprorpiate to combine results to produce a single summary measure. Results of the chi-squared heterogeneity test should not be used to guide the choice of statistical method.

Subgroup analyses:
It is necessary to keep subgroup analyses to a minimum to avoid generating spurious results. Therefore, each subgroup analysis should have a clear rationale, which should be explained in the background, pre-specified in the protocol, and only these subgroup analyses should be conducted. Subgroup analyses have great potential to be misleading so their results should be interpreted with caution.  

Sensitivity analysis: A minimum set of sensitivity analyses for authors to perform includes repeating the analysis:

1. excluding studies of lower methodological quality;
2. excluding any unpublished studies;
3. excluding very large studies (if present);
4. excluding other types of studies, depending on the degree to which there were choices about the inclusion/exclusion criteria (e.g. with/without trials that had different age groups, different dosages);
5. using the risk difference/odds ratio.

We recommend constructing a funnel plot where a reasonable number of studies have been included in a review, to examine the possibility of publication bias.

Outcomes with no data: Authors are encouraged to include outcomes of interest outlined in the protocol in the analysis even when no data are available.

Reporting of results

Review authors should comment on the applicability of their findings, and include any information relevant to health-care decision making, such as adverse events and adverse effects, prescribing data and costs.

Chapter 14: Completing ‘Summary of findings’ tables and grading the certainty of the evidence

Chapter 15: Interpreting results and drawing conclusions