Training for and methods used in authoring

Training

Cochrane runs regular workshops and provides training both at these and online for all authors. Online tutorials are available to take authors through each stage of intervention review development and include:

'Systematic reviews - an introduction' and 'Writing a Cochrane protocol'.

Please click here to see the full availability of these.

For Diagnostic Test Assay review training, please contact the DTA Working Group. 

The editiorial base is also happy to provide training as required by authors.  Please contact us for advice at any time.

Further details of the methods employed in completing a Cochrane Systematic review are given below and links to all resources can be found within the panel on the right. 

Search strategies

Access to Specialised Register by reviewers

The Information Specialist (CIS) will help to provide the review authors with a list of potential trial references identified from the Cochrane Vascular Specialised Register and from searching the most recent issue of the Cochrane Library. Review authors do not have direct access to the Specialised Register except via the Cochrane Central Register of Controlled Trials (CENTRAL), in the Cochrane Library. These lists of potential trial references are updated every three months until a draft review is received by the editorial base. Annual lists are sent to authors of published reviews.

Additional search strategies

Authors are encouraged to search electronic databases, specialised journals, conference proceedings, the reference lists in trial reports and in other published reviews relevant to their topic area, and the Science Citation Index to identify articles that have cited the studies included in the review. Authors are also encouraged to contact pharmaceutical companies to obtain data on unpublished studies.

Study selection

Cochrane Vascular asks that authors restrict their reviews to randomised controlled trials. For some interventions, only quasi-randomised or controlled clinical trials may be available. Selection of studies for inclusion in reviews should be done independently by more than one review author. Disagreements should be resolved by discussion and where possible, by a third author.

Online module:  Searching for studies for a Cochrane intervention review.

Assessment of methodological quality

Cochrane Vascular recommends that assessment of trial quality should done independently by more than one review author. Disagreements should be resolved by discussion and where necessary, by a third author. Authors of reviews are referred to Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions 5.1.0 (updated March 2011) for more detailed information on the 'Risk of bias' tool. Quality assessment checklists are provided by Cochrane Vascular.

Data collection

Cochrane Vascular recommends that data extraction should be done independently by more than one review author. Disagreements should be resolved by discussion and where necessary, by a third author. Further information should be sought from the authors of published trials. Unpublished data should always be identified as such and its source acknowledged.

Online module:  Collecting data for a review.

Analysis

Authors of reviews are referred to Chapter 9 of the Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011] for information on analysing and presenting results in reviews. Data should be entered into RevMan 5.3 by at least two authors using the double data entry function.

Prior to publication all reviews are examined by a statistician for suitability of statistical techniques applied to the data. The Cochrane Vascular statistician has reccommended the following references for authors as they help clarify some common issues:

Absence of evidence is not evidence of absence. Altman and Bland. BMJ 1995; 311:485

Interpreting and reporting clinical trials with results of borderline significance. Hackshaw and Kirkwood. BMJ 2011; 343:d3340

A summary of the statistical guidelines within the Cochrane Handbook for Systematic Reviews of Interventions is given below.

Summary statistics: dichotomous outcomes (Handbook Section 9.2.2)
Cochrane Vascular recommends calculation of the relative risks (risk ratios) because of their ease of interpretation. However, in some circumstances there may be good reasons for preferring a different statistic, e.g. the Peto odds ratio appears to perform best when data are very sparse.

Summary statistics: continuous outcomes (Handbook Section 9.2.3)
Cochrane Vascular recommends calculation of the mean difference (MD) if the outcomes are measured in a standard way between trials. Standard mean difference (SMD) should be used only when necessary, i.e. when outcomes are conceptually the same but measured in different ways (e.g. pain scores using different scales).

All continuous outcome data require a mean and standard deviation for each group from each trial. Often reports present the standard error of the mean (SEM) rather than the standard deviation. This is much smaller, and will, if used in a meta-analysis, give far too much weight to that study. If one study appears to have a much smaller standard deviation than the others, it may be that the SEM has been erroneously reported as the standard deviation. SEM can be converted to standard deviation by multiplying it by the square root of the sample size. If standard deviations are not reported, it may be possible to calculate them from statistics given in the paper: the Handbook (section 16.1.2) presents methods for doing this.

Fixed-effect model versus random-effects analysis: The choice between fixed and random effects is not straightforward and there is considerable debate among statisticians about when each method is preferable. There is inevitably an element of subjectivity in the choice, but it is important that whatever decision is made can be justified.

Fixed-effect analysis may be used if all of the trials in the analysis are sufficiently similar that it is a reasonable assumption that the underlying effect size is the same for all of them. If this assumption is not reasonable, use random-effects analysis to obtain an overall summary, or do not combine the trials. If hetereogeneity has been detected, it may be inapprorpiate to combine results to produce a single summary measure. Results of the chi-squared heterogeneity test should not be used to guide the choice of statistical method.

Subgroup analyses:
It is necessary to keep subgroup analyses to a minimum to avoid generating spurious results. Therefore, each subgroup analysis should have a clear rationale, which should be explained in the background, pre-specified in the protocol, and only these subgroup analyses should be conducted. Subgroup analyses have great potential to be misleading so their results should be interpreted with caution.  

Sensitivity analysis: A minimum set of sensitivity analyses for authors to perform includes repeating the analysis:

1. excluding studies of lower methodological quality;
2. excluding any unpublished studies;
3. excluding very large studies (if present);
4. excluding other types of studies, depending on the degree to which there were choices about the inclusion/exclusion criteria (e.g. with/without trials that had different age groups, different dosages);
5. using the risk difference/odds ratio.

We recommend constructing a funnel plot where a reasonable number of studies have been included in a review, to examine the possibility of publication bias.

Outcomes with no data: Authors are encouraged to include outcomes of interest outlined in the protocol in the analysis even when no data are available.

Online modules:  Meta-analysis - an introduction; and Heterogeneity - an introduction.

Reporting of results

Cochrane Vascular has no written policy on the reporting of results. Review authors are recommended to follow the guidance given in the Cochrane Handbook for Systematic Reviews of Interventions. Where possible, review authors should comment on the applicability of their findings, and include any information relevant to health-care decision making, such as adverse events and adverse effects, prescribing data and costs.