Massage Therapy Canada

Features Collaboration Practice
What are Systematic Reviews

Health care decisions are increasingly being made on research-based evidence, rather than on expert opinion or clinical experience alone.

September 24, 2009  By Bodhi G. Haraldsson RMT

Health care decisions are increasingly being made on research-based evidence, rather than on expert opinion or clinical experience alone.

Systematic reviews represent a rigorous method of compiling scientific evidence to answer questions regarding health care issues of treatment, diagnosis, or preventive services. Systematic reviews increasingly form the basis for making individual and policy-level health care decisions.

The massage therapy profession needs to play an active role in the production of quality systematic reviews to summarize, appraise, and communicate the results and implications of research on massage therapy to the public, other professions and health care decision makers. If not, the massage therapy profession risks being
left out of the loop when it comes to important policy decisions, both at the government level and at third party payers such as insurance companies.

The massage therapy profession has historically relied on the opinions of experts in the field, and information handed down from previous generations of therapists, as the basis for the understanding of how the therapy works, how to use it and when not to use it.


Some of these beliefs have been shown to be correct while others have been shown to have little or no effect in trials. In today’s world of evidence-based health care, the reliance only on expert opinion and unsupported claims is unnecessary given the tools that are available to the practising therapist. These tools come to us from the evidence-based medicine field.

Evidence-based medicine is the conscientious, explicit, and judicious use of the best current evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available clinical evidence (DL Sackett et al., 1996).

This is achieved by asking questions, finding and evaluating all relevant data, and using that information to guide decisions. 

Use of evidence to help define best practices rather than to maintain existing practices will allow the profession to be able to take advantage of new knowledge developments, better our patient care and prevent the practise of massage therapy from rapidly becoming out of date.

Evidence-based medicine is not with out pitfalls, according to Naylor (1995): “Even good evidence can lead to bad practice if applied in an unthinking and unfeeling way.”

Attempting to locate all relevant studies on any given topic can be a daunting task for most
clinicians. In the data base MEDLINE, the US National Library of Medicine indexes important biomedical literature from more than 4000 journals. MEDLINE has over 5,000,000 human citations dating back to 1966 (Hunt, McKibbon 1997). Due to this enormous size, it takes teams of professionals to find and identify the relevant studies. For the busy clinician the best option would be to utilize systematic reviews.

In 1996 the British National Health System (NHS) Centre for Reviews and Dissemination stated, “Systematic review is a scientific tool which can be used to summarize, appraise, and communicate the results and implications of otherwise unmanageable quantities of research.”

Another definition was described by Booth & Haines, (1998) who define systematic review as the “process by which similar studies, identified from a comprehensive trawl of numerous sources, are summarized in easy-to-read graphical or tabular form and then their collective message or ‘bottom line’ presented, together with implications for practice and future research.”     

Although the term ‘meta-analysis’ is often used interchangeable with ‘systematic review,’ it is actually a statistical technique used to combine the results of several studies addressing the same question into a single summary measure (Khan et al., 2000).       
A systematic review is an overview of primary studies that uses explicit and reproducible scientific methods aimed at reducing random and systematic errors of bias in the review and its outcomes.

Meta-analysis is the main method used in systematic reviews to categorize and quantitatively summarize data to provide more precise estimates of the effects of health care  variables than those estimates derived from the single studies alone.

A systematic review may or may not include statistical meta-analysis (Clarke M, Oxman AD, eds. Cochrane Reviewers’ Handbook 4.1, 2000, Mulrow CD. 1997). 

This is what differentiates high quality systematic review from the traditional narrative review, “that has been criticized for being haphazard and biased, subject to idiosyncratic impression of the individual reviewer” (Mulrow CD. 1987). 

Systematic reviews, as with other types of reviews, have an inherent bias. In 2000 Sutton AJ et al. wrote: “Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the final conclusions.” Due to this bias, a good quality review has clear reporting of information obtained using rigorous reproducible scientific methods.

Regardless of the source, all systematic reviews (like all types of research evidence) require critical appraisal to determine their validity and to establish whether and how they will be useful in practice (Hunt, McKibbon 1997).

To date the leading organization in the systematic review field has been the Cochrane Collaboration. The collaboration was founded in 1992 as a response to challenge issued by Archie Cochrane, a British epidemiologist wrote in 1972, “It is surely a great criticism of our profession that we have not organized a critical summary, by speciality or subspecialty, adapted periodically, of all relevant randomized controlled trials.”  

Compared to systematic reviews published in paper-based journals, The Cochrane Collaboration reviews “appear to have greater methodological rigor and more frequent update” (Jadad et al 1998).

To safeguard the rigour and relevance of The Cochrane Reviews, the Cochrane Collaboration has several levels of peer review, including assessments of protocols by editors and reviewers, and evaluations of the reviews by methodology and content experts and by potential end users.

This review process makes it less susceptible to bias and methodological flaws. The Cochrane Library holds a rapidly increasing collection of over 1000 systematic reviews of clinical effectiveness on various topics.

The systematic reviews are produced through a global collaboration based in the United Kingdom. The Collaboration aims to help people make well-informed decisions about health care through the preparation, maintenance, and promotion of accessible systematic reviews of the effects of health care interventions (Cochrane Brochure).   
For clinicians the only guideline that has been proven to distinguish the good from the bad in systematic reviews is the one published by Oxman and Guyatt in the Journal of Clinical Epidemiology 1991.

The table below summarizes the benefits of systematic review as described by Mulrow (1994; as cited in K Stevens, 2001) .

References for part one of this feature can be found at

A systematic review:
• Reduces large quantities of information into a manageable form;
• Establishes generalizabilty across participants, across settings, and treatment variations
    and different study designs;
• Assesses consistency and explains inconsistencies of relationships across studies;
• Increases power in suggesting the cause and effect relationship;
• Reduces bias from random and systemic error, improving true reflection of reality;
• Integrates existing information for decision about clinical care,
economic decision, future research design, and policy information;
• Increases efficiency in time between research and clinical implementation;
• Provides format for better continuous updates of new evidence.

Print this page


Stories continue below