Essentials of Complementary and Alternative Medicine (June 1999)

CHAPTER 5. H
OW TO
 P
RACTICE
 E
VIDENCED
-B
ASED
 C
OMPLEMENTARY AND
 A
LTERNATIVE
 M
EDICINE
Wayne B. Jonas, Klaus Linde, and Harald Walach
Introduction
The Advent of Evidence-Based Medicine (EBM)
Variations in CAM Practice
Managing the Explosion of Medical Information
 
Knowledge Domains: What Is Needed for Clinical Practice?
 
Finding Good Information
 
What Is the Risk and Cost of the CAM Therapy?
 
Deciding on the Use of Outcomes Data or Randomized Trials in Practice
 
Assessing Study Quality: Is the Information Any Good?
 
Efficiency of EBM
 
Balancing Belief, Plausibility, and Evidence in Medical Decisions
 
Is the Diagnostic System Working?
Summary of Steps
Examples of Evidence-Based Approaches to CAM
Chapter References
INTRODUCTION
Previous chapters in this section have described the history of complementary and alternative medicine (CAM), an approach to discussing CAM with patients in the 
modern medical environment, and its implications for medical ethics and science. The topics dealt with in CAM will continue to be increasingly important for the health 
care system and for physicians in general. The establishment of the Office of Alternative Medicine at the U.S. National Institutes of Health in 1992 (now the National 
Center for Complementary and Alternative Medicine), although not the beginning of interest about CAM in this country, in many ways symbolizes the likely 
permanence of these topics. The fact that the first official entity established to address CAM in the West is located in a research organization, rather than a practice, 
licensing, or economic organization, demonstrates the coming-of-age of science in relation to medicine.
THE ADVENT OF EVIDENCE-BASED MEDICINE (EBM)
The idea that science can provide important information for making medical decisions has only recently started to become a reality. As early as 100 years ago, there 
were practically no scientific tools to provide medical information. It is interesting that the first double-blind experiment in the history of medicine was done by 
homeopaths in Nuremberg, Germany, in 1835. To end a public debate about the correctness of homeopathic allegations, an experiment was conducted in which 
volunteers were divided in two groups. The public surgeon of Nuremberg had a homeopathic pharmacist prepare Natrum muriaticum C30 and dispense it in labeled 
vials together with sham vials containing only water. The volunteers were to take the substances and report later on the observed effects. Although the results of the 
experiment were inconclusive, it was the first double-blind study in medical history (
1
).
Clinical trials are also a recent event. The first true experiment on humans, a randomized controlled trial, occurred only 50 years ago. The scientific basis for 
approving drugs for medical use through the U.S. Food and Drug Administration (FDA) is only approximately 20 years old. Only in the past 10 years has a serious 
emphasis on EBM—the use of up-to-date published research evidence to guide decisions in patient care—occurred. The official debut of EBM in the United States 
occurred in 1992 (the same year the Office of Alternative Medicine was established) with a series of articles published in the  Journal of the American Medical 
Association (
2
).
In conventional medicine, the need for an evidence-based approach to medical decision making has been evident for a long time. A similar need for CAM is equally 
evident (although not always acknowledged). The introduction, diffusion, adoption, and abandonment of procedures in medical practice are predominantly influenced 
by social prestige, power, or potential profit rather than by evidence of public benefit (
3
, 
4
 and 
5
).
Clinical experience is notoriously flawed for accurately judging the probability of risk and benefit. Global judgments by experts using informal methods are no better, 
often reflecting lack of rigor (
6
) and shifting recommendations based on specialty interests (
7
). The subjective and unreliable nature of clinical decision making and 
expert opinion is demonstrated by the wide variation of standard procedures in practice, even within relatively small geographic areas (
8
). The variation can be up to 
20-fold across wide geographic areas (
9
, 
10
). Part of this variation is caused by legitimate differences in data that come from research conducted on different 
populations and relate to different specialties (
11
). Another part of this variation is caused by errors and inconsistencies in clinical judgment, even within the same 
specialty (
12
).
VARIATIONS IN CAM PRACTICE
Complementary medicine has similar, and probably even more extreme, problems with clinical variation. In a systematic review of 15 randomized trials of acupuncture 
treatment of asthma, for example, very different treatment strategies were found (
13
). Only two trials by the same author used the same acupuncture points (
14
). The 
type of acupuncture treatment (like many CAM and conventional procedures) largely depends on the practitioner, and perhaps also on the patient.
Likewise, in a meta-analysis of clinical trials of homeopathy, the authors were unable to find three trials by different authors that used the same treatment approach for 
the same clinical condition (
15
). This is partly caused by the greater individualization of CAM therapies, which confounds the problem of accurately estimating the 
clinical effects. Continuous collection of local patient characteristics and outcome data for CAM practices may help make more precise estimates of potential benefits 
and harms as they relate to individual decision-making. Such data, although helpful in estimating the likelihood of benefit, cannot determine whether such effects are 
specific to the therapy. This information can come only through randomized controlled trials.
MANAGING THE EXPLOSION OF MEDICAL INFORMATION
From the perspective of EBM, CAM is but another topical addition to the current explosion of biomedical information. Clearly, no one can master all this information, 
so the need for evidence-based problem-solving skills is essential. The recurring call for both more generalist medical education and the development of 
problem-oriented medical training reflects efforts to provide such skills to medical students (
16
). The complete practice of EBM, however, would require information 
from all six knowledge domains discussed in 
Chapter 4
:
1. Patient preferences and meaning.
2. Mechanisms of action.
3. Safety and efficacy.
4. Treatment effect probabilities in the open clinical setting from observational and outcomes research.
5. Precise estimates of effects through systematic summaries and calculations of confidence intervals when possible.
6. Demonstration of utility and benefit under normal health service conditions examining the impact of access, feasibility, and costs.
In addition, a practitioner would need to be able to assess the relevance of all this information to the particular clinical case at hand, with all its unique nuances and 
circumstances. Rarely will any intervention have such a breadth and depth of information available, and few, if any, practitioners or even groups of experts could 
obtain and analyze all such information.

Knowledge Domains: What Is Needed for Clinical Practice?
Fortunately, most clinical decisions can be made with information from two of the six domains listed above, through randomized controlled trials and outcomes 
research. This presumes that the practitioner is well trained clinically and has the communication and interpersonal skills needed to assess and incorporate patient 
preferences and relevance into the decision-making process. This three-legged stool–clinical expertise, patient relevance, and research evidence–is the foundation 
for evidence-based clinical decisions (
17
, 
18
).
The level of EBM skill used is currently up to the practitioner. It can vary from using pre- evaluated literature to quick summary sheets to week-long continuing medical 
education courses on how to practice EBM. With concerted effort, it is possible for physicians in general practice to base the vast majority of clinical decisions in 
conventional medicine on good evidence (
19
). Once the habit of EBM is established, the practitioner can incorporate CAM topics into this process. 
Table 5.1
 lists 
some bare-bones users guidelines developed by Haynes and colleagues for appraising the validity of the most common types of clinical studies (
20
).
Table 5.1. Minimum Guidelines for Assessment of Study Validity
The physician may be approached by (or discover on questioning) patients already using CAM. Or patients may request information about CAM practices being 
considered. Patients come to physicians to get their opinion about a therapy because of their clinical expertise, or to ask whether there is any research evidence 
supporting the practice. The physician can fulfill this professional role if he or she evaluates the medical literature for the relevance of the therapy to the patient's 
situation. As discussed in 
Chapter 2
 of this book, the first step is to determine why the patient is seeking alternative therapy and whether conventional medicine has 
been inadequate in some way or simply not been adequately applied. If proven conventional therapy has not been adequately applied, then this is explained to the 
patient and rectified. If the patient has a strong preference for CAM or if conventional medicine does not offer adequate therapy, the physician can seek out evidence 
for alternatives.
Finding Good Information
With computerized access, a ready source of credible information is possible. Information on CAM for assisting with evidence-based decisions for patient care is 
becoming increasingly available. A number of groups are working to collate and produce CAM-specific databases; for example, the National Library of Medicine and 
the National Center for Complementary and Alternative Medicine (National Institutes of Health) in the United States, the Research Council for Complementary 
Medicine in England, the BMBF Coordinating Group in Germany, the international Cochrane Collaboration, and some universities and private organizations. 
Practitioners can anticipate that, within the next few years, a number of comprehensive and easily accessible sources of quality CAM literature will become available. 
Some on-line sources of high-quality information that are indispensable for obtaining up-to-date information are listed in 
Table 5.2
 as well as in 
Table 2.5
 in Chapter 
2.
Table 5.2. Some On-line Sources of Quality Medical Information
NAME
INTERNET ADDRESS
SOURCE
Sources of Primary Literature
 
MEDLINE
www.medline.nlm.nih.gov
 or 
www.ncbi.nlm.nih.gov/PubMed
National Library of Medicine PubMed (free Internet 
access to MEDLINE)
CAM Citation Index (includes the Cochrane 
Database of CAM–controlled trials; see 
Appendix 4
)
www.altmed.od.nih.gov/nccam/resources/cam-ci
Office of Alternative Medicine, NIH (now the National 
Center for Alternative & Complementary Medicine)
CISCOM
www.gn.apc.org/rccm/ciscom.html
Research Council for Complementary Medicine, U.K.
Sources of Secondary Literature
The Cochrane Library
www.nihs.go.jp/acc/cochrane/revabst/ccabout.htm
Cochrane Library's field group in CAM (CD-ROM 
available also)
Best Evidence Selection
www.webcom.com/mjljweb/jrnlclb/index.htm
ACP Journal Club and Evidence-Based Medicine 
(CD-ROM available also)
Agency for Health Care Policy and Research 
(AHCPR)
www.ahcpr.gov
Evidence-Based Practice Guidelines Database
Focus on Alternative and Complementary Therapies 
(FACT)
www.exeter.ac.uk/FACT/
Quarterly Journal Club for CAM
S
EARCHING THE
 L
ITERATURE
If a physician uses the approach outlined in 
Chapter 3
 (see Table 3.3) and determines that an alternative therapy is already being used by the patient or may be 
useful to the patient, the physician can search for data on the effects of CAM for that condition. Three types of information should be sought on the CAM therapy:
1. Meta-analyses, or systematic reviews.
2. Randomized controlled trials.
3. Observational or prospective outcomes data.
Table 5.3
 lists the most common search categories that capture this type of data from standard MEDLINE-type databases. If relevant citations are not found using the 
main terms, one can try the other terms listed under the main terms in 
Table 5.3
 to see if they yield studies in that category. Other types of data (e.g., from controlled 
studies, cross-sectional, or retrospective epidemiological studies; cost and health service information; laboratory research and editorials; letters; and so forth) are 
generally less useful for evidence-based decision making. Limiting the literature search to the evidence domains of  attribution and association (which a search in the 

three aforementioned categories of data will do) focuses on the areas most relevant for making decisions in practice.
Table 5.3. Citation Categories and Terms for Searching Databases for Clinical Evidence
When No Research Information Is Found
If the search yields no information from relatively comprehensive database sources, one can reasonably assume that there is scant or no good clinical evidence for 
the CAM therapy with that condition. Knowing that no good information exists is useful, and patients are often grateful for this effort. The physician then knows with 
reasonable confidence that any decisions to use or discourage the CAM therapy are not evidence based and must rely on the physician's own clinical judgment or 
other information.
Not having access to information about CAM practices and products can represent a risk to quality patient care, because patients will often use the product or practice 
without their physician's knowledge or without reliable information. A physician's skill and ability to search for and evaluate reliable quality data can assist both 
physician and patient in making the decision to stop or go forward with a treatment. Providing information about research on CAM practices and combining it with 
good clinical judgment are two of the main services patients seek from physicians.
When Research Information is Found
If a practitioner uses the proper search terms and information sources and finds clinical data documenting the effects or efficacy of a CAM practice for the condition of 
interest, a more evidence-based decision can be made. This information can then be communicated to the patient in the clinical setting. Patients are grateful for this 
effort, because they often seek out a CAM practice after receiving information derived from sources that may not be reliable , such as articles in magazines, 
newsletters, the Internet, radio shows involving advocates of a practice, or the urging of friends and family to try an alternative therapy.
What Is the Risk and Cost of the CAM Therapy?
When reasonable clinical data about a CAM practice are found in meta-analyses and systematic reviews, randomized controlled trials, or observational studies, the 
physician can then decide on its validity and relevance to the patient at hand. The amount and type of evidence required to use or reject the use of each CAM therapy 
depend on a number of factors. However, there are two key screening issues to be considered before evaluating the evidence: the degree of direct risk or toxicity, and 
the cost of the therapy (
14
). Many CAM practice involve over-the-counter (OTC) supplements—medications or self-care techniques that are reasonably safe and 
inexpensive when properly monitored. The physician should distinguish between CAM practices that are low in both risk and cost and those that are higher in risk and 
cost. Practices that are low in risk and often low in cost for most patients include:
OTC homeopathic medications.
Properly delivered acupuncture and manipulation.
Mind–body techniques, such as meditation, relaxation, and biofeedback.
Vitamin and mineral supplementation that is below known toxic ranges and is without potential vitamin/drug interactions.
The financial status of the patient and reimbursement status of the practice should be considered with the patient. When a practice is low risk and low cost, the 
physician and patient want to know the probability of benefit from its use. This information can be obtained from either randomized trials or good observational and 
outcomes studies that are not randomized.
Deciding on the Use of Outcomes Data or Randomized Trials in Practice
Although data from observational and outcomes research may provide sufficient evidence for clinical decisions on low-risk, low-cost practices, data from randomized 
controlled trials are required for practices that have significant potential risk or cost. This includes practices such as:
Herbal therapies.
High-dose vitamins and minerals.
Blood-derived vaccine products.
Instrumentation such as colonics.
Intravenous administration of substances such as hydrogen peroxide and ozone.
Repeated visits or long distance travel for treatments.
These types of practices require specific risk–benefit comparison to either no treatment, placebo, or a standard conventional treatment. In addition, some products 
and practices are extremely expensive, require the patient to travel to obtain services, and are not reimbursable. Even if a therapy is relatively harmless, the high cost 
and inconvenience of such therapies make the use of noncontrolled information inadequate for decision making. Depending upon the category of risk and cost of the 
potential practice, the physician can then obtain and examine the abstracts or research articles found (often also online). For high-risk, high-cost interventions, the 
physician should rely on randomized controlled trials only.
In summary outcomes research, randomized controlled trials, and meta-analyses of controlled trials can provide information about the probability of benefit from a 
practice. Outcomes data alone can often provide evidence for clinical decisions if the practice is safe and low-cost (
21
). For example, if outcomes studies show that a 
nontoxic and low-cost OTC homeopathic remedy has a 75% probability of improving allergic rhinitis (
22
), this information can assist the physician and patient in 
deciding on whether to try it. If, however, the CAM practice is high risk or high cost (e.g., intravenous H
2
O
2
 for allergic rhinitis), a stricter level of evidence on benefits 
and risks from randomized controlled trials is required (
23
).
Assessing Study Quality: Is the Information Any Good?
Once data are found, the physician should apply the minimum available assessment guidelines (see 
Table 5.1
) for evaluating the research. Three criteria can be 
checked quickly:
1. Blind and random allocation of subjects to comparison groups (in controlled clinical trials).
2. Clinical relevance and reliability of the outcome measures.
3. The number of subjects entered in and then analyzed at the end of the study.

How to apply these criteria to any study set is described in more detail below.
T
REATMENT
 A
LLOCATION
When assessing a treatment trial, the physician should examine whether there was adequate concealment of allocation from one treatment group to another, even if 
the treatment itself is not blindable. If the investigators did not know which subjects were going to be assigned to treatment groups, or if a random numbers table was 
used to make subject assignments, the trial's validity is more likely.
R
ELEVANT AND
 R
ELIABLE
 O
UTCOMES
Second, were the outcomes important? Objective outcomes should have been used whenever possible, but not at the expense of more patient-relevant measures. 
Surrogate outcomes–those easy to measure and used as a marker for a disease or illness–should be suspect unless they are known to be tightly linked to the 
relevant clinical outcomes. When the outcome involves a subjective condition (e.g., pain), the investigators should have checked for reliability of measurement. This 
means that the outcome was measured several times (either in this trial or previous tests) and consistently came up with the same results.
W
HAT
 W
AS
 A
NALYZED
?
Finally, was the number of patients analyzed comparable with the number who entered into the study at the beginning—that is, what was the dropout rate? Dropout 
rates of more than 20% in most studies should make the value of the trial information suspect. Ideally, the analysis should be done on all individuals who entered into 
the trial from the beginning of the study. This is called an  intention-to-treat analysis. Some assessment of the number of outcomes measured should also be made. 
The practitioner should ascertain whether the outcomes reported in the study were the original focus of the study (part of the main hypothesis), or whether there were 
multiple outcome measures assessed, and only those that were statistically significant were reported. If the latter is true, the value of information from the trial should 
again be suspect.
O
BSERVATIONAL
 S
TUDIES
These same minimum study quality criteria apply to observational studies, except that concealed random allocation to treatment and comparison groups (see 
Guideline 1, 
Table 5.1
) does not apply. In observational or outcomes studies, the physician should also carefully examine the magnitude of benefit. Two questions 
should be asked:
1. Was the probability of benefit reported worth the inconvenience, risk of adverse effects, and costs of the treatment?
2. Were confidence intervals reported, and were they narrow or broad?
Confidence intervals are the range of minimum-to-maximum benefit expected in 95% of similar studies. If the confidence intervals are narrow, the physician can be 
more confident that the likelihood of benefit found with the patient will be close to that reported. If confidence intervals are broad, one can expect that the likelihood of 
benefit from the treatment will be unpredictable.
If the answers to these quality questions show that there are marked quality flaws in the studies retrieved, the physician can conclude that the evidence for the 
practice is insufficient and should not be used as a basis for clinical decisions.