Critique of the NIH Consensus Conference on Acupuncture
Wallace I. Sampson, MD, FACP
On November 3-5, 1997, the National Institute of Drug Abuse (NIDA) held what it called a Consensus Conference on Acupuncture. The meeting was set up by Alan Trachtenberg, M.D., a former acting director of the Office of Alternative Medicine (OAM) and a strong advocate of acupuncture. The conference was co-sponsored by the Office of Medical Applications of Research (OMAR) with the OAM in a supporting role.
The first question that arose after viewing the speaker program was why there was an absence of speakers known to have done acupuncture research but who had obtained negative results. In 1986, a review of acupuncture research by Vincent and Richardson revealed that a majority of evaluable research papers showed essentially no significant effect from acupuncture for pain, when compared to placebo or inactive methods . Two analyses of the research in 1988 and 1990 showed that the best quality papers were almost uniformly negative, and the weakest or most poorly performed studies were mostly positive [2,3]. The Consensus Conference did not invite or present authors of those studies. Not much had changed since that review, although a number of studies were subsequently done for other conditions, such as asthma and nausea. Yet even those studies showed mixed results.
What of Balance?
The organizing committee was made up largely of NIH staff and extramural members interested in "alternative" methods. Unless particular care was taken to assure a fair balance, the committee whose members were interested in or advocates of unproven methods would be expected to recommend speakers favorable to the subject. There was no sign of such caution. The panel that was asked to evaluate the presentations also seemed to be weighted toward social advocacy rather than of evaluative science. A sampling of the panel suggests its leanings:
- Philip Greenman, DO, College of Osteopathic Medicine, East Lansing, Michigan
- Stephen P. Jiang, ACSW, Executive Director of the Association of Asian Pacific Community Health Organizations, Oakland, California
- Lawrence Kushi, ScD, epidemiologist of the University of Minnesota (son of Michio Kushi, popularizer of the macrobiotic diet)
- Philip Lee, MD, former Undersecretary of HEW (HHS) for Health
- Daniel Moerman, PhD, Professor of Anthropology of the University of Michigan
- Jorge Rios, MD, of the International Health Care Consulting Group
- Leonard Wisneski, MD, of American Whole Health, Bethesda, Maryland
Dr. Lee is author of two books justifiably critical of pharmaceutical industry practices. The chairman was David Ramsey, President of the University of Maryland, Baltimore, which received ongoing grants exceeding $1 million for its pain and acupuncture program from the Laing Foundation of the U.K. and from the OAM itself. The one best-qualified expert on the panel was Howard Fields, MD, PhD, a pain physiologist from the University of California, San Francisco, who did not attend because of illness. I spoke with Dr. Fields after the conference; he did not support the recommendations and stated that the best understanding of acupuncture's perceived effects is as a placebo.
The lack of critical, scientific thinking was apparent in the panel's report, which was 16 pages long. It obviously composed before the conference and changed somewhat after the presentations. Despite the uneven literature and the lack of firm evidence to support the conclusions, the consensus statement panel recommended acupuncture for musculoskeletal pain, some headaches, and nausea. It recommended use for nausea due to chemotherapy based on only three papers.
The statement conceded that placebo effects may be operative in acupuncture effects yet accepted the proposition that acupuncture has more specific, non-placebo physiologic effects as well. In support of that position, the presentations included measurements of chemical changes in brain and nervous tissue (endorphin increase in cerebrospinal fluid, changes in neuro-amine transmitter concentrations, and changes in single proton emission computed tomography (SPECT). However, to my knowledge, such changes have not been demonstrated to be specific to acupuncture. In other words, changes from acupuncture may be—and probably are—non-specific.
The report almost admitted as much by acknowledging that acupuncture is difficult to evaluate by double-blind controlled studies because sham acupuncture produces similar changes. The report then used a peculiar twist of logic to arrive at its recommendations. It concluded that since even sham acupuncture has effects, studies showing no difference between "real" and sham cannot be relied upon to show any lack of effect. In other words, one would never be able to disprove existence of acupuncture's effects. Those familiar with Poppers definitions of science will recognize that position as untestable.
Most experienced researchers, logicians, and even people in the street, would conclude otherwise—that if "sham" points can be almost anywhere on the body, and if "real" acupuncture and "sham" acupuncture show no consistent, significant differences, then why use "real" acupuncture at all? Why not just prick oneself periodically with a small gauge disposable needle and leave it in place for 20 minutes? Why go to a licensed individual who spent one year or more learning where to place needles in the "right" places? Especially when that individual knows little about disease, cannot tell the difference between a serious illness and somatization, does not know the natural history of disease and its alterations by therapies, and has little or no understanding of what scientific biomedicine is about?
That the Consensus Conference was engaged in pseudoscientific reasoning is further illustrated by the rejection of the most obvious and probable reasons for perceived effects. Those are natural history of the disease, regression to the mean, suggestion, counter-irritation, distraction, expectation, consensus, the Stockholm effect (identifying with and aiding the desires of a dominant figure), fatigue, habituation, ritual, reinforcement, and other well-known psychological mechanisms. With such an array of obvious alternative explanations and such fertile areas for productive research, strong bias would be needed to agree to the conference conclusions.
We have concern about what constitutes fairness in evaluation of aberrant medicine. In general, physicians and scientists investigating methods may or may not be advocates of the methods. Peer review in the sciences is usually penetrating and critical. With fringe methods, however, there seem to be few interested who are not advocates, making peers de facto biased observers. Also, advocates of unproven methods seem to be extraordinarily tolerant of competitors' views, even when they might be inconsistent with or diametrically opposed to their own—e.g., homeopathy and orthomolecular medicine, and Ayurvedic and Traditional Chinese Medicine (TCM).
What of Science
Acupuncture research and the conference's conclusions show not only bias but signs of pseudoscientific thinking. Differences between acupuncture groups and control groups, even in positive studies, tend to be quite small, unless compared to doing nothing. A small percent difference between a standard method of known effect and a method of improbable effectiveness carries much less weight than does a small difference ibetween two reasonably and probably effective methods. The same principle applies here. A small (say 10%) difference between two chemotherapy agents is quite meaningful if the difference is in incidence of cure or time, freedom from disease, or time to recurrence. A 10% difference between acupuncture and sham or inactive procedure means little because of the very real possibility that it is due to error or statistical variation. The pursuit of such diminishingly small differences is a characteristic of pseudoscience.
Other other signs of pseudoscientific modalities include the lack of predictability of effects and the lack of deepening understanding of the phenomenon over time. Acupuncture research demonstrates this also.
Press reports of the conference almost uniformly referred to it as though the National Institutes of Health had recommended acupuncture as effective. Reporters and editors obviously did not do two essential things. They did not read the report, which clearly stated that the report was not the opinion or position of the NIH but only those of the people convened. Second, with a few exceptions, they did not check out the report with knowledgeable scientists for commentary.
The San Francisco Chronicle went so far as to run an editorial proclaiming the report an NIH position and ventured misleadingly into irrelevant issues such as the "Establishment's" resistance to new ideas. I submitted an op-ed piece commenting on the editorial and the conference itself, which the Chronicle refused to run because it disliked the title"Mandating Acupuncture." I chose that title because of the likelihood that the Medicare system and private insurance industry would take the conference as a reason to cover acupuncture. Chances are they will, unless some organized resistance appears. That is the way the real world works.
- Richardson PH, Vincent CA. Acupuncture for the treatment of pain: A review of the evaluative research. Pain 23:15-40, 1986.
- Ter Riet G, Kleijnen J, Knipschild P, Acupuncture and chronic pain: a criteria-based meta-analysis. Journal of Clinical Epidemiology 43:1191-1199, 1990.
- Sampson WI and others. Acupuncture: The position paper of the National Council Against Health Fraud. Clinical Journal of Pain 7:162-166, 1991.
This page was posted on March 23, 2005.