National Center for Complementary and Alternative Medicine (NCCAM)

NIH…Turning Discovery Into Health®

Follow NCCAM: Subscribe to our email update Subscribe to the NCCAM RSS feed Follow NCCAM on TwitterRead our disclaimer about external links Follow NCCAM on FacebookRead our disclaimer about external links

Menu
N C C A M Research Blog

Bayes’ Rule and Being Ready To Change Our Minds

May 12, 2014
Josephine P. Briggs, M.D.
Josephine P. Briggs, M.D.

Director
National Center for Complementary and Alternative Medicine

View Dr. Briggs' biographical sketch

Many of the clinical studies funded by NIH do not show the expected benefit. I have discussed this before on NCCAM’s blog. Surprises—positive expectations, negative results—are common in clinical research.

Every once in a while the opposite happens.

Over the course of the last year, the team that conducted the Trial to Assess Chelation Therapy (TACT) has completed a series of analyses detailing their findings.1 The most recent, published in the American Heart Journal (AHJ) compared patients with diabetes who had received both chelation and high-dose vitamins with those receiving placebo chelation and placebo vitamins.2 The authors found that those receiving the active treatment clearly fared better than those receiving placebo. The accompanying editorial in the AHJ reminds readers about the value of equipoise and the need to “test our beliefs against evidence.”3

Most physicians did not expect benefit from chelation treatment for cardiovascular disease. I readily admit, initially, I also did not expect we would find evidence that these treatments reduce heart attack, strokes, or death. So, the evidence of benefit coming from analyses of the TACT trial has been a surprise to many of us. The subgroup analyses are suggesting sizable benefit for diabetic patients—and also, importantly, no benefit for the non-diabetic patient. Clearly subgroup analyses, even if prespecified, do not give us the final answer. But it is also clear that more research is needed to test these important findings.

And TACT findings are indeed a reminder of the importance of retaining equipoise, seeking further research aimed at replicating the findings, and neither accepting nor rejecting findings based on personal biases. The scientific process is designed to weed out our preconceived notions and replace them with evidence.

Bayesian methods are getting a lot of attention in the clinical research literature these days. The Bayes rule involves estimating the probability of a result—the prior—then modifying it with each round of new evidence. Another editorialist, a statistician, examined the TACT results, using a Bayesian approach, and comments: “If we start from a position of skepticism, the results of the TACT trial reduces the degree of skepticism. This is exactly how Bayes analysis helps modify prior beliefs by incorporating new evidence and upgrading knowledge.”4

Or to quote Sharon Bertsch McGrayne, an insightful writer on science and the scientific method, Bayes’ rule commits us to “an unrelenting commitment to changing our minds in the face of new knowledge.”5 So, TACT is a reminder—an open mind is at the center of the scientific method.

 

1 Lamas GA, Boineau R, Goertz C, et al. Oral high-dose multivitamins and minerals after myocardial infarction: a randomized trial. Annals of Internal Medicine. 2013;159(12):797–805.

Lamas GA, Goertz C, Boineau R, et al. Effect of disodium EDTA chelation regimen on cardiovascular events in patients with previous myocardial infarction: the TACT randomized trial. JAMA. 2013;309(12):1241–1250.

Escolar E, Lamas GA, Mark DB, et al. The effect of an EDTA-based chelation regimen on patients with diabetes mellitus and prior myocardial infarction in the Trial to Assess Chelation Therapy (TACT). Circulation. Cardiovascular Quality and Outcomes. 2014;7(1):15–24.

Lamas GA, Boineau R, Goertz C, et al. EDTA chelation therapy alone and in combination with oral high-dose multivitamins and minerals for coronary disease: the factorial group results of the Trial to Assess Chelation Therapy. American Heart Journal. 2014; In press.

2 Lamas GA, Boineau R, Goertz C, et al. EDTA chelation therapy alone and in combination with oral high-dose multivitamins and minerals for coronary disease: the factorial group results of the Trial to Assess Chelation Therapy. American Heart Journal. 2014; In press.

3 Maron DJ, Hlatky MA. Trial to Assess Chelation Therapy (TACT) and equipoise: when evidence conflicts with beliefs. American Heart Journal. 2014. In press.

4 Kaul S. Are concerns about reliability in the trial to assess chelation therapy fair grounds for a hasty dismissal? An alternative perspective. Circulation. Cardiovascular Quality and Outcomes. 2014;7(1):5–7.

5 Sharon Bertsch McGrayne. The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy. New Haven, CT: Yale University Press; 2011.

 

Related Topics

Comments

Comments are now closed for this post.

A lot of these CAM therapies do have a physiological basis that is most often (purposely?)overlooked.EDTA is an anticollagenase and also known to inhibit metallopeptidases by chelation - both collagen and metallopeptidases (part of the matrix metalloproteinases) are involved in plaque formation.  So there you have your mode of action.Unfortunately, the sceptics tend to be blinded by what they want to believe or do not want to believe anyway and see it as their duty to instigate disbelief in the worst way - that does not help anyone.Working up your anatomy and physiology thoroughly will give you an idea why and how these treatments could work. Another treatment with a solid physiological basis is colon hydrotherapy - another favourite of skeptics like Edzard Ernst. I would love to see some research about that one as well. The problem is that less patients will need surgery or become pharmaceutical drug addicts…that is why a lot of research - if any - is usually one-sided and essentially biased. Also, other models of research than RCTs need to be employed as CAM therapies are most often holistic and suited to the individual. Plus they work with subtle energies not yet captured by mainstream science. To reduce this complexicity does not serve CAM very well. We must not forget that RCTs can only inform knowledge, but will still never give the full truth about any CAM. 

“So, the evidence of benefit coming from analyses of the TACT trial has been a surprise to many of us. “. I was pleased to read this statement but was saddened to see that this In Press article still has not been indexed in PubMed (May 13, 2014).  It was published online on April 4 2014, by the American Heart Journal http://www.ahjonline.com/article/S0002-8703(14)00150-1/fulltext, with an accepted date of Feb 27, 2014. Even though this journal is indexed for Medline http://www.ncbi.nlm.nih.gov/nlmcatalog/370465, the issue of how and when “Article In Press” or “Article Based Publishing” articles http://www.elsevier.com/journal-authors/article-based-publishing,  get indexed is an issue.  If this In Press article had been indexed by PubMed how many more people could have been counted in the statement above as the “many of us.” 

Although I am a fervent advocate of the Bayesian method, I doubt that Bayesian reasoning will be of much help in accepting CAM. There is much subjectivity in determining the prior chance and in selection of proof in favour and against any method. The selection of acceptable proof depends on paradigms.As a homeopathic physician and researcher I see the Bayesian method rather as an instrument to improve than to prove a CAM method.Lex Rutten MDBreda, NetherlandsReferences:Rutten ALB. How can we change beliefs? A Bayesian perspective. Homeopathy 2008;97:214-219Rutten ALB, Stolper CF. Diagnostic test evaluation by patient-outcome study in homeopathy: balancing of feasibility and validity. J Eval Clin Practice 2009;15:1230-1235

These TACT results were hardly a success for CAM. The entire group of people without diabetes (the majority tested) were not helped by chelation or multivitamins/minerals. I am underwhelmed by these results, which are typical for CAM.The group of people with diabetes had a marginal result, meaning at best that further testing is needed. However, one important question must be asked: Does the result show an improvement in outcome for people treated by chelation, or a net worse outcome for diabetics treated with placebo, *given that placebos in these trials are usually a sugar pill*? In other words, did the study testing diabetics harm the diabetics with the placebo? Did the treatment actually produce a zero outcome, but only look better as compared to damage caused by a placebo (supposedly neutral but really harmful)? This question really needs an honest, public answer before more money is spent testing chelation on diabetics. 

It’s ABOUT TIME!!  Nutritional/ natural/ holistic/ integrative docs have known this, by their experience in practice, for DECADES!!

There are always those who will resist until the bitter end, when they will say “I always said that this [natural treatment] would be found effective.” 

I find it rather interesting that in a post about Bayesian analysis the irony that the results were so surprising to everyone isn’t recognized. I’m not sure which is more interesting, that or the fact that the results were still shockingly underwhelming. As JerryA a single subgroup analysis in the group most likely to experience an effect and <i>still</i> have it be a marginal effect at that is hardly any sort of vindication of chelation. I also find it very entertaining that a homeopath would think that Bayesian analysis somehow <i>unfairly</i> renders homeopathy completely bankrupt. It is one thing to intelligently argue that cultural or political bias is unfairly altering a Bayes factor, but another one entirely to imply that the entirety of physics, chemistry, and pharmacology all must be spectacularly wrong in order to provide any Bayes factor above zero for any homeopathic nostrum. The fact that homeopathy is entertained as anything more than astrology or alchemy is a demonstrable failure of understanding <i>any</i> statistical analysis, let alone Bayesian. This is, unfortunately, par for the course when it comes to CAM. The thinnest of gruel provides sustenance and justification for what was arguably an unethical trial at the outset and at the very least a colossal waste of taxpayer money. If this exact same trial was done for some novel pharmaceutical and all that was discovered was this meager <i>possible</i> effect size in only one specific subgroup it would be rightly considered an outright failure. This is nothing more than desperately grasping at homeopathically diluted straws. 

Please see my response on Science-Based Medicine: http://www.sciencebasedmedicine.org/of-the-trial-to-assess-chelation-therapy-bayes-the-nih-and-human-studies-ethics/