A recent study in JAMA concluded that chelation therapy may have a small effect on reducing adverse events in patients with a previous myocardial infarction.
What they found is that in the chelation group, 222 out of 839 total patients suffered an adverse outcome. This compared to 261 out of 869 patients in the placebo group who suffered an adverse outcome. Let's take a look at their statistical analysis and you can decide if their conclusion is valid, or if you believe as I do that their raw numbers indicate no statistically significant difference between chelation and placebo.
1. CONFIDENCE INTERVALS: using confidence intervals, the difference in proportions between the two groups is insignificant when using 95% confidence intervals. The difference is 0.01 to -0.08. (see p147-148, Basic and Clinical Biostatistics by Dawson, 1994 for methodology)
2. USING THE Z-APPROXIMATION: using this method to evaluate the difference in two proportions, we find that z= -1.64 which is not significant at the 0.05 level (see p148-149, Basic and Clinical Biostatistics by Dawson, 1994 for methodology)
3. CHI-SQUARE TEST: using this methodology, we also find that the results are insignificant (see p125 - 128, Statistical Methods by Snedecor & Cochran 1989 for methodology).
So far, 3 out of 3 appropriate statistical tests to compare the difference between two proportions has concluded that there is NO BENEFIT from chelation therapy compared to placebo. There are several online calculators that will confirm the above, including medcalc.org
What the authors did in order to get a statistically significant result was to use a special type of hazard ratio, the Cox Proportional Hazards model. In this, the researchers don't simply look at the data at the end of the trial, but rather look at when each patient suffered the event during the trial. When you include when during the trial the patient had the adverse event, a statistically significant difference is found with a p-value of 0.035
The problem with using the Cox Proportional Hazards ratio exclusively, is that this statistical calculation estimates future events. It does not simply look at the raw data. If you look at the raw data, with no estimation or guessing about future events, then chelation has no benefit. However, if you decide to try to predict what would have happened if you had continued the trial for 5 years (instead of the 3 years they actually studied), only then is a statistically significant difference found.
Which is true? My sense of numbers is that you absolutely must analyze the raw data first, and give readers the statistical analysis of the raw numbers first. These raw numbers are actual observations, without trying to predict future events. Then, and only then, should researchers use future prediction models.
I am very skeptical about this trial and disappointed that such a prestigious journal such as JAMA would publish such research without insisting on an analysis of the raw data that did not rely on future prediction.
As the great Yogi Berra once said, "It's tough to make predictions, especially about the future."
Statisticians need to listen to Yogi more.
