A brief methodological commentary on the retraction of our homeopathy ADHD meta-analysis
We had rejoiced too soon. Last summer, I reported that we were able to publish a meta-analysis on homeopathy in ADHD, which showed a significant effect size of g = 0.6 [1]. It was recently retracted by the journal, not by us.
The background: We had made an extraction error, namely positively coding an effect size that should actually be negatively coded. This is one of the pitfalls in a meta-analysis that I have now stumbled across myself. Because you always have to ask yourself: Do the effects of a study point in the direction of the hypothesis, i.e. do they support the assumption that the difference speaks for the effectiveness of a treatment, or against it? In this case [2], the result was not only not significant in favor of homeopathy, but even pointed in the other direction. This should have been marked with a minus sign in the analysis, which I had simply overlooked. And my colleagues didn’t notice it either, so this very stupid mistake crept in.