The concrete example – the treatment of chronic pain with inflammation inhibitors
I suggested in the first chapter that you should pick out a specific established treatment routine of any illness and research on what data this directive is based. Then you should find few original studies to examine whether the patients included in these are comparable with those you see yourself in general practice. The idea behind it is (as you surely remember): Randomized trials achieve internally valid results, but often suffer from a lack of generalizability. I want to use this example as a demonstration of my claims. However, it is quite common that the reality turns out to be much more colorful and skewed than you could ever imagine.
Follow me, therefore, on my own journey:
So I’ve done my homework (more precisely: I have asked my assistant, Majella, for help. I also want to thank her here for her great work and to acknowledge her contribution to my work.) First, I acquired the newest treatment guidelines of the American Association of Anesthesiology published by their Task Force on Chronic Pain Management, a working group appointed especially for this task in 2010. [1]. Chronic pain, so assumes the common person, is initially and mainly treated with medication. At least that is what is written in the introduction of all the original studies and reviews dealing with chronic pain treatments (for a few examples see [2-5]). Back pain, especially chronic back pain, constitutes the bulk of chronic pain. It makes sense, therefore, that this also is initially treated mainly with painkillers.
This approach seems reasonable at first, and also very scientific. Pain medicine, after all, was developed and approved specifically for this reason. The same applies to the “Chronic Pain Management Guidelines”, which discuss a variety of methods and naturally also include drug treatments. Simple painkillers such as aspirin don’t work for chronic pain. Chronic pain is pain that has persisted for 6 months or over, or pain that keeps recurring. One could treat pain with non steroidal anti-inflammatory drugs (NSAIDs). These are substances that inhibit the so-called cyclo-oxygenase (Cox) enzymes. The body needs these enzymes to synthesize prostaglandins, which play a crucial role in the inflammatory cascade that leads to pain. There are at least two different cyclo-oxygenases (Cox1 and Cox2) which have different functions. The oldinflammatory drugs – aspirin, ibuprofen, diclofenac – act on both. Attempts to develop compounds that would only inhibit Cox2 and leave Cox1 untouched were successful, and some of these substances have been approved and even praised as highly effective. However, some of these new drugs have extremely serious side effects, such as death from heart failure, leading to their withdrawal. One infamous example is the scandal surrounding Vioxx – a Cox-2 inhibitor- produced by Merck. In England 5% of all prescriptions are NSAIDs; in the US their side effects are responsible for an estimated 16,500 deaths and over 100,000 hospitalizations [6].
So, NSAIDs, which are essentially Cox inhibitors, are recommended by the guidelines for the treatment of chronic pain. This is supported by the scientific literature.
The disadvantage of the guidelines of the American Association for Anesthesiology is that the bibliography used to support it was too thick to be published with it. Not everyone has a personal assistant at hand, as I do, who could search for this literature through the Internet labyrinth. We found the bibliography and inspected it. The guidelines are based on five studies [3,4,6-8]. Remember, we are looking at the treatment guidelines for chronic pain, especially back pain. Here is a closer look at the data:
Berry and colleagues (1982) divided 37 patients with chronic low back pain into three groups. One group received a placebo and two groups received either naproxen, a typical NSAID, or Difluisal, a substance that is no longer approved. Please take a guess how long the treatments were administered for this study. Remember: we’re talking about chronic back pain. No, it was not six weeks, not four either, but a whole two weeks. The result: naproxen works better than the other two substances and Difluisal was not found to be better than the placebo. The pain is slightly decreased. Long-term effect – Unknown. Long term side effects – Not tested. If you divide 37 by 3 it is obvious that not too many patients were examined in this study. What do you think: is this data a good basis for generalizations? Is this data a good basis to evaluate the long-term use of this drug in many patients, not just for 2 weeks but perhaps for two or more years?
Driessen and colleagues (1994) examined the efficacy of ibuprofen, a relatively wellknown pain and inflammatory agent, in 30 chronic back pain patients over 2 weeks, and compared it with diclofenac, another typical Cox-inhibitor. This study did not have placebo and had no treatment controls. Six patients from the Diclofenac and four patients from the ibuprofen groups withdrew from the study because of adverse effects. Otherwise the trends of the two groups were similar with a side effect rate of 40% in the diclofenac and about 25% in the ibuprofen group. Again, we know nothing about longterm effects.
The other three (of the five) studies examined Cox2 inhibitors. These studies were all relatively large (700, 400, 300 patients) and also examined the Cox2 inhibitor treatment (sometimes in two different concentrations) for a relatively long period (4 weeks to 3 months) against a placebo. The drugs were all effective. The disadvantage is that two of the three studies investigated a Cox2 inhibitor which had already been taken off the market before these guidelines were published because its side effects were too great.
So there are only three studies [as a basis for this guideline, not necessarily in the literature!] supporting the use of NSAIDs, the most commonly used medications for chronic pain, of which two studies are very tiny and short and give no information about the long-term effects and little information about the side effects. When a study does show its side effect percentage, it is very high. The third study lasted three months. Conclusively, we still do not know what happens when chronic pain patients take such medicines for longer periods. Such data are not available or are not cited by the guidelines.
The examination of the inclusion criteria shows that patients with other illnesses – such as depression, anxiety, or other physical conditions – were not included in these studies. Therefore, the generalizability of the results is limited.
But wait. Can it really be that these five studies is all we have? You may think it unlikely. And you’re right. If one searched a little, and I did a little searching because it was sufficient to prove my point- then you’ll find:
In 2000 the Cochrane Collaboration, a group of scientists whose goal is to systematically obtain, make easily available, and evaluate the knowledge gained from studies, published a highly competent overview [2]. They conclude that despite being among the most used anti-pain substances in 2000 (and even currently) there is not enough evidence for the effectiveness of the NSAIDs (!). This overview summarized 53 studies, including the two mentioned earlier which are quoted in the American Association for Anesthesiology guidelines. The Cochrane Collaboration concludes in its overview that NSAIDs are not suitable for treating chronic back pain because they are ineffective. This finding mirrors an earlier one [5], which also concluded that such substances are only effective for a short time, about two weeks, and are not effective for the treatment of chronic pain.
Does the American Association for Anesthesiology guideline consider these findings? No. Why not? Good question. What do you think?
These findings are more alarming than I would have expected in my wildest dreams. These studies examine the most commonly used substances for a syndrome which occurs very frequently, and there are dozens of such studies. The overview of all these studies shows that these substances do not work well enough. But the latest guidelines still recommend them. And everybody is taking these substances and risking many side effects. What does that mean?
Firstly this quite obviously means that medical activity is much less scientific than one commonly assumes. A large-scale field study in England came to the same conclusion, that general practitioners use scientific information only as one of many information sources [9]. These general practitioners value informal information more: examples and tips from colleagues and their own experience.
Secondly this means that the much touted scientific evidence is not taken as seriously as you think. We are all under the same assumption that scientific medicine is based on pure scientific data. As we see from this example, this is not always true. Why is that? I suspect this is due to different interests and preconceptions. We all think that the drugs work; this is after all why they are created. And they do work within certain limits. These limits are often stretched, as we see in the example of NSAID therapy for chronic low back pain, but we deliberately choose not see that.
In this particular case, even a serious consideration of the available data would help and we would never dream of recommending NSAIDs for the treatment of chronic pain. The British regulator, NICE (National Institute for Clinical Excellence), did review the data, and in its latest guidelines recommends movement, manipulation and mobilization, and acupuncture as the only effective measures.
What do we learn from this? That medical quality assurance and guidelines do not remove the individual need to be well informed and to decide what is best. A recent study has shown that there are many contradicting guidelines – and of these there are so many that no one pays attention to them any longer. This leads to anarchical behavior, the exact opposite to their original intention [10]. But that is only a marginal effect.
What does this mean methodologically? That the much-touted EBM pyramid doesn’t work in practice. On the one hand there are not as many studies as you might think. On the other hand, the results of the studies that do exist are apparently ignored (if they are seen as not suitable). Why is this so? Maybe because there are other implicitly important sources of information used by physicians and patients that are not reflected in formalized studies. My best guess is that doctors (and patients) have a different model of coming to their medical conclusions in their minds and the evidence hierarchy of EBM is exactly what it is: a guideline. And guidelines are inherently often ignored or violated, especially if they are unreasonable.
In principle, humans have an implicit layered and circular model of gaining knowledge, I guess. We use multiple sources of information and it is unnatural to rely on just one. This alone is reason enough why the EBM method of hierarchical evidence does not work. But there are also methodological reasons to prefer a different approach, as will be described in the next chapters.
Literature:
- Task Force on Chronic Pain Management (2010) Practice guidelines for chronic pain management. Anesthesiology 112:810-833.
- van Tulder, M.W., Scholten, R.J., Koes, B.W., Deyo, R.A. (2000) Nonsteroidal anti-inflammatory drugs for low back pain. Spine 25:2501-2513.
- Katz, N., Ju, W.D., Krupa, D.A., Sperling, R.S., Rodger, D.B., Gertz, B.J. … Borenstein D. VCLBPS Group. (2003) Efficacy and safety of rofecoxib in patients with chronic low back pain. Results from two 4-week, randomized, placebo-controlled, parallel-group, double-blind trials. Spine 28:851-859.
- Birbara, S.A., Puopolo, A.D., Munoz, D.R., Sheldon, E.A., Mangione, A., Bohidar, N.R., Geba, G.P., EPS Group. (2003) Treatment of chronic low back pain wtih etoricoxib, a new cyclooxygenase-2 selective inhibitor: improvement in pain and disability – a randomized, placebocontrolled, 3 month trial. Journal of Pain 4:307-315.
- Koes, B., Scholten, R., Mens J., Bouter, L. (1997) Efficacy of non-steroidal anti-inflammatory drugs for low-back pain: a systematic review of randomized clinical trials. Annals of the Rheumatic Diseases 56:214-223.
- Coats, T.L., Borenstein, D.G., Nangia, N.K., Brown, M.T. (2004) Effects of valdecoxib in the treatment of chronic low back pain: results of a randomized, placebo-controlled trial. Clinical Therapeutics 26:1249-1260.
- Driessens, M., Famaey, J.P., Orloff, S., Chochrad, I., Cleppe, D., de Brabant, G., … Soenen, M. (1994) Efficacy and tolerability of sustained-release ibuprofen in the treatment of patients with chronic back pain. Current Therapeutic Research 55:1283-1292.
- Berry H, Bloom B, Hamilton EBD, Swinson DR: Naproxen sodium, diflunisal, and placebo in the treatment of chronic back pain. Annals of the Rheumatic Diseases 1982; 41:129-132.
- Gabbay, J., le May, A. (2004) Evidence based guidelines or collectively constructed “mind lines”? Ethnographic study of knowledge management in primary care. British Medical Journal 329:1013-1017.
- Carthey, J., Walker, S., Deelchand, V., Vincent, C., Griffiths, W.H. (2011)Breaking the rules: understanding non-compliance with policies and guidelines. British Medical Journal 343: d5283