Evaluating Risk versus Benefit of Drugs for MS
 

http://www.slate.com/id/2150354

This link does an excellent job of pointing out the pitfalls of evaluating the benefit of a drug.

So, if a drug company, (lets take Biogen as an example), says that there are 47% less attacks on Tysabri, this article points out how that tells you very little about the benefit of Tysabri. It can mask the real data which might show that it is far less effective than that.

I hope you find the article as good as did I. I had known this before, but these folks put it in terms that are easy for non-statistics folks to comprehend.

I see the point you are making, but for any drug that someone takes for a long time for MS, the relative reduction seems relevant. A 33% reduction in relapses means two relapses in three years instead of three relapses in three years...that is if we make the possibly quite incorrect assumption that the drugs have approximately the same effect on everyone who takes it.

Yes.

But, if, using your example, its Three relapses in 3 years on placebo , and two relapses in 3 years on Tysabri, but you as a patient only have been getting 1 relapse every other year, it makes the risks look possibly less worth the benefit.

Or, it ist 3 relapses on placebo in 3 years, 2 relapses on Tysabri in 3 years, and 2 relapses on Copax and Rebif in 2.8 years, now what risks does the patient want to make in light of benefit.

I should add I do not know the # of relapses per year, I'm just building on y our example.

xo++ seems to know this stuff off the top of his head and might be able to fill in the blanks better.

The point being is that percentages do not give us a good picture of benefit when we weigh it against risk.

This article is probably the most enlightening article, that I have ever read. What little respect I had for big Pharma, just dropped another several points.

Not to mention, our Docs, who are freely prescribing these drugs to us, without advising us of the real benefits versus risks.

Shame on them all.:(

From now on, before I take a Med, I will personally find out what the Real v Relative benefits and risks are, before I take it.

For instance, I am on Paxil, touted to be an SSRI without addictive components, however, after I am on it, I find out that you just can't stop taking it. You have to wean yourself off and even then, you are subject to residual withdrawal SX.:mad: FDA made them put a black box warning on Paxil in 2002, but too late for many, who had horrific withdrawal symptoms, while weaning off of the drug.

What ticks me off is that the mfgs of Paxil, knew this long before they were forced to put the black box on the package. They were touting it as nonaddictive, right upm to the time the FDA came down on them.:mad:

Thank you so much for posting this BBS, and I am very interested to hear Mark/ox's thought on this. I do respect his opinion.:cool:

The 33% reduction is for the CRABS, approximately. Tysabri is a lot more effective. The problem with actual numbers is that some people naturally have a much higher relapse rate than others. Presumably someone who has a higher natural relapse rate is more likely to have one or more fewer relapses over the next year or two than someone with a lower natural relapse rate. That's why the relative risk reduction makes more sense to me.

But that is also why subjects are randomly assigned to the different conditions (e.g. Ty versus placebo vs. abcr), because the random fluctuation in the usual number of relapses should even out over groups: that is, if the number of subjects in each group is high enough to combat these individual differences.

Yes Sally, Paxil is one of the more difficult ADs to wean off of. For some folks, however, such as folks who have depression and OCD or other anxieties, the benefit of Paxil outweighs the down side. For others, however, they could have just as easily been put on Prozac, which doesnt have that bad weaning downside.

Thank you for posting this BBS.

Matt, you and I have talked about the way these statistics are presented, and that IMHO, they are inflated.

The article that I referenced at one time during our discussion was on an LDN forum . . . so perhaps the thought was that this may be biased. This was the article:

Is This the Drug for Me?
By Art Mellor

This article is the third in a series that discusses things to consider when evaluating trial results for a new treatment. In this instalment, I'd like to cover a set of questions to help you be more informed when reviewing the results of a treatment trial.

There's no formula for deciding whether a particular study was done "right" or not. You need to look at the results, ask yourself (and your neurologist) a lot of questions, and assess the answers in the context of your own situation. Below, I'll cover a good starting set of things to be concerned about for any treatment trial results.

Start with the inclusion criteria used for the study. What sort of people did they recruit? Are they similar enough to you? If you have progressive MS and the study only includes relapsing-remitting, it may not apply to you at all.

How many people were in the trial? How many completed it? Why is that number different if not everyone finished? When looking at other reported results, notice how many people were included in reporting that result.

How long did the study last? Is that long enough to assess benefit in MS? Were the benefits seen in the study maintained after the time period of the study?

Look at the criteria they measured to determine efficacy of the treatment. Is that something that is important to you? Is it entirely meaningful? Perhaps they measure the number of relapses, but not the severity? Since the number of lesions in your brain can't currently be correlated to disability, does it matter if they increase or decrease?

Watch out for how results are reported. You need to notice whether they are relative or absolute. If 20% of the placebo arm had relapses, and only 10% of the active drug arm did, then there was a 50% reduction in relapses compared to placebo (that's relative), but only a 10% reduction overall (absolute). Guess which one gets advertised?

Generally, benefits get reported as relative measures, because that usually amplifies the number and makes it look better, without being incorrect. Side effects are usually reported as absolute numbers. If 1% of the participants on placebo had their leg fall off, and 4% of those on active drug did, you wouldn't see it listed as a 400% increase in fallen off legs.

Ask yourself if the listed benefits are worth the risks of taking the drug. These risks include the side effects, impact on life-style, and financial burden.

If a drug changes your annual relapse rate from .82 to .67, an 18% reduction, your first impression might be excitement. But fractional relapses, which often result from averaging or annualizing, can be deceiving.
Let's turn these decimal numbers into fractions with a common denominator: .82 is about 12 relapses every 15 years, and .67 is 10 relapses every 15 years. Are you willing to stick yourself with a needle daily or weekly, suffer the site reactions and side effects, all to have 2 fewer relapses over the next 15 years? While I'm being a little dramatic, this is how you decode the numbers used in marketing to have meaning to you.

I urge you to get the prescribing information for the drugs you take (see last month's article for how) and read through the clinical trials section and practice asking questions as you read the results. Chances are you'll have a very different view of the drug than you did before, and not necessarily a worse one.

--

Art Mellor was diagnosed with MS in 2000 and co-founded the Accelerated Cure Project for MS in 2001, a non-profit organization dedicated to curing MS by determining its causes. You can learn more at www.acceleratedcure.org or contact Art at art@acceleratedcure.org

http://www.ldnresearchtrust.org/
> Newletters
> Use the drop-down box to pick "Dec"

The approx. 30% reduction in relapses rates (which is a secondary measure of the trials anyway - and does not necessarily correlate with disease progression), are RELATIVE to placebo. In absolute terms, there was ONLY 10%. Since it's 'averages', a small % of people may have seen a huge reduction, and the others saw nothing. That's the bad part about stats . . .

The exact numbers are available though, if anyone wants to look at them on the FDA site. Even then, the "relative" vs. "absolute" concept must be taken into account with these numbers.

Cherie

Well, here's my contribution to the discussion. I dug out an article I had posted on BT1, about the problems with studies done by drug companies. The link no longer works, so I'm pasting the whole article. I saved it in April, 2006.

Drug firms' studies invite skeptics
Self-funded analysis touts own products

BY SHANKAR VEDANTAM
Washington Post

Pharmaceutical giant Eli Lilly and Co. recently funded five studies that compared its antipsychotic drug Zyprexa with Risperdal, a competing drug made by Janssen.

All five showed Zyprexa was superior in treating schizophrenia. But when Janssen sponsored its own studies comparing the two drugs, Risperdal came out ahead in three out of four.

In fact, when psychiatrist John Davis analyzed every publicly available trial funded by the pharmaceutical industry pitting five new antipsychotic drugs against one another, nine in 10 showed that the best drug was the one made by the company funding the study.

"On the basis of these contrasting findings in head-to-head trials, it appears that whichever company sponsors the trial produces the better antipsychotic drug," Davis and others wrote in the American Journal of Psychiatry.

Such studies make up the bulk of the evidence that American doctors rely on to prescribe $10 billion worth of antipsychotic medications each year. Davis pointed out the potential biases in design and interpretation that produced such contradictory results.

Other experts note that industry studies invariably seek to boost the image of expensive drugs that are still under patent. Moreover, they say, the trials are relatively brief and test drugs on patients with simpler problems than doctors typically encounter in daily practice.

By contrast, when the federal government recently compared a broader range of drugs in typical schizophrenia patients in a lengthy trial, two medications that stood out were cheaper drugs not under patent. The medication that worked best for patients with severe, intractable schizophrenia was clozapine, whose sales lag well behind every other drug in its class.

And an earlier leg of the study found that the largely unused drug perphenazine had about the same risks and benefits as far more expensive competitors that are widely assumed to be safer.

Reliance on industry-sponsored studies is not limited to psychiatry, but experts say the problem is exacerbated in areas of medicine where the goal of trials is not to demonstrate cures but to measure symptomatic relief, which allows more latitude in how the results are interpreted and marketed.
Now a growing chorus of experts is asking whether the research establishment needs to be reoriented toward publicly funded studies that might better guide clinical decisions and the billions of tax dollars the government itself spends on treatment.

"A perfectly independent agency has to be set up that says, 'Here are the areas where trials must be done,' " said Drummond Rennie, deputy editor of the Journal of the American Medical Association. "There will be two classes of trials — the believable ones and the non-believable ones."

The problem is not that companies fabricate results, experts say. Researchers, in fact, want drugmakers to sponsor more studies, not fewer. But ostensibly valid industry studies can be misleading in multiple ways, Davis said. Some use too low a dose of a competitor's drug, while others choose statistical techniques that show their drug in the best light. Virtually all test drugs on patients with relatively straightforward problems.

Davis warned that the circular results he found could undermine the confidence of clinicians and patients, and even cast doubt on medications that are genuinely superior. He and Rennie also questioned academic researchers' role in these studies.

Davis, who joked in an interview that he no longer gets to fly first class to Tokyo and Monte Carlo since he stopped accepting money from pharmaceutical companies, guessed that 90 percent of industry-sponsored studies that boast a prominent academic as the lead author are conducted by a company that later enlists a university researcher as the "author."

Drugmakers defend their studies, and Davis emphasized that the drugs do help patients. But doctors, he said, cannot afford to take the results at face value.

Sara Corya, medical director for neuroscience at Eli Lilly, a company Davis singled out for praise for the quality of its studies, said that conflicting results do not cancel each other out, and that they help clinicians understand the strengths of different drugs.

Corya and Davis noted that Lilly has strict rules to prevent author-shopping.
"The reality is that even in head-to-head comparisons, study results will differ for a variety of reasons, some transparent, some opaque," added Mariann Caprino, a spokeswoman for Pfizer, whose antipsychotic drug Geodon did not perform as well as Zyprexa in two trials funded by Eli Lilly. Pfizer's own studies found that Geodon was superior to Zyprexa in one trial and inferior in another.

"What this all means," Caprino said, "is there is no substitute for the judgment and experience of the clinician in selecting among a fortunately broad palette of medicines."

There was a time, 3-5 years ago, when I would have heartily defended the stats and the drug companies. However, I've experienced the meds since then. Avonex was a total waste of money IMHO and I worsened with the low dose of interferon.

Rebif has reduced my relapses (or maybe age has) to the point where I have not been debilitated from a relapse in a little over three years (versus 3-4x/year). Sideffects are mostly localized (site reactions) and managable.

As I see and hear more...I become far more lenient in my valuation of various medicinal benefits. The human Spirit and determination has as much impact on MS (from what I've witnessed) as does a specific drug or dietary or supplement regimen.

On the "old" BT forum, Mark looked at the CRAB stats and compared them with what Biogen has been touting with Tysabri. He came up with a number that said Tysabri was only about 12% more effective than the CRABs.

Now after reading this article comparing relative and absolute numbers, does anyone really think that Biogen is being truthful about the efficacy of Tysabri?

Harry

I'm not so sure it is a matter of being truthful, Harry, as the stats that are reported are accurate.

It is more a matter of what numbers are more relevant to us as consumers.

At a minimum, I want to know the average number of people who benefited, over & above the ones that saw just as much benefit by using placebo.

What they are presenting though, is a number relative to those on placebo. So, if placebo reduced relapses by 20%, and the drug reduced relapses by 30%, they advertise this as a 50% improvement [B].

30% - 20% = 10% --> 10%/20% = 50% improvement . . . wow!

People misunderstand this to mean that there IS a 50% improvement over using nothing.

The bottom line is that only 10% of the people did better then those on placebo or nothing, right?

But, many would argue "this is the way it is done". :cool:

Cherie

Absolutely BBS, and I am still on Paxil and have been for about 6 yrs, and it's saved my sanity. But, I am on the highest dose recommended and it's losing it's effectiveness for me, and I may have to replace it with another AD?

If I had known the truth about Paxil from the beginning, I may or may not have chosen to take it. I should have been given that choice and not hoodwinked into believing otherwise.

Actually I tried Prozac first, but it made my anxiety worse. It was like taking an upper.:eek: ...For Me!

Hi Harry,

That's not what I said (sorry).:) I just quoted the AFFIRM Tysabri monotherapy trial results, which found a relative risk reduction of disability progression of about 43% but an absolute risk reduction of 12% (29% placebo vs. 17% on Tysabri, or a difference of 12%).

By contrast, the phase III Avonex trial found a relative risk reduction of disability progression of about 37%, but an absolute risk reduction of 13% (35% placebo vs. 22% Avonex).

So in a two year period, the number of people who would not suffer disability progression as a result of Tysabri or Avonex, is about the same, based on these trial results.

Mark

Just trying to understand this... How is the relative risk reduction of disability progression calculated?

Why is the placebo absolute risk reduction higher than the drug? How is it calculated?

Hi Mark,

Sorry for the lack of recollection memory:) Your comparison, IMHO, vs placebo improvement even makes Tysabri less of the blockbuster that Biogen has been preaching.

I still have a "feeling" that there are going to be some unexpected problems with the user group of this drug once the numbers start to increase.

Harry

If you read the link below, it explains it pretty clearly. Much better than I could do. :)

http://www.slate.com/id/2150354

Cherie,

Unfortunately most drug companies don't report their trial data in the manner that you would like to see. It is reported from the marketing/sales point of view because, as we know, that sells drugs!

And with a disease like MS, impressive advertised numbers will attract a lot of attention.

Harry

Yes Harry, that was my point. :) On the most important measure of effectiveness -- impact on disability -- Tysabri was no more effective than Avonex.

Tysabri fared better on other less important measures of effectiveness.

Mark

This is what this from Ectrims seems to be saying too:

Assessing the net clinical benefit and absolute risk reduction of disease-modifying drugs in relapsing-remitting multiple sclerosis

V. Samuel, K. Akhras, R. Bennett, R. Glanzman, A. AL-Sabbagh (Detroit, Rockland, New York, USA)

-----------------------------------------------------------
Background: Clinicians and payers tend to compare the results of individual clinical trials utilizing relative risk reduction (RRR) for each disease modifying drug (DMD); however, inter-study variability, heterogeneity of the patient population, different baseline characteristics, and inconsistent placebo behaviors from different clinical trials makes comparison based on RRR inappropriate. Measurement of absolute risk reduction (ARR) may be a more appropriate assessment of benefit.

Objective: To compare baseline disease severity levels and ARRs across Class 1 DMD studies for multiple sclerosis.

Methods: Class 1 studies of various DMDs were reviewed and analyzed to compare baseline Expanded Disability Status Scale (EDSS) scores and relapse rates across DMD study populations. The clinical benefit of DMDs versus placebo for 2-year relapse rates and proportion of progression-free patients was compared across all studies using ARR and number needed to treat (NNT).

Results: The proportion of patients with 2, 3, or >=4 relapses at baseline for subcutaneous (sc) interferon-beta (IFNB)-1a treated patients were 41%, 33%, and 26% respectively. For natalizumab-treated patients, the proportion of patients with 1, 2, or >=3 relapses at baseline were 58%, 32%, and 9%, respectively with no reported data for intramuscular (im) IFNB-1a, IFNB-1b, and glatiramer acetate (GA).

The proportion of patients with baseline EDSS scores of >=3 was greater among IFNB-1a sc-treated patients compared with IFNB-1a im and natalizumab (41%, 33% [1], and 33%, respectively), with no reported data for IFNB-1b or GA. The ARRs for annualized relapse rates were 0.15, 0.24, 0.40, 0.41, and 0.45 for IFNB-1a im, GA, IFNB-1b, IFNB-1a sc, and natalizumab respectively (NNTs: 6.7, 4.2, 2.5, 2.4, 2.2). ARRs for progression-free patients were 0.13 [2], 0.03, 0.08, 0.12, and 0.12 for IFNB-1a im, GA, IFNB-1b, IFNB-1a sc, and natalizumab respectively (NNTs: 7.7 [2], 33.3, 12.5, 8.7, 8.7).

Conclusion: Given the difference in baseline risk and severity levels across studies, ARR is the appropriate way to compare DMDs in the absence of head-to-head clinical trials. Results based on ARR showed that IFNB-1a sc has comparable therapeutic effect on efficacy measures when compared to natalizumab.

1. Estimated from distribution of Baseline EDSS figure in the Clinical Review section (page 28) of Summary Basis of Approval for IFNB-1a im.
2. Apply to subset of patients (57%) who completed 2-year data.

http://registration.akm.ch/einsicht....NMASKEN_ID=900

Thanks for posting this link. Like you I have had some bad experiences with Avonex. My doctors keep telling me I need to get back on something but I have this deep gut feeling that keeps telling me NO. My experiences with my deep gut feelings during my life usually turns out to be right so I keep telling the doctors NO!

I can look back at the last 10 years and see 5 of decline and 2 of Avonex and monthly steroids and feeling pretty crappy and not getting much better, then 3 and a half years on Rebif and getting better and better and better. I'd never have believed it possible 5 years ago that i could have recovered as much lost ground as I have especially with that horrendous foot injury thrown in the mix.

My recommendation: If you've only tried one thing and it hasn't worked, give one of the others a go. There's a good chance you'll hit on something that works for you.

I'm sure when the insurance companies' docs read this kind of report, they aren't exactly going to be eager in recommending that Tysabri be covered without a lot of red tape.

Harry