[reverett123]

The bad news is that we can't trust the research.
The good news is that the system is showing signs of cleansing itself.

http://www.sciencedaily.com/releases...1124203712.htm

"...Because of their findings of publication bias and selective reporting, the authors conclude that "the information that is readily available in the scientific literature to health care professionals is incomplete and potentially biased."

In a commentary on the research, An-Wen Chan from the Mayo Clinic in Rochester (uninvolved in the study) says this new research makes an important contribution to the growing body of evidence that the trial literature is skewed towards reporting favorable results. "Biased reporting of results from NDA trials is particularly concerning because these journal articles are the only peer reviewed source of information on recently approved drugs for health care providers, who will have had limited clinical experience with these new treatments," Dr Chan says. "There are also substantial cost implications if the efficacy is overestimated and the drugs overused."

[boann]

what do you think of the following - and just for argument's sake, please assume that i am both comprehending and representing these papers accurately -

1. A retrospective study claiming to have found a causal relationship between Drug A and Event B supported its claim by presenting 11 cases in which the commencement or an increase in dosage of Drug A preceded Event B. There was no mention of what percentage of the total study population taking Drug A those 11 represented, nor was there mention of the prevalence of Event B in the general population.
2. A study mining the FDA’s adverse event reporting system database claimed to have found an unusually high reporting ratio for Drug C and Event D. The authors do not mention that approx 97% (i.e., all but one) of these reports were made after a study claiming to have found an association between the two was published and highly publicized.
3. A study claiming to have found an association between Drug X and Event Y rested its claim on the comparison of a prevalence of Event Y among those exposed to Drug X to a prevalence in the general population. No source was provided for the prevalence in the general population, nor was there a statistically significant difference between the two prevalences
4. A study claiming to have found an association between Drug Q and Event R rested its claim on a comparison between the prevalence of Event R among a population taking Drug Q and the prevalence of Event R in the general population. However, the authors, without changing their terminology to reflect this, expanded Event R to include what might be termed sub events that were explicitly excluded from the Event R as measured and referred to elsewhere in the literature. The authors also include in their prevalence of Event R with Drug Q subjects who had experienced Event R prior to exposure to Drug Q.

each of these studies was published in a peer-reviewed journal and none received any public criticism.

[reverett123]

"Presentation is everything!"
But there is some hope. I posted this under another thread, but it belongs here.
http://www.nytimes.com/2008/11/22/he...3&ref=business

Originally Posted by boann

what do you think of the following - and just for argument's sake, please assume that i am both comprehending and representing these papers accurately -

1. A retrospective study claiming to have found a causal relationship between Drug A and Event B supported its claim by presenting 11 cases in which the commencement or an increase in dosage of Drug A preceded Event B. There was no mention of what percentage of the total study population taking Drug A those 11 represented, nor was there mention of the prevalence of Event B in the general population.
2. A study mining the FDA’s adverse event reporting system database claimed to have found an unusually high reporting ratio for Drug C and Event D. The authors do not mention that approx 97% (i.e., all but one) of these reports were made after a study claiming to have found an association between the two was published and highly publicized.
3. A study claiming to have found an association between Drug X and Event Y rested its claim on the comparison of a prevalence of Event Y among those exposed to Drug X to a prevalence in the general population. No source was provided for the prevalence in the general population, nor was there a statistically significant difference between the two prevalences
4. A study claiming to have found an association between Drug Q and Event R rested its claim on a comparison between the prevalence of Event R among a population taking Drug Q and the prevalence of Event R in the general population. However, the authors, without changing their terminology to reflect this, expanded Event R to include what might be termed sub events that were explicitly excluded from the Event R as measured and referred to elsewhere in the literature. The authors also include in their prevalence of Event R with Drug Q subjects who had experienced Event R prior to exposure to Drug Q.

each of these studies was published in a peer-reviewed journal and none received any public criticism.

[paula_w]

I'm trying to decide if the theme of my final advocacy years, based on movie title, should be "The Bucket List" or "There Will Be Blood."

Originally Posted by boann

what do you think of the following - and just for argument's sake, please assume that i am both comprehending and representing these papers accurately -

1. A retrospective study claiming to have found a causal relationship between Drug A and Event B supported its claim by presenting 11 cases in which the commencement or an increase in dosage of Drug A preceded Event B. There was no mention of what percentage of the total study population taking Drug A those 11 represented, nor was there mention of the prevalence of Event B in the general population.
2. A study mining the FDA’s adverse event reporting system database claimed to have found an unusually high reporting ratio for Drug C and Event D. The authors do not mention that approx 97% (i.e., all but one) of these reports were made after a study claiming to have found an association between the two was published and highly publicized.
3. A study claiming to have found an association between Drug X and Event Y rested its claim on the comparison of a prevalence of Event Y among those exposed to Drug X to a prevalence in the general population. No source was provided for the prevalence in the general population, nor was there a statistically significant difference between the two prevalencesA study claiming to have found an association between Drug Q and Event R rested its claim on a comparison between the prevalence of Event R among a population taking Drug Q and the prevalence of Event R in the general population. However, the authors, without changing their terminology to reflect this, expanded Event R to include what might be termed sub events that were explicitly excluded from the Event R as measured and referred to elsewhere in the literature. The authors also include in their prevalence of Event R with Drug Q subjects who had experienced Event R prior to exposure to Drug Q.

each of these studies was published in a peer-reviewed journal and none received any public criticism.

[lurkingforacure]

If this kind of "oversight" and "restrospective" omissions were in the corporate world, these people would be in jail. Think of Enron, etc. Or the polical arena, where every nickel or nonmonetary benefit must be disclosed or there is hell to pay (not that it should be otherwise, but really, if some politician fails to disclose that he got a new bathroom, somehow that does not bother me nearly as much as a drug company/research team skewing data).

Why is this is not illegal where people's LIVES, as opposed to money, are at stake? What a world we live in. But like Rick said, the times do seem to be a'changin', and not a moment too soon. I wouldn't want to do anything to stifle research, but perhaps we should start to consider imposing liability to those involved in manipulating data in the drug research arena.

[boann]

This is a long one, but it is, in my opinion, fascinating, in a bad way.

The first study on my list came out of the Mayo Clinic Rochester (MCR). Since it is patently ridiculous to claim to have found a causal relationship based on the identification of 11 instances, and the thought that a researcher would be unaware of the need for more data in order to reach/support that conclusion is equally ridiculous, in May of '07 I registered a complaint of research misconduct with the MCR.

Falsification of data is defined in part by the FDA as follows:

"Falsification of data includes .... omitting data in such a way that the data do not represent what actually occurred...omitting data that were obtained and ordinarily would be recorded" (www.fda.gov/oc/gcp/slideshows/2003/gcp2003.ppt)

The first substantive correspondence I received back responded as if I had stated the opinion that there was no association - that was not what I had stated - I had said in no uncertain terms that the authors failed to provide the data necessary to evaluate the presence and strength of an association/causal relationship, which makes he MCR’s response ridiculous, because in the absence of that data it is impossible for *anyone* to legitimately have *any* opinion regarding the presence or absence of an association.

I pointed this out to them in my response.

In the second substantive communication I received from them, which I was told came from someone on the Mayo Clinic Committee on the Conduct of Ethical Research, seemed to be written in response to a communication I certainly never wrote. Specifically, reference was made twice to the complainant (that would be me) having stated that randomized, double blind etc trials were the best way to resolve questions like this. Never once in all my communication with the MCR had I so much as breathed the words “clinical trial.”

The responded had also somehow gotten the idea that it was the MCR’s press release publicizing the study that disturbed me – not the study itself. It would be impossible to misread anything I had written to that degree.

The respondent ultimately claimed that the study in question didn’t have to present supporting data because it was a preliminary study describing a small number of ‘anecdotal” cases, and that because the study did not present itself as something it wasn’t, i.e., a double blind etc trial, the use of the word “association” was appropriate in this context, which is ludicrous in and of itself, and gets even more so when one considers that the title of the article states quite clearly that Drug A causes Event B.

Other mind bogglers included the following:

“The scientific process, on the other hand, involves communication of ideas between physicians and scientists in published articles so that any interested physicians and scientists can study the issue. This requires publication of even preliminary results, if they have the potential to be important. It is then the responsibility of journalists to be sure the press release does not overstate the conclusions.”

YOWSA! So, physicians and scientists communicate ideas via press releases, and articles based on press releases? And journalists are responsible for the content of press releases issued by, say, the MCR?

God help us.

S/he also brought up the fact that it was published in a peer-reviewed journal, as if that rendered any study immune to this sort of examination.

So, I listed all of my points over again, and threw in the fact that I had never mentioned clinical trials. It was at that point that I started requesting that I be bcc’d on the forwarding of any of my emails to anyone for review and also that they forward to me copies of my already forwarded emails, because when you get right down to it, I have no evidence that the most recent respondent ever even saw a word I wrote – the response was just copied and pasted into an email from a guy in the compliance dept.

Over the next month as I periodically wrote to check and see if there was any response to my most recent communication, I reiterated my request to receive copies of forwarded mails past and future.

I also requested a copy on the MCR’s policy on the handling of claims of research misconduct. Institutions that receive federal funding, as MCR does, are required by federal regulation to provide anyone who asks with a copy of this policy.

Nearly five weeks after my response to the email described above was sent, the gentleman I had been dealing with at MCR ignored my request for a copy of the policy and handed me off to a colleague. I wrote to the colleague to introduce myself, gave a quick outline of my complaint and the ways in which it had been misinterpreted to date, and again requested a copy of their policy on research misconduct.

The colleague also ignored my request and in her response to me dismissed my complaint out of hand.

I resorted to deception – called the legal dept and pretended to be a student doing a paper on research misconduct policies. My request was received in a friendly fashion and I was to receive the policy (which was in the process of being revised) the next day via fax. For reasons I can’t recall, I revealed my identity shortly before the policy was to be faxed to me. The woman in the legal dept went ballistic, ranting at me about how the case was closed and it would not be re-opened. I reminded her I was not asking that it be re-opened, I was only asking for a copy of their policy.

When I received the policy, it was clear that they had not adhered to it, and that, together with their refusal in the form of ignoring my request, to provide the policy, led me to the Office of Research Integrity (ORI) – the refusal to fulfill my request was a violation of federal regulation, after all, but the gentleman there responded as if I were making a complaint of research misconduct and said he had no jurisdiction there - I would have to go to the institution itself.

I pointed out that this was not a research misconduct issue, it was a question of the violation of CFR 42 Part 93.302(a)(1)):

"Covered institutions must, upon request, provide their policies and procedures on research misconduct to ORI, authorized HHS personnel, and members of the public."

I asked him if compliance with this federal regulation was optional.

I thought he responded, but i am having trouble finding the email if he did. Upshot was no action, response or no response.

And so my quest, which started in May of '07, ended in December of '07.

It is interesting to note that the lead author on the study in question left the MCR shortly after I first filed my complaint in May of '07. I have no further details on the departure, only that it occurred and roughly when.