Today in Coronavirus study news: This is fine
June 3, 2020 12:37 PM Subscribe
Studies That Most Likely Led WHO to Halt Hydroxychloroquine COVID-19 Trials are Under Fire Amid Questionable Data from Surgisphere (The Science Times, June 3, 2020) Other COVID-19 studies also drew from Surgisphere datasets.
Earlier: Antimalarial drug touted by President Trump is linked to increased risk of death in coronavirus patients, study says (Washington Post, May 22, 2020) An analysis of 96,000 patients shows those treated with hydroxychloroquine were also more likely to suffer irregular heart rhythms
The study, published Friday in the medical journal Lancet is the largest analysis to date of the risks and benefits of treating covid-19 patients with antimalarial drugs. Like earlier smaller studies, it delivered disappointing news to a world eager for promising treatments for the novel coronavirus as the global death toll grows to more than 300,000.
The hydroxychloroquine study: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis (The Lancet, May 22, 2020); May 25th Lancet discussion at the Statistical Modeling, Causal Inference, and Social Science blog
However: A mysterious company’s coronavirus papers in top medical journals may be unraveling (Science, June 2, 2020) On its face, it was a major finding: Antimalarial drugs touted by the White House as possible COVID-19 treatments looked to be not just ineffective, but downright deadly. A study published on 22 May in The Lancet used hospital records procured by a little-known data analytics company called Surgisphere to conclude that coronavirus patients taking chloroquine or hydroxychloroquine were more likely to show an irregular heart rhythm—a known side effect thought to be rare—and were more likely to die in the hospital. [...]
But just as quickly, the Lancet results have begun to unravel—and Surgisphere, which provided patient data for two other high-profile COVID-19 papers, has come under withering online scrutiny from researchers and amateur sleuths. They have pointed out many red flags in the Lancet paper, including the astonishing number of patients involved and details about their demographics and prescribed dosing that seem implausible.
Today: In response, The Lancet's issued an Expression of Concern: Important scientific questions have been raised about data reported in the paper by Mandeep Mehra et al—Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis1—published in The Lancet on May 22, 2020. Although an independent audit of the provenance and validity of the data has been commissioned by the authors not affiliated with Surgisphere and is ongoing, with results expected very shortly, we are issuing an Expression of Concern to alert readers to the fact that serious scientific questions have been brought to our attention. We will update this notice as soon as we have further information.
Also: Hours before the Lancet's statement, the New England Journal of Medicine issued its own "EOC" about a different study (involving cardiovascular disease, blood-pressure meds -- including angiotensin-converting enzyme (ACE) inhibitors -- and COVID-19 patients) which used Surgisphere data.
Furthermore: Surgisphere's own unvetted "retrospective observational study" is getting greater side eye: "In a preprint first posted in early April, Surgisphere founder and CEO Sapan Desai and co-authors conclude that ivermectin, an antiparasitic drug, dramatically reduced mortality in COVID-19 patients." Ivermectin and COVID-19: How a Flawed Database Shaped the Pandemic Response of Several Latin-American Countries (Barcelona Institute for Global Health blog, May 29, 2020) Researchers detailed their concerns in open letters on May 28 (the hydroxychloroquine study) and June 2 (the cardiovascular disease & drug therapy study).
Lastly: Governments and WHO changed Covid-19 policy based on suspect data from tiny US company (The Guardian, June 3, 2020) A Guardian investigation can reveal the US-based company Surgisphere, whose handful of employees appear to include a science fiction writer and an adult-content model, has provided data for multiple studies on Covid-19 co-authored by its chief executive, but has so far failed to adequately explain its data or methodology. Data it claims to have legitimately obtained from more than a thousand hospitals worldwide formed the basis of scientific articles that have led to changes in Covid-19 treatment policies in Latin American countries. It was also behind a decision by the WHO and research institutes around the world to halt trials of the controversial drug hydroxychloroquine.
On Wednesday, the WHO announced those trials would now resume.
Earlier: Antimalarial drug touted by President Trump is linked to increased risk of death in coronavirus patients, study says (Washington Post, May 22, 2020) An analysis of 96,000 patients shows those treated with hydroxychloroquine were also more likely to suffer irregular heart rhythms
The study, published Friday in the medical journal Lancet is the largest analysis to date of the risks and benefits of treating covid-19 patients with antimalarial drugs. Like earlier smaller studies, it delivered disappointing news to a world eager for promising treatments for the novel coronavirus as the global death toll grows to more than 300,000.
The hydroxychloroquine study: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis (The Lancet, May 22, 2020); May 25th Lancet discussion at the Statistical Modeling, Causal Inference, and Social Science blog
However: A mysterious company’s coronavirus papers in top medical journals may be unraveling (Science, June 2, 2020) On its face, it was a major finding: Antimalarial drugs touted by the White House as possible COVID-19 treatments looked to be not just ineffective, but downright deadly. A study published on 22 May in The Lancet used hospital records procured by a little-known data analytics company called Surgisphere to conclude that coronavirus patients taking chloroquine or hydroxychloroquine were more likely to show an irregular heart rhythm—a known side effect thought to be rare—and were more likely to die in the hospital. [...]
But just as quickly, the Lancet results have begun to unravel—and Surgisphere, which provided patient data for two other high-profile COVID-19 papers, has come under withering online scrutiny from researchers and amateur sleuths. They have pointed out many red flags in the Lancet paper, including the astonishing number of patients involved and details about their demographics and prescribed dosing that seem implausible.
Today: In response, The Lancet's issued an Expression of Concern: Important scientific questions have been raised about data reported in the paper by Mandeep Mehra et al—Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis1—published in The Lancet on May 22, 2020. Although an independent audit of the provenance and validity of the data has been commissioned by the authors not affiliated with Surgisphere and is ongoing, with results expected very shortly, we are issuing an Expression of Concern to alert readers to the fact that serious scientific questions have been brought to our attention. We will update this notice as soon as we have further information.
Also: Hours before the Lancet's statement, the New England Journal of Medicine issued its own "EOC" about a different study (involving cardiovascular disease, blood-pressure meds -- including angiotensin-converting enzyme (ACE) inhibitors -- and COVID-19 patients) which used Surgisphere data.
Furthermore: Surgisphere's own unvetted "retrospective observational study" is getting greater side eye: "In a preprint first posted in early April, Surgisphere founder and CEO Sapan Desai and co-authors conclude that ivermectin, an antiparasitic drug, dramatically reduced mortality in COVID-19 patients." Ivermectin and COVID-19: How a Flawed Database Shaped the Pandemic Response of Several Latin-American Countries (Barcelona Institute for Global Health blog, May 29, 2020) Researchers detailed their concerns in open letters on May 28 (the hydroxychloroquine study) and June 2 (the cardiovascular disease & drug therapy study).
Lastly: Governments and WHO changed Covid-19 policy based on suspect data from tiny US company (The Guardian, June 3, 2020) A Guardian investigation can reveal the US-based company Surgisphere, whose handful of employees appear to include a science fiction writer and an adult-content model, has provided data for multiple studies on Covid-19 co-authored by its chief executive, but has so far failed to adequately explain its data or methodology. Data it claims to have legitimately obtained from more than a thousand hospitals worldwide formed the basis of scientific articles that have led to changes in Covid-19 treatment policies in Latin American countries. It was also behind a decision by the WHO and research institutes around the world to halt trials of the controversial drug hydroxychloroquine.
On Wednesday, the WHO announced those trials would now resume.
…I'm so confused - is hydroxychloroquine good or bad? Or is this about how the answer is "actually, we don't know because the people who were saying it was bad were super-sketch"?
posted by EmpressCallipygos at 12:56 PM on June 3, 2020 [27 favorites]
posted by EmpressCallipygos at 12:56 PM on June 3, 2020 [27 favorites]
Void. Howling. Gah.
posted by bcd at 1:02 PM on June 3, 2020 [1 favorite]
posted by bcd at 1:02 PM on June 3, 2020 [1 favorite]
whoa:
The Guardian’s investigation has found:posted by jenkinsEar at 1:06 PM on June 3, 2020 [13 favorites]
A search of publicly available material suggests several of Surgisphere’s employees have little or no data or scientific background. An employee listed as a science editor appears to be a science fiction author and fantasy artist. Another employee listed as a marketing executive is an adult model and events hostess.
The company’s LinkedIn page has fewer than 100 followers and last week listed just six employees. This was changed to three employees as of Wednesday.
While Surgisphere claims to run one of the largest and fastest hospital databases in the world, it has almost no online presence. Its Twitter handle has fewer than 170 followers, with no posts between October 2017 and March 2020.
Until Monday, the “get in touch” link on Surgisphere’s homepage redirected to a WordPress template for a cryptocurrency website, raising questions about how hospitals could easily contact the company to join its database.
Desai has been named in three medical malpractice suits, unrelated to the Surgisphere database. In an interview with the Scientist, Desai previously described the allegations as “unfounded”.
In 2008, Desai launched a crowdfunding campaign on the website Indiegogo promoting a wearable “next generation human augmentation device that can help you achieve what you never thought was possible”. The device never came to fruition.
Desai’s Wikipedia page has been deleted following questions about Surgisphere and his history.
Or is this about how the answer is "actually, we don't know because the people who were saying it was bad were super-sketch"?
This seems to be the case. That last Guardian link has a lot of detail on the sketchy nature of Surgisphere and reasons why they don't seem like a trustworthy actor.
I'm not really sure if I understand Surgisphere's motivation for poisoning the well in this way, though. To me, a lay person, this seems like the kind of thing that would be quickly caught by the regular operation of the scientific community, which it thankfully has been.
posted by whir at 1:08 PM on June 3, 2020 [8 favorites]
This seems to be the case. That last Guardian link has a lot of detail on the sketchy nature of Surgisphere and reasons why they don't seem like a trustworthy actor.
I'm not really sure if I understand Surgisphere's motivation for poisoning the well in this way, though. To me, a lay person, this seems like the kind of thing that would be quickly caught by the regular operation of the scientific community, which it thankfully has been.
posted by whir at 1:08 PM on June 3, 2020 [8 favorites]
Yes: "actually, we don't know because the people who were saying it was bad were super-sketch"
But, simultaneously, a bunch of the people /pushing/ hydroxychloroquine are also super sketch... Only proper science can tell us for sure.
Here's an NYT profile of one of the people who first pushed hydroxychloroquine. He sounds like a prototypical asshole with a good track record and an actual lab... so, could go either way, AFAICT. (A bunch of other, bigger grifters in the right wing media got involved downstream.)
posted by kaibutsu at 1:09 PM on June 3, 2020 [9 favorites]
But, simultaneously, a bunch of the people /pushing/ hydroxychloroquine are also super sketch... Only proper science can tell us for sure.
Here's an NYT profile of one of the people who first pushed hydroxychloroquine. He sounds like a prototypical asshole with a good track record and an actual lab... so, could go either way, AFAICT. (A bunch of other, bigger grifters in the right wing media got involved downstream.)
posted by kaibutsu at 1:09 PM on June 3, 2020 [9 favorites]
Unfortunate that equal and opposing scam artists don't cancel each other out.
posted by Horkus at 1:13 PM on June 3, 2020 [25 favorites]
posted by Horkus at 1:13 PM on June 3, 2020 [25 favorites]
To be clear, there are other, untainted studies showing that hydroxychloroquine resulted in increased mortality and no benefit, such as the VA study [preprint pdf], summarized here.
posted by jedicus at 1:18 PM on June 3, 2020 [43 favorites]
posted by jedicus at 1:18 PM on June 3, 2020 [43 favorites]
When there is money to be made off a vaccine - its amazing what lawyers can do to torture test the validity of the study that tested the validity of a potential moneymaker.
posted by Nanukthedog at 1:27 PM on June 3, 2020
posted by Nanukthedog at 1:27 PM on June 3, 2020
I hate this news so much because it offers a few ripe cherries for ol' numbnuts to pick and point to as delicious proof he was right all along.
posted by DirtyOldTown at 1:34 PM on June 3, 2020 [15 favorites]
posted by DirtyOldTown at 1:34 PM on June 3, 2020 [15 favorites]
When there is money to be made off a vaccine - its amazing what lawyers can do to torture test the validity of the study that tested the validity of a potential moneymaker.
Which moneymaker? The same company put out a study that showed ivermectin curing people instead.
posted by BungaDunga at 1:36 PM on June 3, 2020 [1 favorite]
Which moneymaker? The same company put out a study that showed ivermectin curing people instead.
posted by BungaDunga at 1:36 PM on June 3, 2020 [1 favorite]
Here's how to tell if you are dumb like me, read the article, then have absolutely no fucking clue what it was telling you. Does hydroxychloroquine work? I dunno. Does hydroxychloroquine kill you? I dunno.
posted by Keith Talent at 1:38 PM on June 3, 2020 [6 favorites]
posted by Keith Talent at 1:38 PM on June 3, 2020 [6 favorites]
I guess this is a reminder that the infuriatingly slow pace of academic publishing has some real benefits. Like, obviously there are still retractions at the best of times. And obviously there's a good reason to be in a hurry on this one. But I bet some of these red flags would have been easier to spot if everyone was working at a normal pace.
posted by nebulawindphone at 1:44 PM on June 3, 2020 [8 favorites]
posted by nebulawindphone at 1:44 PM on June 3, 2020 [8 favorites]
Strange timing, but this is on the front page of the WashPo right now:
Hydroxychloroquine, a drug promoted by Trump, failed to prevent healthy people from getting covid-19 in trial
Hydroxychloroquine, a drug promoted by Trump, failed to prevent healthy people from getting covid-19 in trial
Hydroxychloroquine did not prevent healthy people exposed to covid-19 from getting the disease caused by the coronavirus, according to a study being published Wednesday in the New England Journal of Medicine.posted by gwint at 1:45 PM on June 3, 2020 [8 favorites]
The study is the first randomized clinical trial that tested the antimalarial drug as a preventive measure, according to researchers at the University of Minnesota Medical School who conducted the trial. It showed that hydroxychloroquine, which has been touted by President Trump, was no more effective than a placebo — in this case, a vitamin — in protecting people exposed to covid-19.
Strange timing, but this is on the front page of the WashPo right now:
More like impossibility ironic timing. I'd bet when people hear that a study was likely sketch and full of shit, they are doing to think it's the one mentioned by the WaPo that says Hydroxychloroquine doesn't help, which was (as far as we know) a non-sketch study, rather than about the actual sketchy study where no one involved is a scientist and the data is likely made up.
posted by sideshow at 1:51 PM on June 3, 2020 [2 favorites]
More like impossibility ironic timing. I'd bet when people hear that a study was likely sketch and full of shit, they are doing to think it's the one mentioned by the WaPo that says Hydroxychloroquine doesn't help, which was (as far as we know) a non-sketch study, rather than about the actual sketchy study where no one involved is a scientist and the data is likely made up.
posted by sideshow at 1:51 PM on June 3, 2020 [2 favorites]
Keith Talent, the hell of it is, no one knows for certain. As per the second "below-the-fold" paragraph, small studies indicate that the drug has no therapeutic use in COVID-19 treatment. The VA study jedicus linked had 368 patients who received no benefit; if anything, the VA "discovered a possible link between hydroxychloroquine and higher death rates." (As of May 27th, 1,300 COVID-19 patients at the VA had received the drug.) When the Lancet published a study purportedly based on 96,000 patients, the resulting bigger splash influenced global policies.
Today, the results of the University of Michigan's study, "the first randomized clinical trial testing the drug for post-exposure prevention of the coronavirus," which enrolled "a total of 821 adults in the U.S. and Canada, who had been exposed in their household or through work as first responders or in health care," indicates taking hydroxychloroquine prophylactically "has no benefit over a placebo in preventing COVID-19. [...] Forty percent of trial participants taking hydroxychloroquine also developed non-serious side effects like nausea, upset stomach and diarrhea."
[In a comment in a different thread, I linked this "coronavirus as a blood vessel disease" Salon piece, & some researchers' takes at Science... and as it turns out, this idea is supported by the disease trajectory in ten adult coronavirus patients: three patients in April 20's "Endothelial cell infection and endotheliitis in COVID-19," and seven patients in "Pulmonary Vascular Endothelialitis, Thrombosis, and Angiogenesis in Covid-19," from May 21. (Published in The Lancet, and the New England Journal of Medicine, respectively -- but with so few patients, you can tell Surgisphere wasn't involved.) See also: May 29's Doctors race to understand inflammatory condition in kids]
posted by Iris Gambol at 2:18 PM on June 3, 2020 [3 favorites]
Today, the results of the University of Michigan's study, "the first randomized clinical trial testing the drug for post-exposure prevention of the coronavirus," which enrolled "a total of 821 adults in the U.S. and Canada, who had been exposed in their household or through work as first responders or in health care," indicates taking hydroxychloroquine prophylactically "has no benefit over a placebo in preventing COVID-19. [...] Forty percent of trial participants taking hydroxychloroquine also developed non-serious side effects like nausea, upset stomach and diarrhea."
[In a comment in a different thread, I linked this "coronavirus as a blood vessel disease" Salon piece, & some researchers' takes at Science... and as it turns out, this idea is supported by the disease trajectory in ten adult coronavirus patients: three patients in April 20's "Endothelial cell infection and endotheliitis in COVID-19," and seven patients in "Pulmonary Vascular Endothelialitis, Thrombosis, and Angiogenesis in Covid-19," from May 21. (Published in The Lancet, and the New England Journal of Medicine, respectively -- but with so few patients, you can tell Surgisphere wasn't involved.) See also: May 29's Doctors race to understand inflammatory condition in kids]
posted by Iris Gambol at 2:18 PM on June 3, 2020 [3 favorites]
Wait, ivermectin, the dewormer? The stuff that's virtually free in the rest of the world but costs tens to hundreds of dollars for human use in the USA? Can't remotely see a reason why a dodgy review shop would shill that, nope nope nope 🍔
posted by scruss at 2:22 PM on June 3, 2020 [6 favorites]
posted by scruss at 2:22 PM on June 3, 2020 [6 favorites]
From the Science Mag link:
posted by mhum at 2:29 PM on June 3, 2020
The study doesn’t properly control for the likelihood that patients getting the experimental drugs were sicker than the controls, says Matthew Semler, a critical care physician at Vanderbilt University. “If you have a physician sitting with two patients who have coronavirus, and the physician chooses to give one of them hydroxychloroquine, they’re doing it for a reason,” he says. The patient may be relying on high levels of supplemental oxygen, for example, or getting worse over time. But those kinds of details aren’t available about the patients in the Lancet study, he notes.Are you fucking kidding me? I'm just a layperson but this seems like the very first thing you have to control for when talking about experimental drugs. How did this make it past peer review?
posted by mhum at 2:29 PM on June 3, 2020
Hydroxychloroquine alone will not to the job. You need to take hydroxychloroquine, and azithromycin, and zinc, and this stapler, and this paddle-ball game, and this chair, and possibly snort some ether.
posted by delfin at 2:44 PM on June 3, 2020 [18 favorites]
posted by delfin at 2:44 PM on June 3, 2020 [18 favorites]
You need to take hydroxychloroquine, and azithromycin, and zinc, and this stapler, and this paddle-ball game, and this chair, and possibly snort some ether.
I hereby volunteer for the ether portion of the study. Can provide own stapler.
posted by Abehammerb Lincoln at 2:56 PM on June 3, 2020 [11 favorites]
I hereby volunteer for the ether portion of the study. Can provide own stapler.
posted by Abehammerb Lincoln at 2:56 PM on June 3, 2020 [11 favorites]
Whole situation with peer review and scientific journalism is very bad and has been for a while. Like so many other things, the rot has been there for a while, the pandemic just put enough stress on things for them to break.
The thing is that you can generally trust "science" as a body of knowledge -- i.e. if you carefully read and synthesize a whole body of literature in an area, generally speaking, you are more likely than not going to reach good conclusions. In this context, peer review is not supposed to be an absolute guarantee of quality, it's just supposed to be one part of the broader scientific conversation.
But reading, thinking, and learning is hard; lots of people just want to see individual papers as little sources of facts that they can cite in isolation without understanding them; science journalists are particularly prone to fall into this trap. And university administrators want a way to objectively rank scientists that doesn't require understanding what they are working on.
People desperately want there to be some way of distinguishing the quality of papers that does not demand any effort from them. The result is very, very strong pressure for peer review to be something much more than it actually is, to make it a guaranteed stamp that a result is both completely true and important. When the reality is it just means three overworked volunteer scientists didn't find anything obviously wrong, and a journal editor thought publishing it would make them look important.
posted by vogon_poet at 3:07 PM on June 3, 2020 [13 favorites]
The thing is that you can generally trust "science" as a body of knowledge -- i.e. if you carefully read and synthesize a whole body of literature in an area, generally speaking, you are more likely than not going to reach good conclusions. In this context, peer review is not supposed to be an absolute guarantee of quality, it's just supposed to be one part of the broader scientific conversation.
But reading, thinking, and learning is hard; lots of people just want to see individual papers as little sources of facts that they can cite in isolation without understanding them; science journalists are particularly prone to fall into this trap. And university administrators want a way to objectively rank scientists that doesn't require understanding what they are working on.
People desperately want there to be some way of distinguishing the quality of papers that does not demand any effort from them. The result is very, very strong pressure for peer review to be something much more than it actually is, to make it a guaranteed stamp that a result is both completely true and important. When the reality is it just means three overworked volunteer scientists didn't find anything obviously wrong, and a journal editor thought publishing it would make them look important.
posted by vogon_poet at 3:07 PM on June 3, 2020 [13 favorites]
XKCD provides an alternative way of talking about preprint papers...
"According to a new PDF..."
posted by justsomebodythatyouusedtoknow at 3:08 PM on June 3, 2020 [8 favorites]
"According to a new PDF..."
posted by justsomebodythatyouusedtoknow at 3:08 PM on June 3, 2020 [8 favorites]
Desai’s Wikipedia page has been deleted following questions about Surgisphere and his history.
The Guardian doesn't go into detail, but it's worth noting that Sapan Desai's Wikipedia page was deleted in 2010 (for notability reasons), not recently. You can see this on the Talk page for Surgisphere and the deletion discussion about the page. You can also see the Wikipedia page for Sapan Desai in 2010 via the Wayback Machine.
posted by oulipian at 3:32 PM on June 3, 2020 [2 favorites]
The Guardian doesn't go into detail, but it's worth noting that Sapan Desai's Wikipedia page was deleted in 2010 (for notability reasons), not recently. You can see this on the Talk page for Surgisphere and the deletion discussion about the page. You can also see the Wikipedia page for Sapan Desai in 2010 via the Wayback Machine.
posted by oulipian at 3:32 PM on June 3, 2020 [2 favorites]
People desperately want there to be some way of distinguishing the quality of papers that does not demand any effort from them.
Journal quality and correspondingly rigorous peer review was meant to act as an imperfect proxy for this on the whole. Not this fucking imperfect though!
It's hard to overstate how shit this is. As vogon_poet puts it, the rot has been there, but USUALLY it's in the lesser branches of the tree where the consequences aren't so dire. The rot has now spread to one of the main branches, the FUCKING LANCET (Impact factor ~60!!!) who got peer reviewers who did a terrible job, and an editor who let a bullshit paper through on the most politically charged topic in medicine of this century......
The problems with the paper and its authors were not subtle. Yet because of the relentless hype by murdering bullshit merchants and authoritarians re:covid19 & HCQ, the rational community grasped at straws to quickly disprove their lies before they could continue to do damage. Smart people were taken in because it confirmed our (rational) prejudices against hucksters, but we swapped one huckster for another.
This debacle will be used to beat scientists with for the next decade, by people with bullshit to sell and power to wield, and we did it to ourselves this time.
It will actively harm science, medicine and the millions upon millions of humans who rely on these for their fucking lives!
The good news is science is self-correcting by its nature; moreso than any other major fucking human endeavor. The bad news is that correcting things ourselves doesn't protect us anymore from dilution of our influence and the rise of more superstition, bullshit and power.
The Lancet should fire some editors for one thing, and blacklist the criminally negligent peer reviewers who did such a terrible job.
posted by lalochezia at 3:48 PM on June 3, 2020 [24 favorites]
This came out today in the NEJM: A Randomized Trial of Hydroxychloroquine as Postexposure Prophylaxis for Covid-19: After high-risk or moderate-risk exposure to Covid-19, hydroxychloroquine did not prevent illness compatible with Covid-19 or confirmed infection when used as postexposure prophylaxis within 4 days after exposure.
It does not appear that any of the information was drawn from Surgisphere.
posted by Comrade_robot at 4:02 PM on June 3, 2020 [4 favorites]
It does not appear that any of the information was drawn from Surgisphere.
posted by Comrade_robot at 4:02 PM on June 3, 2020 [4 favorites]
The study is the first randomized clinical trial that tested the antimalarial drug as a preventive measure, according to researchers at the University of Minnesota Medical School who conducted the trial. It showed that hydroxychloroquine, which has been touted by President Trump, was no more effective than a placebo — in this case, a vitamin — in protecting people exposed to covid-19.
Worth pointing out that this NEJM study is a randomized, double-blind study, which hasn't been the case for other studies, like those pushed by Trump and his people.
The Lancet should fire some editors, but they did the right thing and followed up bad science with an admission of error. Correcting errors is how good science works.
The larger problem is that this will be used as misinformation by Trump opportunists to again push a drug that has absolutely no clinical application for Covid-19, which is effectively a poison for those who do not need to take it for the purposes it is indicated.
posted by They sucked his brains out! at 4:48 PM on June 3, 2020 [3 favorites]
Worth pointing out that this NEJM study is a randomized, double-blind study, which hasn't been the case for other studies, like those pushed by Trump and his people.
The Lancet should fire some editors, but they did the right thing and followed up bad science with an admission of error. Correcting errors is how good science works.
The larger problem is that this will be used as misinformation by Trump opportunists to again push a drug that has absolutely no clinical application for Covid-19, which is effectively a poison for those who do not need to take it for the purposes it is indicated.
posted by They sucked his brains out! at 4:48 PM on June 3, 2020 [3 favorites]
As a pharmacoepidemiologist with experience using health insurance claims data and electronic medical records, I can agree that this paper has its issues. The absence of a clear description of the data sources used is a big red flag. But after reading the paper, in particular the Discussion section, I think that the reaction by the press is overblown and sensationalized. The authors describe many of the limitations that have been brought up, they are cautiously interpreting their results, and suggest that randomized trials are necessary to confirm their findings.
Nobody should have stopped their randomized trials because of these findings!
There are problems with all sorts of data. Real world evidence has its own issues, one mentioned above is that it's hard to control for confounding by indication (when a sicker patient gets a different treatment than a healthier patient). But the benefit of real world evidence is that we can effectively control for many issues of confounding and bias by designing the study well and doing things like matching on propensity score. The other (huge) benefit is that we can have results, something to work with and inform us, much quicker and more cheaply than a randomized trial.
This type of research has its place and I don't see why the media is piling on. There is so much we don't know about hydroxychloroquine and I see this paper as just one imperfect step towards getting closer to understanding.
posted by stripesandplaid at 4:49 PM on June 3, 2020 [6 favorites]
Nobody should have stopped their randomized trials because of these findings!
There are problems with all sorts of data. Real world evidence has its own issues, one mentioned above is that it's hard to control for confounding by indication (when a sicker patient gets a different treatment than a healthier patient). But the benefit of real world evidence is that we can effectively control for many issues of confounding and bias by designing the study well and doing things like matching on propensity score. The other (huge) benefit is that we can have results, something to work with and inform us, much quicker and more cheaply than a randomized trial.
This type of research has its place and I don't see why the media is piling on. There is so much we don't know about hydroxychloroquine and I see this paper as just one imperfect step towards getting closer to understanding.
posted by stripesandplaid at 4:49 PM on June 3, 2020 [6 favorites]
Are you fucking kidding me? I'm just a layperson but this seems like the very first thing you have to control for when talking about experimental drugs. How did this make it past peer review?
The Lancet paper, while an observational study, does control for confounding by indication, which is the type of bias Dr. Semler refers to. The authors use propensity scores to "match" treated patients to control patients so that they look similar in terms of their characteristics: age, demographics, disease severity, and so on. Propensity score matching is a standard technique that has been used tens of thousands of times to do these kinds of studies. When you can't do a randomized trial, PSM is close to the next best thing in that the matching process emulates the similarity that you usually get between the treated and control groups through randomization.
The one flaw of PSM is that it doesn't control for variables that you can't measure, but the authors of the Lancet paper follow it all up with a sensitivity analysis that tries to quantify how big of an unobserved bias would have to exist in order to reverse the conclusion of the study, i.e., to make their findings statistically non-significant. They call it a 'tipping point analysis' and find that the magnitude of the unobserved bias would have to be pretty large to reverse their findings (supplementary material, section S8)
posted by un petit cadeau at 5:07 PM on June 3, 2020 [1 favorite]
The Lancet paper, while an observational study, does control for confounding by indication, which is the type of bias Dr. Semler refers to. The authors use propensity scores to "match" treated patients to control patients so that they look similar in terms of their characteristics: age, demographics, disease severity, and so on. Propensity score matching is a standard technique that has been used tens of thousands of times to do these kinds of studies. When you can't do a randomized trial, PSM is close to the next best thing in that the matching process emulates the similarity that you usually get between the treated and control groups through randomization.
The one flaw of PSM is that it doesn't control for variables that you can't measure, but the authors of the Lancet paper follow it all up with a sensitivity analysis that tries to quantify how big of an unobserved bias would have to exist in order to reverse the conclusion of the study, i.e., to make their findings statistically non-significant. They call it a 'tipping point analysis' and find that the magnitude of the unobserved bias would have to be pretty large to reverse their findings (supplementary material, section S8)
posted by un petit cadeau at 5:07 PM on June 3, 2020 [1 favorite]
> I'm not really sure if I understand Surgisphere's motivation for poisoning the well in this way, though.
My bet would be that it was done for the benefit of a company with a competing proprietary product.
posted by at by at 5:42 PM on June 3, 2020 [1 favorite]
My bet would be that it was done for the benefit of a company with a competing proprietary product.
posted by at by at 5:42 PM on June 3, 2020 [1 favorite]
It seems like a number of these articles and blog posts dance around a very likely issue without actually saying it: that the underlying data is completely made up from scratch by Surgisphere and its unrealistically tiny workforce. The owner describes what they do as heavy on AI approaches to data analysis - but AI could create plausible fake data easier than it would be to navigate the complexities of getting access to the real data. Reputable researchers are lured with the fake data, which is noted to be extraordinarily (unrealistically?) complete, detailed and timely.
The apparent goal of this fraud, so far as I can think, would be to go short or long on stocks in pharmaceutical companies. They'd have been planning a long game, (or maybe playing this more than once already in a less charged environment). The time was right to pounce during the pandemic when lots of preprints are getting disproportionate attention, peer-review has been compressed - and in any case they got these papers published. I'd be looking at trading patterns in affected companies and dig into what is likely a deeply shelled set of underlying companies, researchers, etc beneath Surgishphere. Pharma share values are extraordinarily sensitive to the results of clinical trials (harder to fake) and this use of existing data "almost as good" can probably trigger the same market responses.
It would require a psychopath to be running this operation but unfortunately they are not in short supply.
posted by Rumple at 5:49 PM on June 3, 2020 [4 favorites]
The apparent goal of this fraud, so far as I can think, would be to go short or long on stocks in pharmaceutical companies. They'd have been planning a long game, (or maybe playing this more than once already in a less charged environment). The time was right to pounce during the pandemic when lots of preprints are getting disproportionate attention, peer-review has been compressed - and in any case they got these papers published. I'd be looking at trading patterns in affected companies and dig into what is likely a deeply shelled set of underlying companies, researchers, etc beneath Surgishphere. Pharma share values are extraordinarily sensitive to the results of clinical trials (harder to fake) and this use of existing data "almost as good" can probably trigger the same market responses.
It would require a psychopath to be running this operation but unfortunately they are not in short supply.
posted by Rumple at 5:49 PM on June 3, 2020 [4 favorites]
Do note that India continues to advise HCQ as a prophylactic for frontline medical personnel.
From Healthcare workers & SARS-CoV-2 infection in India: A case-control investigation in the time of COVID-19
From Healthcare workers & SARS-CoV-2 infection in India: A case-control investigation in the time of COVID-19
Compared to controls, cases were slightly older (34.7 vs. 33.5 yr) and had more males (58 vs. 50%). In multivariate analyses, HCWs performing endotracheal intubation had higher odds of being SARS-CoV-2 infected [adjusted odds ratio (AOR): 4.33, 95% confidence interval (CI): 1.16-16.07]. Consumption of four or more maintenance doses of HCQ was associated with a significant decline in the odds of getting infected (AOR: 0.44; 95% CI: 0.22-0.88); a dose-response relationship existed between frequency of exposure to HCQ and such reductions (χ2 for trend=48.88; P<0.001). In addition, the use of PPE was independently associated with the reduction in odds of getting infected with SARS-CoV-2posted by Gyan at 9:59 PM on June 3, 2020
If it's Beat the Lancet time, we might give them credit for issuing the EoC within 12 days. It took 12 years between their publication of Andrew Wakefield's 1998 MMR vaccine = autism paper and its retraction. It's a problem that far fewer people read retractions than the original splashy paper; especially if they are invested in the results, so the genie can't be stuffed back in the bottle. On Wakefield, which had a sample size of 12, you could have wished for the Lancet to be more "extraordinary claims require extraordinary levels of proof". The current paper ironically had numbers too big to be credible. Call Dr Goldilocks, she knows all the reasonable datasets.
posted by BobTheScientist at 11:50 PM on June 3, 2020 [7 favorites]
posted by BobTheScientist at 11:50 PM on June 3, 2020 [7 favorites]
It looks increasingly like Surgisphere was just making shit up.
U.S.:
U.S.:
The Scientist has reached out to some of the largest health systems in the states hit hardest by the coronavirus pandemic to inquire whether they participated, but could not find any that did.Australia:
Instead, a number of hospitals confirmed that they did not contribute data, namely, New Jersey health systems RJWBarnabas Health and Cooper Health, NYC Health + Hospitals and NYU Langone in New York, and Illinois-based health systems Rush and Advocate Health Care.
Northwest Community Hospital, which employed Desai as a vascular surgeon until February 10 this year, did not contribute data, it confirms in an email to The Scientist. Nor did coauthor Mendeep Mehra’s institution, Brigham and Women’s Hospital.
In an interview last week, Desai told The Scientist that he would ask if any hospitals were willing to come forward voluntarily to confirm their participation, but has not provided any names yet.
The Guardian has since contacted five hospitals in Melbourne and two in Sydney, whose cooperation would have been essential for the Australian patient numbers in the database to be reached. All denied any role in such a database, and said they had never heard of Surgisphere. Desai did not respond to requests to comment on their statements.posted by away for regrooving at 12:22 AM on June 4, 2020 [3 favorites]
Here's an NYT profile of one of the people who first pushed hydroxychloroquine. He sounds like a prototypical asshole with a good track record and an actual lab... so, could go either way, AFAICT. (A bunch of other, bigger grifters in the right wing media got involved downstream.)"
Pr Raoult is the talk of the town here. The thing I can't wrap my mind around is that he defined a very simple therapeutic protocol, which involved testing, and then, provided that results were positive, giving both Hydroxychloroquine and azithromycin. This was supposed to be accomplished in the first phase of Covid infection, to avoid symptoms due to a high viral load. His protocol is not designed to cure people who have already developed symptoms and are bound to get to the hospital. It is a prophylactic measure aimed at people who are infected but not seriously sick. It is in that context that the protocol was supposed to work. The thing I don't understand is that from the very beginning authorities (France and abroad) insisted to put the protocol to the test with patients who were in dire need of a cure, in the ICU. Pr Raoult, and as days went by, even the layman, informed by the press, wondered at the trials, which were pointless, since the protocol wasn't designed for serious symptoms and couldn't do anything against them. Trials went on though. Such deafness could only fuel conspiracy theories.
I have no idea whether this set of medicines could work the right way. I wish someone would have tried to test it properly. We've grown suspicious of a government who declared that masks were useless when we didn't have them, and as a result, I tend to consider that the joined effort (French government + pharmaceutical industry) to discredit the protocol is due to the fact that the French government wasn't and isn't ready to test people for some unknown reason, while the industry is moved by its own motives.
posted by nicolin at 12:54 AM on June 4, 2020 [5 favorites]
Pr Raoult is the talk of the town here. The thing I can't wrap my mind around is that he defined a very simple therapeutic protocol, which involved testing, and then, provided that results were positive, giving both Hydroxychloroquine and azithromycin. This was supposed to be accomplished in the first phase of Covid infection, to avoid symptoms due to a high viral load. His protocol is not designed to cure people who have already developed symptoms and are bound to get to the hospital. It is a prophylactic measure aimed at people who are infected but not seriously sick. It is in that context that the protocol was supposed to work. The thing I don't understand is that from the very beginning authorities (France and abroad) insisted to put the protocol to the test with patients who were in dire need of a cure, in the ICU. Pr Raoult, and as days went by, even the layman, informed by the press, wondered at the trials, which were pointless, since the protocol wasn't designed for serious symptoms and couldn't do anything against them. Trials went on though. Such deafness could only fuel conspiracy theories.
I have no idea whether this set of medicines could work the right way. I wish someone would have tried to test it properly. We've grown suspicious of a government who declared that masks were useless when we didn't have them, and as a result, I tend to consider that the joined effort (French government + pharmaceutical industry) to discredit the protocol is due to the fact that the French government wasn't and isn't ready to test people for some unknown reason, while the industry is moved by its own motives.
posted by nicolin at 12:54 AM on June 4, 2020 [5 favorites]
The apparent goal of this fraud, so far as I can think, would be to go short or long on stocks in pharmaceutical companies.
Since there is no medical evidence or other reasons for the promotion, it seems obvious that the various hucksters here (i.e., Fox News hosts and people in the administration, very much including Trump) have been promoting hydroxychloroquine as a way of manipulating stocks or redirecting federal money. This made-up research article seems like just more of the same.
I hope that eventually a journalist (or better, a regulating authority) can find the evidence to show these connections.
posted by Dip Flash at 5:32 AM on June 4, 2020
Since there is no medical evidence or other reasons for the promotion, it seems obvious that the various hucksters here (i.e., Fox News hosts and people in the administration, very much including Trump) have been promoting hydroxychloroquine as a way of manipulating stocks or redirecting federal money. This made-up research article seems like just more of the same.
I hope that eventually a journalist (or better, a regulating authority) can find the evidence to show these connections.
posted by Dip Flash at 5:32 AM on June 4, 2020
> un petit cadeau: "The Lancet paper, while an observational study, does control for confounding by indication, which is the type of bias Dr. Semler refers to."
Ah, ok. That's a bit of a relief (i.e.: regarding the Lancet's peer review process). I guess it serves me right for relying on the secondary reporting and not checking the original paper myself. Of course, the deeper problem is that the underlying data was probably faked but that's separate from this issue.
posted by mhum at 8:57 AM on June 4, 2020 [1 favorite]
Ah, ok. That's a bit of a relief (i.e.: regarding the Lancet's peer review process). I guess it serves me right for relying on the secondary reporting and not checking the original paper myself. Of course, the deeper problem is that the underlying data was probably faked but that's separate from this issue.
posted by mhum at 8:57 AM on June 4, 2020 [1 favorite]
MetaFilter: a poison for those who do not need to take it for the purposes it is indicated
posted by slogger at 9:45 AM on June 4, 2020 [2 favorites]
posted by slogger at 9:45 AM on June 4, 2020 [2 favorites]
Although the authors of the article did do some propensity score adjustments, a number of statisticians (including Gelman in the original post) have also complained that they feel this was done poorly -- I believe that was what prompted the initial scrutiny of the paper, before people realized the data didn't make any sense and might be made up entirely.
posted by vogon_poet at 11:50 AM on June 4, 2020 [2 favorites]
posted by vogon_poet at 11:50 AM on June 4, 2020 [2 favorites]
OK, I wondered how these drugs actually did whatever they were supposed to do and wow, I did not think it would be this fascinating. Count me in for Camp "We should be testing this drug"
posted by rebent at 2:12 PM on June 4, 2020
posted by rebent at 2:12 PM on June 4, 2020
Lancet, New England Journal retract Covid-19 studies, including one that raised safety concerns about malaria drugs. “We can no longer vouch for the veracity of the primary data sources,... ”
posted by StephenB at 1:40 AM on June 5, 2020
posted by StephenB at 1:40 AM on June 5, 2020
Tide's in: Retracted studies may have damaged public trust in science, top researchers fear (The Guardian, June 5, 2020)
posted by Iris Gambol at 6:15 PM on June 8, 2020
posted by Iris Gambol at 6:15 PM on June 8, 2020
A large, randomized UK trial of several different potential Covid-19 therapies has halted the hydroxychloroquine arm of the study after finding no benefit. The leader of the study was blunt:
This study, the VA study, and another randomized study showing it had no preventative value all make a very convincing case that it is unhelpful at best and harmful at worst. An enormous amount of incredibly valuable time, expertise, and money were wasted proving that Didier Raoult's unfounded hypothesis was indeed wrong. Much of that effort was only required because Donald Trump kept pushing it. Raoult and Trump have cost untold millions of dollars and many lives, directly and indirectly.
posted by jedicus at 10:36 AM on June 10, 2020 [4 favorites]
“This is not a treatment for COVID-19. It doesn’t work,” Martin Landray, an Oxford University professor who is co-leading the RECOVERY trial, told reporters.Again, this is a large, randomized trial that was completely independent of the tainted/falsified Surgisphere data.
This study, the VA study, and another randomized study showing it had no preventative value all make a very convincing case that it is unhelpful at best and harmful at worst. An enormous amount of incredibly valuable time, expertise, and money were wasted proving that Didier Raoult's unfounded hypothesis was indeed wrong. Much of that effort was only required because Donald Trump kept pushing it. Raoult and Trump have cost untold millions of dollars and many lives, directly and indirectly.
posted by jedicus at 10:36 AM on June 10, 2020 [4 favorites]
FDA revokes emergency authorization for antimalarial drugs touted by Trump as covid-19 treatment (WaPo, June 15, 2020) The FDA said it had concluded that it “is no longer reasonable to believe” that hydroxychloroquine and chloroquine are effective in treating the disease. “Nor is it reasonable to believe,” the agency said, "that the known and potential benefits of these products outweigh their known and potential risks.”
The FDA’s change in position was outlined in a letter from Denise Hinton, the agency’s chief scientist, to Gary Disbrow, the acting director of the Biomedical Advanced Research and Development Authority, who requested the revocation. The FDA noted in the letter that the request was an “interagency effort” prompted by new clinical trial results and other information.
[Also at CNN; neither article mentions whistleblower Dr. Rick Bright, of "Top vaccine expert says he was fired for resisting Trump on hydroxychloroquine" (The Guardian, April 22, 2020) fame]
posted by Iris Gambol at 12:53 PM on June 15, 2020 [1 favorite]
The FDA’s change in position was outlined in a letter from Denise Hinton, the agency’s chief scientist, to Gary Disbrow, the acting director of the Biomedical Advanced Research and Development Authority, who requested the revocation. The FDA noted in the letter that the request was an “interagency effort” prompted by new clinical trial results and other information.
[Also at CNN; neither article mentions whistleblower Dr. Rick Bright, of "Top vaccine expert says he was fired for resisting Trump on hydroxychloroquine" (The Guardian, April 22, 2020) fame]
posted by Iris Gambol at 12:53 PM on June 15, 2020 [1 favorite]
« Older A Poem for this Time | Where murderous chokehold cops still earnin' a... Newer »
This thread has been archived and is closed to new comments
posted by tclark at 12:39 PM on June 3, 2020 [7 favorites]