“You pay less, I earn more”… or how UC and Springer Nature made a seemingly win-win agreement

Win Win 306/365
CC-BY-ND Dennis Skley

And yet another agreement! While it was celebrated over the ocean as “the largest OA deal ever signed in the US” or a “milestone” for OA, we Europeans are now used to these “groundbreaking” contracts announcements every other week. So much that I have already written one in March on the German Springer/DEAL and another one in May on the Faustian Elsevier/Dutch consortium. So all things come in threes, and for a good reason, as Californians give us some food for thought on the financial side of the agreement.

First of all, it should be noted that the contract between Springer Nature (SN) and the University of California (UC) has not yet been written, but that only the Memorandum of Understanding (MoU) was made public this week1. This publication derives from a clear commitment on the part of the universities to make the negotiation processes and the principles governing the choice of subscription, support or no deal transparent to the local academic communities, but also more broadly to all stakeholders interested in these issues.

As we are almost in the middle of the year, the fact that the agreement has been signed for the years 2020 to 2023 has a first important consequence: all the mechanisms necessary for the identification of authors, for the various payments and for monitoring will probably not be in place before the end of the year (SN is committed to this by 1 January 2021). In practice, UC will pay in 2020 an undisclosed amount name “UC 2020 spend” for a Read & Publish in which the Publish part will be free of charge. It is only over the next three years that mechanisms will appear, which combined originality is at the heart of this post.

The Muti-payer model.
Getting authors and funders involved

One of the originalities of this contract with Springer is the adoption of a model first experimented in the UC/PLOS agreement, with the splitting of an APC into two distinct blocks: the first 1000 dollars which will be systematically paid by the university and the rest which will be paid by the authors if they have the possibility to do so. This mechanism smells like a device invented by economists, and it is one, a professor at UC Berkeley, who describes its purpose in The Scientist:

“In the US, there already were multiple funding sources—libraries paid for subscriptions, and when authors wanted to publish open access, they paid a surcharge on top of that out of their funds,” says MacKie-Mason. “The key thing here is that we’re integrating those into a single contract. That creates cost control for the institutions and the researchers [during the transition to open access], which is critical because the cost of scholarly publishing has been exploding.”

So the solution to the “new serial crisis” would be to imply authors as UC people have repeatedly stated2, but aren’t they already with classical “one shot APCs”? The idea to combine APC with institutionnal support in a contract is here pushed to the limit as we will see. In some “transformative agreements”, there is no way for a third party to understand who in the end pays what and from which source, especially in consortiia stteings. Here, it is quite the opposite as in the who MoU, a clear separation is made between two sources:

  1. The UC – would it be California Digitaly Library or UC itself – takes in charge a 750,000$ reading fee, 1000$ for each APC and, as we will detail, more if authors can’t pay. All these will be counted apart in “UC Fully OA Spend”, “UC Hybrid Spend” and of course the reading fee.
  2. The authors would pay the “APC remainder”, whoever is the original funder, and these sums play a very limited role into the contract, are not agregated under specific names.

So the splitting is not only made for each article, but for the total contract as “cost regulation” supported by Mackie-Manson but in fact only on the UC side, authors could spend whatever they wish on APC, and benefit from the UC participation. In consequence, as authors shall pay, they have the possibility to opt out of OA in hybrid journals, which is the default option. Consequently, the deal does not guarantee that all UC corresponding authors articles will be OA, but only those who wish so and, to some extent, that are ready to pay, favorable to hybrid journals, or APC gold open access supporters. The division and authors’ choice are highly visible in an exception in the contract. If, despite very short deadlines, SN was able to implement the entire workflow before the end of 2020, then it could start invoicing APCs. Under no circumstances would UC have anything to pay, but authors could be solicited:

Should Springer Nature implement the Multi-payer Model before January 1, 2021, Springer Nature may begin collecting the APC Remainder under the terms of the model […]. If the corresponding author does not have research funds available to cover the APC Remainder, then Springer Nature shall not collect an APC for those articles. No UC Fully OA or Hybrid Spend payments will be charged during this time (article 3.8.2).

It is hard to imagine a corresponding author who can get free APC deciding to pay, unless their grant is nearing completion and they cannot spend it otherwise. But this provision does indeed support the idea of two decoupled payers, as the rules applying to them may differ, the first (UC) not paying in 2020 before being obliged to contribute, the second remaining in a logic of choice throughout the contract. But what exactly are the amounts to be paid?

Price, Volume, Participation :
an equation to determine an Hybrid bill

The price calculation formulas are not yet complete, since the agreement is not signed, but the foreseeable variations are known throughout the contract. For full OA journals, there will be a base price in 2020, with a maximum increase of 3.5% per year. This base price is certainly not the catalog price, since it is specified that ” If at any time during the agreement the then-current list price APC is lower than the APC to be charged under the agreement, the current, lower APC will be charged instead” (art. 3.3). The issue of prices and volumes is most complex when it comes to hybrids APC. First of all, unit pricing is almost constant with the same prices in 2020, 2021 and 2022, and a maximum increase of 2% in 2023. But while the paid volume published in full OA appears unlimited, the paid volume published in Hybrid journals is very constrained.

First the number of articles published in Hybrid by the corresponding authors in 2019 and 2020 is calculated, and the smallest of the values is taken, which becomes the Base article number. The minimum volume of articles is then simply defined as 85% of this number, over time. On the other hand, the maximum number depends on two variables: first, an “inflation” of the authorized volume, of 5% per year, then a calculation that depends on the effective participation of the authors in the publication scheme. Indeed, the parties expect that between 30% and 40% of the authors of articles will choose to publish in hybrid AO rather than revert to a paywalled publication. (orange curve) If the program is successful, more than 60% of the authors adhere, then the red curve defines the maximum number of articles; symmetrically, in case of failure – less than 30% – it is the yellow curve that defines this maximum number.

In a close fashion to the agreement with DEAL, Springer defines a volume control on Hybrid, which can lead up to a third more articles published than the current Hybrid APC. But the consequences of going over this limit are very different than the German counterpart : UC is not anymore paying its 1000$ above the maximum, but authors – if they chose so, must pay the APC remainder. On the other end, if the minimum is not reached, UC shall pay “the average hybrid APC for UC corresponding authors from the previous year for the number of articles necessary to bring the total to the minimum. In 2021, the average hybrid APC from 2019 ($3208) shall be used.” So Springer Nature is sure to have (almost) its money back and UC has a control mechanism which prevents a high rise of its Hybrid spend by volume control.

Hard capping the total costs.
Will UC pay less in the end?

Until now, it seems that we analyse another “cost-neutral” agreement that in practice could absolutely become a high rise contract : APC individual price inflation, unlimited payment for full OA articles, controlled max rise of hybrid OA would contribute to a larger bill for UC. Then comes the most original point of the UC/SN contract : a hard cap on the sum of fluctuating bills. In fact, some agreements, typically the JISC ones, include a price control that says “we will pay this, period”. Of course, the trade off is most often a defined, limited volume. Here, as we read it in article 3.6.

In each year of the contract, the Total UC Spend shall be subject to a fee control mechanism, as set out below. All fee control mechanisms are computed in relation to the license fees paid by UC for Springer journals, Adis Journals, Palgrave journals, andacademic journals on nature.com in 2020 (“UC 2020 Spend”).

So the starting “subscription” – ie Read & Publish – set price caps the whole price of the contract, once again in a very precise and shall I write, twisted way. Starting from the “UC 2020 spend”, in 2021 you can not exceed 95% of that sum: if it is the case, then UC gets some reading fee part, and if it is not enough, refunding from SN. So the max is clear and -5% compared to the starting year. But in 2022 and 2023, you can not exceed 98% of that sum ; if it is the case you get only the Reading fee back and nothing else. In other words, there is in fact no fixed maximum payment, and certainly not a garantuee that UC would pay less in 2022 and 2023 than in 2020, and as we don’t know what were the different bills, even more less than 20193. The UC part is very confident on the result as the associate executive director of the California Digital Library, Ivy Anderson, stated : “The new agreement is expected to save the system money overall, but the exact cost will depend on the number of articles UC researchers publish”.

Whatever the final outcome, and one can think, given the complexity of the provisions that the UC part has run many simulations on its final bill, there are three lessons to be learned from this MoU. First, in the absence of price transparency, it is difficult for outsiders to determine whether an agreement is really financially interesting or whether it mechanically leads, as with subscription formulas, to higher prices paid by higher education institutions. Secondly, this agreement builds a link between the payment of authors and that of the university: it therefore allows the direct inclusion of research funders, while ensuring traceability and monitoring of flows for each of the parties. It also contains incentives on the behaviour of authors, who would benefit from using the UC workflow to partially or totally reduce their own payment. But it is the ability to capture money from funders, third parties to the contract, that is striking, with certainly Coalition S members in mind.

Consequently, thirdly, it is the de facto guarantee of Springer’s revenues by encouraging new spending in the form of APC in subsidizing them. Making new provisions to turn the Nature journals into a hybrid goes in the same direction. In a similar way to “Pure Publish” agreements that goes with a discount on APC, the UC agreement is a transformative one as it explicitly changes universities from fund providers to fund collectors for publishers, with the hope of a diminishing or stable bill in exchange for that service.

  1. We saw on the Dutch case that there could be quite significant differences between an MoU and the actual contract []
  2. See this piece on Impact of Social Sciences LSE Blog []
  3. I previously wrongly tweeted that they would pay less, as I thought the reference was UC 2019 spending []

Living in a post-Ingelfinger world or… The HCQ-COVID-19 publication show

Disclaimer: this post does not address the merits of the treatments proposed by the IHU team nor their risks, and even less the fact that Prof. Raoult would be a genius, a madman or a top scientist who got lost along the way.

It all started with a video, posted on February 25th, 2020, then entitled “Covid-19: endgame”, and put by IHU Méditerranée-Infection on Youtube. In that less than 2 mn video clip, extracted from the end of a seminar, Didier Raoult states that COVID-19 is “probably the easiest respiratory infection to treat” and that chloroquine (CQ) is effective and already “recommended for all clinically positive cases” in China. It wasn’t the first time this infectious disease star recommended CQ and its cousin molecule, Hydroxychloroquine (HCQ) to fight viral infections. Indeed, as early as 2007, he presented these drugs as “an interesting weapon to face present and future infectious diseases worldwide” in the International Journal of Antimicrobial Agents. (IJAA). Framed as a recycling of these antimalarial drugs, the article constituted a literature review, mainly of in vitro studies, and was part of the scientific and medical strategy of the IHU, the repositioning of old molecules, free of rights, towards new uses. And this possibility of reuse was taken up in a letter sent on February 11th, 2020 to the same journal (IJAA), accepted the same day and published on Februray 15th.

The series of IJAA publications continued. The day after the Youtube video, a new article was submitted, specifically dedicated to the use of CQ as a treatment for the COVID-19 epidemic. Accepted the next day, February 27th and published a week later, it repeated the efficacy claims observed by the Chinese and as a result of clinical recommendation. This assertion is based in particular on one of the strangest references I have ever encountered. Indeed, it is a letter of exactly ten lines published in BioSciences Trend, which body is copied below :

The coronavirus disease 2019 (COVID-19) virus is spreading rapidly, and scientists are endeavoring to discover drugs for its efficacious treatment in China. Chloroquine phosphate, an old drug for treatment of malaria, is shown to have apparent efficacy and acceptable safety against COVID-19 associated pneumonia in multicenter clinical trials conducted in China. The drug is recommended to be included in the next version of the Guidelines for the Prevention, Diagnosis, and Treatment of Pneumonia Caused by COVID-19 issued by the National Health Commission of the People’s Republic of China for treatment of COVID-19 infection in larger populations in the future.

Defined as an “abstract” on the journal site, but without any other body of text, this “article” doesn’t seem to be fully supported by the 7 references listed. It relies mainly an in vitro study from early February, already widely cited, which indicates that CQ could be effective. In fact, It wasn’t until February 29 that the results of a CQ clinical study were submitted to a Chinese journal, before being published on March 6. But let’s go back to the IHU timeline.

Ten days later, a second video was put on Youtube, presenting the results of an observational study made in Marseille and showing the effects of HCQ alone and in combination with an antibiotic, azithromycine (AZ). So there was a slight shift: going from CQ to HCQ and adding an antibiotic. The main result is only the absence of virus in nose and throat, so it is not clinical results but Didier Raoult drew from results to tell his audience their consequences for the clinical institution he manages:

The fact that you no longer have the virus changes the prognosis. Actually, that’s what infectious diseases are all about. If you don’t have the germ anymore, you’re saved… You have a right to be tested here, and if you’re tested, you have a right to be treated here. That is what we will do.

So basically, for him, results were so good that you HAD to treat people when they are tested positive. No more trials or research needed, the time for clinical medicine had come, hoping other places would follow his lead. Slides were available on the same webpage but no link to an existing paper, though the same day, but not mentionned in the video, a preprint was submitted to MedrXiv Simultaneously, as it is often the case with biomedical preprints, it was submitted to a journal… the ever-welcoming IJAA, who accepted it, as usual, one day later and published it on March, 20th. Before we come to the extraordinary fate of this paper, let us go back to the title of this post and its interest at this point.

From preprints to preprints:
the life and dearth of the Ingelfinger rule

We can observe from the two examples above a pattern of scientific communication: the IHU first posts videos, then produces preprints and finally publishes articles in academic journals – here IJAA. This is very unusual, at least in contemporary times, but happened in various ways during centuries of scholarly communication. The idea that you had first to communicate with your peers through a journal before getting to “the public” is neither constant nor dominating in all disciplines. In our era, it was pushed at a key moment in the mid-1960s. Back then, a first wave of preprints was being supported by NIH and was gaining momentuml in some biomed communities through Information Exchange Groups (IEG) that would circulate by air mail printed copies of unpublished manuscripts1. Nature started a campaign against the “preprint galore” and a few European and US biology and biochemistry journals editors-in-chief met in Vienna in 1966 to get rid of them by stating that : “The journals listed below will not consider manuscripts for publication if preprints, of essentially identical content, are to be distributed, in substantial numbers, by an agency independent of the author or of the publisher of the journal. “2

That led to the termination of the IEG experiment by the NIH in 1967. Two years later, the New England Journal of Medicine (NEJM) editor-in-chief, Franz J. Ingelfinger, coined the rule of acceptance of a paper, based on his interpretation of “sole contribution”, de facto forbidding even “circulation-controlled journals” to print something ahead of the NEJM3. In the same sentence, he remarkably included “news media”: he therefore aimed not only at the exclusive circulation of the article within scientific communities, but also to the prohibition of dissemination of its content to journalists and other medical news enthusiasts. In the early 1970s, his work to promote this exclusivity had a double effect: this practice was given the name Ingelfinger Rule, and many high-profile journals adopted it explicitely. While at the beginning of the 21st century the Ingelfinger Rule was often interpreted as a means to fight against the duplication of papers, its aims were more about controlling the circulation of knowledge in order to protect the newsworthiness of “general medical journals”4 and to organize communication about medical academic papers in a specifc way, favorable to a limited number of journals.

Indeed, as Vincent Kiernan beautifully described in his 1997 article5, the Ingelfinger Rule had become prevalent in Anglo-American journals. It is in particular the efforts of the International Committe of Journal Medical Editors (ICMJE) that built it as a “publishing standard”, which effect was for these journals and their editors-in-chief to simultaneously operate a double control:

  1. control on the authors by requiring them not to reveal the content of their articles, and even less so share the figures and other synthetic representations of results.
  2. control on journalists by providing them with preprint copies of articles in advance, while imposing an embargo on them until actual publication by the journal.

As a result, the general press (free of charge) advertises the content of the journals – it is not an article by Dr. X & Y., but an article from the NEJM or The Lancet – and organizes the dissemination of “medical discoveries” by strengthening the influence of these journals both within academic communities and within press professionnals and the general public. To conclude his paper, Kiernan questions the durability of such practics in the Internet-era and points out the effect of ArXiv preprints, citing the efforts of the ICMJE to extend the Ingelfinger rule to e-prints, with the argument of the direct consequences of biaised or false medical knowledge for the public.

The biomedical field resisted 15 more years to preprints and the Ingelfinger Rule largely stood6, even if it was adapted to emergency contexts, such as the AIDS epidemic. But Kiernan’s forecast came into reality, notably with the creation of BiorXiv in 2013 and the subsequent success of preprints in biology and biomedicine, until preprints became quasi-articles. Consequently, the Ingelfinger rule was dropped by numerous journals and publishers, even if NEJM itself keeps a case by case policy.

Prof. Raoult and his videos, possibly including slides with the figures so dear to the NEJM, thus live in a post-Ingelfinger world, in which academics can directly ensure their communication, not only in terms of content, but also in terms of comments, criticism, reporting or response. Indeed, we will see that the primary communication is not the only one modified by the abandonment of this rule, but the complete organization of the journal’s centrality in the whole chain of scientific communication.

Chaos and creation around one paper

Let us go back to this first publication by Raoult’s team on the effects of HCQ on viral porting, published in the IJAA on March 20, 2020. At the time of writing this post, the article has received 1124 citations according to Google Scholar but also thousands of tweets, blog posts and other references in press articles according to PlumX, a company owned by Elsevier, itself the IJAA publisher. The early circulation of the article was not based on a press release of the IJAA, but on Raoult’s own video and that of his various networks. As Wired recounts, with the help of a lawyer, a retired doctor, a shared google doc and an interview to Fox News, an heterogeneous assemblage à la Bruno Latour, the study published in the IJAA won a quote in a Tweet from the President of the United States the day after its publication:

That Trump endorsement of course had enormous consequences on the HCQ market, the launching of clinical trials, self-medication HCQ practices and the scope of public discussion on the efficacy and dangers of such a treatment. We won’t directly treat these important questions here, but keep on following the exotic trajectory of the publication itself. Simultaneouly to the Trump tweet, a PubPeer thread was lauched on the famous post-publication comment platform, but contrary to the Voinnet affair7, most of the first commentators signed their critiques. Among other topics, the communication trajectory of the paper helped the critique: for example, Leonid Schneider noticed the discrepancies between the figures attached to the video and the ones drawn in the published paper.

Above and beyond Pubpeer, three reviews were quickly published, questionning many aspects of the IJAA paper. The first one is a twitter thread by a master student on March, 22nd ; the second one is a zenodo 18-pages paper by three British/Irish statisticians on March, 23rd ; the third one was a blog post by a very famous Dutch microbiologist and scientific misconduct specialist, Elizabeth Bik on March, 24th. So only four days after publication – still four times the actual reviewing IJAA delay – the paper is being trounced online. Among the many points, let us note that the publishing history was being questioned, some noticing the differences between the first “preprint” on IHU website and the final paper, others underlying the lack of changes, an hint for them on how tenuous the peer review process has been., the 24h delay being surprising to every commentator. The fact that one of the authors was also the editor-in-chief of IJAA was underlined, as well as the “vanishing” of 6 patients (among 26 treated by the combined drugs), which could completly change the statistical value of the results.

While Prof. Raoult was fighting for HCQ to be authorized for general physicians in France, the online discussion kept on going until the learned society, the International Society of Antimicrobial Chemotherapy (ISAC) behind the journal, made a troubling press relase on April 3rd:

“ISAC shares the concerns regarding the above article published recently in the International Journal of Antimicrobial Agents (IJAA). The ISAC Board believes the article does not meet the Society’s expected standard, especially relating to the lack of better explanations of the inclusion criteria and the triage of patients to ensure patient safety. Despite some suggestions online as to the reliability of the article’s peer review process, the process did adhere to the industry’s peer review rules. Given his role as Editor in Chief of this journal, Jean-Marc Rolain had no involvement in the peer review of the manuscript and has no access to information regarding its peer review. Full responsibility for the manuscript’s peer review process was delegated to an Associate Editor. Although ISAC recognises it is important to help the scientific community by publishing new data fast, this cannot be at the cost of reducing scientific scrutiny and best practices. Both Editors in Chief of our journals (IJAA and Journal of Global Antimicrobial Resistance) are in full agreement.”

So the paper has a lot of problems, but stuck by the peer review rules. This cryptic PR became even more troubling a week later as it was “replaced” by an ISAC and Elsevier press release. In fact, the journal is not owned by the learned society, but by the Publisher, only being an “official society journal”. This second PR is streamlined compared to the first one as the “not meeting standard” sentence has disappeard and an announcement of post-publication peer review audit. Through this example, we measure how much different is the situation from what was prevalent under the Ingelfinger Rule. But it is with another Raoult’s team paper that science communication came back to its 17th century roots.

From presidential visit to media frenzy:
the marginalization of journals in scholarly communication

After a follow-up study published at the end of March which made less headlines and as some HCQ trials on diverse patient groups were starting to being published, it is with another observationnal study that Prof. Raoult showed the world how he was really managing scholarly communication. On April 9th, the French president, Emmanuel Macron unexpectidely visits IHU Mediterrannée and meets with Prof. Raoult, who presents him the results of its ongoing study. There was no press, but members of the IHU had recorded the arrival of Macron and posted it, making it available to the whole French media.

Here we need to go back to the origins of scientific communication, even before journals were born, when the quality of witnesses – meaning mostly royalty kinship – were an important element of the credit given to the narrative of an experiment or an observation8. In our times, it became a two-way credit flux: Macron was showing his will to base public health on evidence-based, all the more given by a star scientist, while Raoult was legitimizing his position in the French public health landscape, where critics of his methods and results were numerous.

The next day, Raoult made public his first results, not in the form of a preprint or slides with an associated video, but as a simple tweet with the abstract and a summary table.

This tweet was of course massively picked up, commented on and aroused strong media interest, all the more so as the results reinforced those of the previous study by moving from a purely biological effect to a clinical effect: “The HCQ-AZ combination, when started immediately after diagnosis, is a safe and efficient treatment for COVID-19, with a mortality rate of 0.5%, in elderly patients. It avoids worsening and clears virus persistence and contagiosity in most cases. ” Four days later, Prof. Raoult was invited in Dr Oz show, a famous TV host in the US, harshly criticized for his often unproven medical advice.

At the day of the interview, there was no preprint and the paper was not even submitted to a journal. Yet, Prof. Raoult presents his results as facts. It was only on the 20th that the manuscript was sent to Travel Medicine and Infectious Disease,9, with 10 days for peer review and a publication on May, 5th. Tens of thousands of comments on Facebook and tweets have followed according to PlumX,10 though media as much endorsed the results as they reported the methodological limits os the study – mostly the absence of a control group.

This study is undoubtedly a borderline case in the marginalization of journals, with communication aimed primarily at peers being out of step with announcements to political leaders and media outlets. Nevertheless, the massive availability of preprints, abstracts or other materials on topics such as the effectiveness of masks or tests, the persistence of coronavirus on this or that surface, or cases of cure, has led to significant media coverage. From the point of view of the public authorities and the general public, it could have strengthened the authority of academic journals, again in a position to assert their necessity as a obligatory passage point for public dissemination. But this return to grace assumed that the journal peer review is an effective barrier against “bad science”, an hyptohesis which has been dismissed by thirty years of studies and literature.

Prestige journals in epidemic times:
an economy of reputation crumbling down?

Indeed, prestige journals are bad for methodology: they don’t follow their own standards on reporting clinical trials, and more generally disicplinary standards. Yet they remain prized places to publish, even during the pandemic where preprints are so trendy because of the urgency to share results and knowledge. And some HCQ papers have been quietly published in such journals, until one observationnal study seemed to close the dabate on this treatment efficacy and risks.

For this study, there was no advance communication, no preprint but a straight article published in The Lancet by 4 authors. Oh, yes, there is a little gem still there on Twitter : two days before online publication, the “first author” answered a tweet by Richard Horton, editor-in-chief of The Lancet:

https://twitter.com/MRMehraMD/status/1263034198870429696

The reaffirmation of their confidence in the journal peer review system, even in times of health emergency, is comforting. And their trust is shared by the highest health authorities. On May 22nd, the study was published and asserted on the basis of a gigantic aggregation of almost worldwide patient databases that HCQ is not only inefficient, but also a very dangerous for COVID-19 patients. This announcement came at a time when many ongoing trials are displaying HCQ treatment arms. As a result, the WHO decided the next day to evaluate the continuation of its Solidarity study and announced its position on May 25th:

“Having met on 23 May 2020, the Executive Group of the Solidarity Trial decided to implement a temporary pause of the hydroxychloroquine arm of the trial, because of concerns raised about the safety of the drug. This decision was taken as a precaution while the safety data were reviewed by the Data Safety and Monitoring Committee of the Solidarity Trial. “

Nevertheless, in a manner similar to Prof. Raoult’s article, statisticians then look at the content of the article, the data it provides, and begin to point out obvious errors. But for some it was more a police investigation than data re-analysis: how can there be only 4 authors (and no acknowledgements) for such a study? Why are the hospitals involved not mentioned? What is this mysterious enterprise – Surgisphere – unknown until recently, which provides this data? What is the career of its manager and co-author of the paper? Putting apart questions about the company, 6 days after publication, they end up writing an open letter to the authors and the journal, signed by 201 colleagues and endorsed by James Watson11. They mainly point out the necessity to open the data, even more considering the extraordinary results, and describe obvious errors, questionning the quality of the database and the way (including ethics) data was gathered.

The Lancet and the authors were very prompt in responding to these criticisms: in fact, on May 30 a correction was published, covering very minor aspects. : “the numbers of participants from Asia and Australia should have been 8101 (8·4%) and 63 (0·1%), respectively. One hospital self-designated as belonging to the Australasia continental designation should have been assigned to the Asian continental designation.” Of course, the conclusion was a classic in those corrections : “There have been no changes to the findings of the paper.” But critics keep on pushing on the problems, would they be HCQ supporters, Prof. Raoult himself stating “fake data” or “manipulated data” on Twitter or clinicians trying to find coherence between the papers’ data and their own. So, only 3 days after the correction, The Lancet puts an expression of concern on the paper:

“Although an independent audit of the provenance and validity of the data has been commissioned by the authors not affiliated with Surgisphere and is ongoing, with results expected very shortly, we are issuing an Expression of Concern to alert readers to the fact that serious scientific questions have been brought to our attention”.

The paper was still saveable, thanks to the independant impeding audit. Alas, another 2 days and the 3 authors who do not belong to Surgisphere threw in the towel by stating they haven’t seen the data, and demanded the retraction of the article. The Lancet officialized it, provoking expression of outrage, the questioning of the seriousness of the journal and… the reactivation of the suspended trials. Thus, in less than a week, the worldwide study published in what many consider to be “one of the best medical journals in the world” has been awarded the 3 labels commonly used in post-publication peer review – Correction, Expression of Concern, Retraction12 – nullifying the evidence claimed on May, 22nd. But the Surgisphere story goes beyond that article: another paper, published by NEJM on the “same kind of data” was retracted on the same day. Moreover, there are at last two regions – South America and Africa – which have and will suffer from public health policies being developed on preprints and data published by Surgisphere. While #LancetGate was trending on twitter, in-depth inquiries were being made on Surgisphere and the 4th author of study who, ironically, coauthored a paper entitled : “Combating Fraud in Medical Research’ in 2013 !

Science at its best:
boring, negative results

To conclude this story on scholarly communication, we have to add that most HCQ articles have not been given the same media treatment and have not been communicated in fancy ways by authors: a preprint on BiorXiv or MedrXiv, then an article with often no spectacular results and limitations because of the number of patients, their previous health conditions, incomparability between groups, etc. One day before the retractions, the same NEJM published the first randomized-control trial on post-exposition use of HCQ, so close to the “Raoult treatment” – AZ not being included. Here is part of the abstract published:
Side effects were more common with hydroxychloroquine than with placebo (40.1% vs. 16.8%), but no serious adverse reactions were reported.After high-risk or moderate-risk exposure to Covid-19, hydroxychloroquine did not prevent illness compatible with Covid-19 or confirmed infection when used as postexposure prophylaxis within 4 days after exposure.”

What do we get from this abstract? That the article is a typical example of those “negative results” that fail to be published, leading to significant biases in the evaluation of treatments in clinical trials through a “publication bias”13. And yet, not because of its own interest, originiality, breakthrough knowledge, but because of its relevance to public health in an epidemic situation, this trial has been published by the other “world’s best medical journal”.

While predictions of “really bad science to come” have sounded true for most commenters and supported by a high number of retractions, the COVID-19 academic publication landscape has also shown a massive uptake on preprints, public education on scientific controversies, conflict of interest and statistical analysis and furthermore… yes, publication of null results in prestige journals. Whether you think this is a total mess and you prefered the Ingelfinger rule depends on the way you conceive academic research and scholarly communication. Back then, preprints were non-existent in biology and social networks had to be invented, but The Lancet published the Wakefield paper on the link between MMR vaccine and autism. Was it a better time?

  1. See Cobb, Matthew., 2017. “The prehistory of biology preprints: A forgotten experiment from the 1960s.” PLoS biology 15.11 []
  2. Thorpe, W. V. (1967). International Statement on I nformation Exchange Groups. Science, 155(3767), 1195-1196. []
  3. Ingelfinger, Franz. “Definition of” sole contribution”.” N Engl J Med 281 (1969): 676-677. []
  4. Ingelfinger, F. J. (1977). The general medical journal: for readers or repositories?. New England Journal of Medicine, 296(22), 1258-1264. []
  5. Kiernan, V. (1997). Ingelfinger, embargoes, and other controls on the dissemination of science news. Science Communication, 18(4), 297-319. []
  6. See as an example this defense of the rule by Nature in 2010, five years after having written they were ok with preprint servers []
  7. see Torny, Didier. “Pubpeer: vigilante science, journal club or alarm raiser? The controversies over anonymity in post-publication peer review.” 2018 and Guaspare, Catherine, and Emmanuel Didier. “The Voinnet Affair: Testing the Norms of Scientific Image Management.Gaming the Metrics: Misconduct and Manipulation in Academic Research (2020): 157. []
  8. See the classic book Shapin, S., & Schaffer, S. (1985). Leviathan and the air-pump: Hobbes, Boyle, and the experimental life (Vol. 109). Princeton University Press []
  9. A journal in which one of the authors is an associate editor have underlined Raoult’s critics []
  10. The story is quite different within the academic world with “only” 21 citations until now, far much less than the March study. In fact, many observationnal studies and trials were competing with this study []
  11. EDIT June 9th: James Watson made a fantastic interview on an australian radio where he gets into detail about how he started and run this 5-days inquiry, hear it there []
  12. On the standization of journal policies, see Pontille, D., & Torny, D. (2017). Beyond Fact Checking: Reconsidering the Status of Truth of Published Articles. []
  13. There is a huge literature on this topic in the last 30 years, see as an example this The Lancet article, Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867-872. []