Members of ACTA and ACTA STInG were amongst the signatories of the open letters by Dr James Watson, statistician at the University of Oxford Centre for Tropical Medicine and Global Health, expressing concerns about the quality of the statistical analysis and data integrity for two recent COVID-19 publications: (i) the use of hydroxychloroquine and chloroquine for treating COVID-19 in the Lancet , and (ii) the effect of pre-existing treatment with angiotensin-converting enzyme (ACE) inhibitors and angiotensin-receptor blockers (ARBs) on COVID-19 in the NEJM.

In response to the Lancet publication, the WHO, UK and France temporarily paused recruitment to hydroxychloroquine arms in ongoing COVID-19 trials and France changed its national recommendation for the use of hydroxychloroquine.

The concerns raised by the letters; An open letter to Mehra et al and The Lancet, published in the Lancet and An Open Letter to Mehra et al and The Lancet, published in NEJM resulted in both journals issuing an 'Expression of Concern' (see: The Lancet and NEJM) and subsequently the authors retracted both papers (see: The Lancet and NEJM).

The data analytic company involved, Surgisphere Corporation (Chicago, IL, USA), had also been promoting a range of online prediction tools related to COVID-19, but these too have been removed following an article in The Scientist exposing their lack of validation, in which ACTA STInG member Prof Andrew Forbes was interviewed after he deciphered the online code. 

Further details of Surgisphere have emerged in related articles published by The Guardian newspaper and the BMJ.

This situation illustrates the key role that statisticians can play, not only in scrutinising research claims, but in assuring the quality of research in the first place. There was no evidence of any statistical expertise among the authors of the retracted papers, although they acknowledged a Surgisphere employee ‘for their helpful statistical review of the manuscript’.

More importantly, it raises questions around the quality of the current process of peer-review for journals and reinforces the importance of itemising data sources (STROBE guidelines) and sharing data (journal policy statements) for external, independent validation. Dealing with a global pandemic makes scientific review standards more important than ever.