Last week there was another very public case of a journal article being retracted as a result of academic misconduct. This time it was in the Journal of the American Medical Association (JAMA), with the lead author – Dr Anna Ahimastos, working at Melbourne’s Baker IDI – reportedly admitting she fabricated data.
Sadly, the story is all-too familiar. But this is not to say that science is imperiled, only that we need to ensure the reward and support structures in academia promote the best practices rather than corner cutting.
We have only recently begun looking closely at how the scientific literature could function better, and what can go wrong. And there are conflicting opinions on how to handle underlying problems.
Peer review is currently the primary tool we have for assessing papers prior to publication. Although it has its strengths, especially when overseen by skilled editors, it can’t pick up all instances of fraud or sloppy scientific practices.
In the past these errors may have lain hidden for many years, or never come to light. Now, post publication scrutiny is picking up more and more papers with questionable data. This is leading to corrections, or even retractions. Websites such as Retraction Watch have sprung up to document these retractions.
Peerless research
To non-academics, this might all seem rather surprising. Isn’t science governed by strict protocols for performing and reporting research?
Well, no. Unlike industrial processes, for example, which have standard operating procedures and oversight, science is usually organised locally. Expert laboratory heads typically have the responsibility for the oversight of their laboratories’ work.
Many laboratories work as part of larger collaborations, which may have their own checks and balances in place, as do the academic institutions to which they belong. Even so, ultimately the researchers and individual laboratories are responsible for their own work.
The medical sciences have developed their own standards of reporting studies, including clinical trials. But even these standards are not employed universally.
The system of rewards within science is possibly even more perplexing. Academia is a highly competitive profession. The basic training in science is a PhD, with more than 6,000 awarded each year in Australia alone, which is many more than can ever end up as career researchers, even at the lowest level.
The situation gets worse the more senior a researcher gets. According to a 2013 discussion document less than 5% of those who were originally awarded PhDs find permanent academic positions. Even these senior researchers rarely have permanent positions, but are instead expected to compete for funding every few years.
And the primary way academics compete is in the number of papers they publish in peer reviewed journals, especially the handful of what are considered to be top journals, such as Science, http://www.nature.com/link text and The Lancet.
Under pressure
Why does this all matter? Doesn’t this competition lead to selection of the best of the best in research and a faster pace of advancement of science? In fact, the reverse may be the case.
In a seminal paper published in 2005, provocatively titled Why Most Published Research Findings are False, John Ioannidis discussed a number of reasons why research may be unreliable. One finding was that papers in highly competitive areas were more likely to be false than papers in less competitive fields.
In 2014, the Nuffield Council on Bioethics probed these issues among UK researchers in a year long study. What they found was alarming.
Researchers stated that there was strong pressure on them to publish in a limited number of top journals, “resulting in important research not being published, disincentives for multidisciplinary research, authorship issues, and a lack of recognition for non-article research outputs”. Even worse was that the need to get into these top journals led to “scientists feeling tempted or under pressure to compromise on research integrity and standards”.
What can be done? Increasingly, groups of scientists are coming together to develop standards in reporting, conduct and reproducibility. Organisations such as the Committee on Publication Ethics (COPE), which I chair, advise editors on how to handle problem papers.
Perhaps most interestingly, a number of technological innovations have arisen that could lead to more reliable science, if adopted widely. Probably the most important innovation is that of Open Science, i.e. open access to research publications, and open access to the data and methodology that underpins those publications.
But we also need to develop ways to reward scientists who do make their publications, data and methodology open for scrutiny, and don’t just pursue publication in top journals.
Research data organisations, such as the Australian National Data Service (ANDS), are developing the infrastructure for systematic and standardised ways of linking to data, but as yet funders and institutions do not routinely reward such behaviour.
In the end, science is a human endeavour. And like humans everywhere, those who work in it will do what they are rewarded for, for better or for worse. So we need to make sure those reward structures are encouraging good quality research, not the opposite.
Virginia Barbour is Chair of the Committee on Publication Ethics. She was previously Chief Editor, PLOS Medicine.