Charles R. Twardy

Follow @ctwardy on Micro.blog.

Avoiding the High Cost of Peer Review Failures

Wilderness & Environmental Medicine’s editor Neil Pollock frequently writes an Editor’s note about peer review, publishing, and reproducibility. I wish he were more concise, but he often raises good points, and I expect none of my colleagues read WEM, so here’s a digest. This month’s essay is Avoiding the High Cost of Peer Review Failures.
Skipping a review of the problem, and a note about unintended consequences of open access, we get to some clear caveat emptor:

The legitimacy of journals cannot be confirmed by name or impact factor scores, and often not by promises made regarding peer-review standards…. Many predatory journals have credible and even inspiring names. They can also manufacture or manipulate impact factor scores and blatantly mislead regarding peer-review practices. [including ignoring reviews]

Caveat emptor:

Mindfulness, and more than a small degree of cynicism, is necessary to critically evaluate the legitimacy of any journal.

How you will be tempted to fail:

…getting through “peer review” with no more than trivial editorial comments may seem reasonable for the person or team thinking that their words are gold.

Being invited to review may also confer an aura of legitimacy. Such events could result in additional manuscripts being submitted to the same journal.

Stop being cats:

The inherently independent nature of researchers can lead to avoidance of conversations regarding research publication. [Discuss concerns and establish institutional guidelines to avoid being trapped by predatory journals.]

For example: [breaking his sentences into bullets]

  • Did a person or team publish in such a journal inadvertently or to get around research weaknesses?
  • Should full (or any) credit be given for publications in journals found to be predatory?
  • Should job candidates with a history of publication in predatory journals be considered?
  • Should articles published in journals employing predatory practices count in tenure packages?
  • What scrutiny of the effort of flagged authors is warranted?

It’s been awhile since I read Shapin, but I’m reminded of early scientific societies and the network of trust built up by personally recommending new members. At this point I can’t see submitting to a journal that isn’t already known to my colleagues and field. But replicability indices (here | here | here…) show even that is not enough - Pollock is right that your department and institution has to ask some hard questions.