The Anxiety of Peer Review and Its Discontents

Our next contribution to the theme of Publication comes from Maria Bonn. Maria has long been involved in digital publishing, having served as director of the University of Michigan’s Scholarly Publishing Office, and, more recently, associate university librarian for publishing at the University of Michigan Library. In this post, she shares some thoughts on the future of peer review.  —bM

Peer_'Bonne_Louise_d'Avranches'_(Pyrus_communis)I have watched the IPRH posts on Publication with interest.  After all, I have spent many years close to publishing, working as a writer, a publisher, a librarian and a reader. I was pleased to be invited to write for the blog and happily agreed to consider peer review, something about which there is a lot to say. Since then, I have weighed the relative value of the titles “The Anxiety of Peer Review” and “Peer Review And Its Discontents” (the allusions themselves are telling). While I pondered, over my twitter feed one day came a pointer to a new and voluminous bibliography by omnivorous bibliographer Charles W. Bailey, entitled: “Transforming Peer Review.” This, naturally, became an occasion to ask when peer review became an activity for transformation and what precipitated that transformation?

Peer review has a venerable history in the scholarly academy, dating back, by most popular accounts, to the “Royal Society of London’s 1752 creation of a ‘Committee on Papers’ to oversee the review and selection of texts for . . . Philosophical Transactions.” (Here and throughout I rely heavily on Kathleen Fitzpatrick’s history of peer review in her book, Planned Obsolescence; of Kathleen, more anon).  The process of review, commentary and recommendation, usually undertaken by two reviewers, most often anonymously, has become the de facto standard of quality in scholarly journal literature and, in more recent decades, of scholarly monographs. At its best, peer review assures validity in both methodology and findings, it assures that important relevant literature has been addressed, and it even assists with the elegance of expression.  And yet it makes a lot of people cranky, and those who are cranky are also voluble.

24-C17-Phil Trans

Besnier’s flying machine of 1678. Likely would have benefited from peer review.
© University of Wales Lampeter.

So we often hear that peer review is broken. That assertion seems to mean a variety of things:

  • Peer review is slow: Since reviewers are usually scholars with their own substantial teaching and research loads, working for little or no incentive other than professional gratification or maybe some free books, peer review duties can be deferred for months, even years, delaying time to publication. Naturally, this is especially problematic for research that is groundbreaking or in competition with similar projects.
  • Peer review is corrupt: It is a game that can be manipulated by authors (who might only suggest reviewers who will be receptive of their work), by publishers (who may select “easy” reviewers in hopes of moving a publication into the market or “hard” reviewers to scuttle a publication that lacks market viability), or by reviewers themselves (who might advise rejecting a work that threatens or upstages their own work).
  • Peer review is inadequate: One common strain of this criticism is that the process too narrowly defines expertise and therefore does not bring in the valuable concerns of those outside the academy. A second strain expresses concern that many fields have not yet established community standards by which to evaluate new forms of expression, especially digital forms, so peer review is idiosyncratic and perhaps arbitrary.

With these possible states of breakage in mind, I turn to three examples of new publishing and the ways in which they address the need for peer review:

One: I helped to establish Philosopher’s Imprint, a now highly successful digital, open access, peer-reviewed journal of philosophy. Its editors were frustrated by the slow pace of publication in their field and wanted valuable work to reach a broader audience, and they looked to Internet publication to address these concerns. But they were wary that their colleagues would look skeptically at this new form of publication. To address this anxiety they settled on a strategy that I’ll describe glibly as “reject early and often.” They put submissions through the most stringent peer review, only publishing work that was vetted as “the very best,” a strategy that resulted in a quickly achieved reputation for high quality scholarship.

Two: Kathleen Fitzpatrick, cited above, in trying to salvage and maintain the best of traditional peer review and to also cast a broader net for guidance in her work, had her book put through a traditional university press review process, but also posted an online version supported by commenting tools, to which any reader could contribute suggestions and rebuttals. The results of this diverse peer review then became part of her ongoing research agenda into “publishing, technology and the future of the academy.”

Three: A well-published and tenured faculty member of my acquaintance was considering publishing through the university library (mostly because the price his usual Press wanted to attach to his book was, by his estimate, equivalent to locking it in a box). I appreciated his instincts and warned him that the volume might not then benefit from peer review. His response (I paraphrase): “There are eight people who know more about this than I do. I know them all. I’m going to send the manuscript to each of them and ask for comments.” He did, some of them read it, and the book was revised accordingly.

These are three briefly related examples that we might explore in depth later here on the IPRH blog, but I hope that you can see in them something praise-worthy: in all three cases there is a degree of self-consciousness with which the producers approached the question of quality review. We are now and will no doubt continue to be in a period where peer review is under public scrutiny and discussion, where its value co-exists with questions about that value. Under such conditions, I would argue that the best way of shedding light on the ways and means of peer review, and of understanding the certification that it affords, is to not let peer review be the metaphorical “man behind the curtain.” Through transparency in our own practices, we can and should encourage an open and voluble discussion of how we establish and communicate the standards by which we evaluate scholarship. And that should make us better at both doing and deploying scholarly work.

digital_transformation

Just in time to remind me that peer review is indeed under scrutiny—and as if in response to my call for discussion—at the very moment that I was drafting this post, the hashtag #sixwordpeerreview was getting very busy on twitter. The hashtag marks six-word (as you would expect) summaries/re-phrasings and parodies of reviews. The tweets have been nicely storified and depict a panoply of scholarly discontents with the nature of peer review, but also make gestures towards its potential value.

This post was reviewed and edited by an interested colleague who has worked as collection development librarian at a major research library and a director of a university press. The editor of the IPRH blog also reviewed it. The author welcomes comments from readers.

mariaMaria Bonn is a senior lecturer at the Graduate School of Library and Information Science (GSLIS), where she is developing courses on the role of libraries in scholarly communication and in publishing. Before arriving at the University of Illinois, she was a professor of English at several colleges and universities, both in the U.S. and abroad.

Advertisements
This entry was posted in Theme: Publication (2013-14), Yearly Theme and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s