Gabriel Rossman explains how to review an academic journal article

….and not be a jerk about it, making the manuscript worse. Go read it.

So many times, reviewers make papers worse. Yep. I know we’re all supposed to walk around and talk about how peer review improves the process, but I have had that happen only once or twice, and it wasn’t the time that one of my reviewers broke anonymity and discussed the process in a high profile piece in an APA platform. I understand why reviewers, like this guy, like to take credit for the contribution. Reviewing is a lot of work, but honestly, the paper did not really improve much: by “improvement” he meant (a) “after round after round of reviews, the paper focussed on an issue I wanted it to instead of what the author wanted to discuss” and (b) “I forced the author to use the method I wanted.” As it is (a) was fine–I published the other material elsewhere, but (b) sucked because I had to torture the data to use his method, and the results are much harder to interpret than if we’d just stuck with a simpler method that accomplished everything it needed to. Why run a multi-level regression to establish correlation? Gaaaah.

I gave in. Why? I needed a paper in JAPA, and it was clear that the reviewers–all of whom were obvious even before the one broke his anonymity—-were in love with their own ideas, had a stranglehold on the paper, and I was doing a hostage negotiation, not a revision.

I’m excited to see how creating a dialogue around a manuscript might work. In a discussion-based format, I probably would have had another methodologist back me up to be able to say to this reviewer: “Hey, she’s right, she’s not doing a causal analysis, she’s just looking for a correlation.” (It was an exposure study; I don’t have to prove what’s causing the exposure, only that there ARE differing levels of exposure.)