So I have been doing the scholarly gig for awhile, and I have Opinions. Here are some of my pet peeves that reviewers do, in no particular order.
There it is, inevitably, a complaint about qualitative methods since you can’t generalize, even though you have not generalized in your paper, the paper is simply not generalizable and thus should not be published. Um, yeah, we know; all of us took research design, sweetie, all of us; don’t agree to review qualitative work if you have the belief that the only work that should be published is generalizable in the social science sense of the word. You already know you are going to recommend rejecting the paper based on method alone, and anybody who agrees to review a paper just so they can have the petty joy of hitting “reject” is a dick.
Let somebody who believes the methods well enough to distinguish when they have been done properly. Otherwise you have appointed yourself the epistemology sheriff in town and nobody who isn’t a philosopher needs an epistemology sheriff in their town.
Sigh; go write your own papers that answer these questions then and bloody tell me whether I’ve answered the question that I actually posed in the damn paper well enough to merit a contribution.
Oh, gosh, you mean the paper didn’t solve the dispute between Rawls and Harsayani and the dispute between neo-marxists and postmodernists and didn’t resolve the man-nature dichotomy? Jeez, where did this author get a degree? The correspondence school PhD from the University of Rancho Cucamonga?
That last one really gets me. As soon I read that hackneyed phrase “there is a rich literature”…I’m thinking, “would you shut the hell up?”
Reviewers have no evidence on what an author knows and what they don’t know based on the 7000 words in any given manuscript. Maybe I do know that literature and I just didn’t think it was germane. Or I just couldn’t wedge it into the manuscript. At some point, the onus is, in fact, on a reviewer to explain why “this rich literature” is germane to the point, or else you look like a
a) somebody who is playing a childish “I gotcha” game (“I know something you don’t know neeeeener neeeeeener neeener.” OR
b) somebody angling to get yourself cited.
Really, I got no problems with (b). We all the know the score, and while it is unseemly to use your power as a reviewer for your own agenda, I’m willing to grant that it’s nowhere near Mussolini’s level when it comes to abuse of power. We will all live with these petty grabs, as long as reviewers have enough decency to know the limits. (Don’t reject a paper because it failed to cite you; I have seen reviewers do this, and, um, no.)
If you are going to try to squeeze a citation for yourself out of it…then damn it, just cut to the chase:
“The framing here suffers because it needs to draw somewhat from the extant research on X. In that literature, they have found some important things about Q and R that strike me as germane to the question about yada. Some key explorations in that literature are One Cite, Two Cite, Three Cite, My Cite. Those might help fix this problem.”
The freaking end. Spare us all the “I know a literature youuuuuuuu doooooon’t knooooooooow.”
Ok, I get that I am snarky in print (I am also snarky in person), but honestly, by the time you read my letter, I’ve edited out all I really wanted to say and just the left the “I’m right, you’re wrong” part. Don’t go blathering on for pages about how something is wrong, with italics and various other emphatic phrasing, if you aren’t going to expect an author to come back at you with a similarly strong opinion. If you feel strongly about a point, say so and say why and move on–and expect that authors are going to feel strongly about some points, too.
On one of my last papers, I had to deal with a reviewer who demanded we add another case study, and I said I wasn’t willing to do that because it diluted the point of what was, on its face, an interesting single case. I did point out that I understood the social science problems, and that it is perfectly valid to decide in a specific journal that they aren’t interested in historical methods in general, and if that really is an editorial choice for this journal, then that’s certainly understandable and we could take the paper to a history journal.
This reviewer comes back, complains that I am being mean, and whines that he or she “spends a lot of time and thought in reviewing” and by the time I am reading that sentence, I’m ready to barf.
Would you look out for your own ego? You didn’t seem to be terribly worried about mine when you went on lecturing me for paragraphs about the purpose of multiple case study design, like I’m undergraduate who didn’t read the Yin book.
Really? You spend time and thought reviewing? Ya think you are the only sailor on the Pequod here? I review on average about 30 papers a year, and have done so for about five years, and that’s not counting the papers I do for lots of PhD students and young scholars personally. I have been a reviewer on several of the last “best paper” award winners, and I review just about every book proposal that comes my way. Ok? You know what that and $1.75 gets us? A ride on the bus. It’s my job. I try to do a good job, and I’m glad that this reviewer, too, tries to do a good job, but please stop making a tussle about who is right or wrong about something–particularly something that may come down to taste and editorial preferences–about you.
For one thing, I try to say to authors “hey, I think this here might be a big deal” versus “this here is a small deal.” If an author doesn’t take your suggestions, it’s because they think you are wrong. That’s ok. People are wrong sometimes. Sometimes, people are wrong about other people being wrong. It happens, too.
There are two sides to this. If an author has blown off something I consider to be a big deal, I want to know the reason. Maybe it’s a good reason I didn’t think of.
And some writers really are wasting your time as a reviewer: they don’t care what you think and they are too lazy to revise. Now, there are many, many complaints that people can make about me, but one is not that I am insufficiently responsive to reviwers. I do a lot to deal with their suggestions, and if somebody has got an important point, I will turn myself inside out to deal with it.
But reviewers are not customers at a Burger King where they have a right to have it their way just because they were kind enough to serve as reviewers. There has to be a give-and-take, and while I want my time and energy respected as a reviewer, authors deserve to retain authorial control over the manuscript’s scope, voice, and vision. If they give me a good reason for why they are right and I am not, then that’s the end of it. I might have additional concerns, but those are a separate conversation. It’s not my paper, it’s theirs.
If I have authors who do not take my suggestions seriously, I make a decision based on what is there. If I really do think the manuscript should be rejected based on the problem I pointed out, I do that. If these are just matters of improvement, but the paper is acceptable, I say that they should improve the manuscript but I’m out as a reviewer and the paper is basically acceptable if less than what it could be with more revision. Nobody needs a reviewer they aren’t listening to, and I got other stuff to do.
Once I had a problem with a multi-case study manuscript where I objected to how the cases were presented. I thought the authors were getting in their own way, and it wasn’t clear that they had a control. I suggested changing the way the cases were presented, and/or adding a case if they didn’t actually have a control.
This struck me at the time as a pretty big deal.The paper just read like a meandering mess to me, and other reviewers seemed to me to be reacting to the problems in organization. Organizing multiple cases is hard. The journal editor strongly emphasized another reviewer’s comments, instead of mine, and I shrugged and went on with my life, even though I thought the editor was making a mistake.
The author’s came back to me with the response “It’s always possible to suggest another case, but doing so adds nothing new, so we did not take this suggestion.”
Ok, but the suggestion had also included the possibility of reorganizing and clarifying if they already had a sense of what the case controls were. I read the paper and had the exact same problem, so I wrote a short second round review and said, “hey, this paper’s structure and organization is not helping it. I strongly advise re-outlining it for clarity…” and then I made a couple of suggestions.
The journal editor again did not emphasize my suggestion, so it came back from the second revision with the same smelly pile of organizational problems it had all along, and I refused to review it. I told the editor that I’d told both him and the authors twice before what the problem was, they didn’t listen to me, and now he’d put the authors through two rounds of revisions, and my only suggestion was to rewrite the entire first half. He had two other reviewers who hated the paper by now and wanted to reject it, and the editor actually called me on the phone and said he needed me to step in because he was neither a method nor content expert. I said that I didn’t have any other critiques besides the one I’d already made, to no avail, twice before.
This struck me–and the editor agreed–as a mishap in editing. Perhaps because I wasn’t emphatic enough in my original review, both he and the authors missed that my point was a pretty important one. They had a paper they had revised multiple times, and the only thing I could add at this point was a suggestion that could potentially lead to a great deal of revision–restructuring work is sometimes not at all straightforward–and that I also couldn’t recommend publishing unless that work were undertaken, and that we were not here because I hadn’t pointed it out before, but because the editor hadn’t noticed that I was saying that something that was a basic problem for the manuscript. This was a great editor, too. He just didn’t see it. Maybe I should have yelled and screamed as the paper was heading for the cliff, but I think pointing out the cliff multiple times is sufficient to the point.
The editor accepted the paper, warts and all, which is fine. I personally wouldn’t have, but I am rather famous for rewriting and overwriting and perfectionism and maybe I was wrong. Maybe the authors and other people see the connections in the structure that I never did.