ATTENTION CONSERVATION NOTICE: The qual versus quant distinction that old timers have grown up with is dated, and it probably wasn’t even useful back in the day. Most of us academics are dinosaurs, so be humble when you throw poop around the dinosaur cage.
This piece by Stephen Porter crossed my desk via Twitter the other day from Noah Smith (@Noahpinion, who is wonderful, and you should follow), and at the time I shot back some opinions on Twitter. But it’s bothered me ever since, so I thought I would write a fuller response here. I don’t know Stephen Porter or his work, but that said, I did read his bio and a couple of his papers after this blog post.
Let’s start with the overall snarky tone of the piece. As somebody who is frequently snarky, it raises a red flag. I know full well when I do it, and it’s not good scholarly behavior on a blog or anywhere else. When somebody is snarky about a topic that shouldn’t normally generate anger or condescension, it’s a warning sign, and the warning sign is simply that the author’s ego is at stake in the writing. If you really have the full force of both soundness and validity in your argument, you don’t need snark to bully the reader into believing you or to frighten dissenters from challenging you. I’m as guilty of this as anybody.
From that onward, the argument is cherrypicked, overreaching, and blind to the overall research context we all live in.
Let’s start here:
His response to this tweet was:
Of course, the whining and outrage was predictable. More here:
I assume BMJ is the British Journal of Medicine.
So whining and outrage go together, and reactions to a medical journal’s business model of scholarly research dissemination is mere “outrage” instead of legitimate critique of a journal that is extracting from scholars free content to sell at exorbitant prices…about inconsequential matters such as health. Okaaaay.
There actually are some pretty damn good reasons that medical research absolutely needs qualitative research, and some of the most important medical studies ever done have been qualitative. We know a lot more about the effects of toxins on the human body because of opportunistic studies of rare events like industrial accidents or London’s “killer fog.”
And, btw, what does shunning qualitative research mean for bioethics research? It should bother us when a medical journal is not interested in the casuistry of field practice. It’s one thing if BMJ intends to specialize and expects those researchers to go to specialty journals, but that’s not the same as the “it’s a low priority for us because it doesn’t sell” rationale.
The next point that strikes me as incorrect is this one:
Let’s face facts: it’s a quant world now. Policymakers and stakeholders don’t want to hear stories about the lived experience or any other such nonsense. Funders are increasingly adopting a similar mindset
The facts are, there isn’t any evidence to back up this assertion. The facts are…policymakers and stakeholders–an amorphous, ill-defined group of people, so God only knows who they are, but they of course agree with what Porter thinks…are often not terribly interested in any research of any kind unless it supports their interests.
But before I get too far into that, let’s deal with “don’t want to hear stories about the lived experience or any other such nonsense.” So if market interest is your measure of worth, then fine, but you should probably note that in the list of New York Times Best Sellers, historians outnumber economists roughly 8 to 1 (where economists would be doomed without the not-strongly quantitative Thomas Piketty), historians routinely win the National Book Award (year in and year out, actually, where economists have never posted a win, not ever*), and The Immortal Life of Henrietta Lacks–a qualitative book on bioethics and history–just landed a feckin movie deal. We should probably note that Freakonomics would not have been what it was in terms of runaway best sellers if it hadn’t had a research collaboration with a qualitative social scientist and a writing connection with a journalist. (And a platform in the NYT).
Matthew Desmond is currently tearing up the book sales with a book of “stories.”
I kind of think people are interested.
If your idea of qualitative research is that it is just about the “stories of the lived experience” and “nonsense”…then you aren’t qualified to make assessments of qualitative research because you don’t know what you are talking about. Yes, there are ethnography and interview studies still out there, and I find them often to be quite valuable (Henrietta Lacks, and many others). I’ve obviously done a fair amount of them in addition to my quantitative research because I’m not an ideologue about methods. I care about questions and getting answers.
But more than that, big data are–or should be if you are awake–entirely changing the distinction between quantitative and qualitative. With digital technologies and social media, you are getting millions of data that confound the traditional tools of econometrics. Later on, Porter says qualitative people are “dinosaurs” but with his characterization of qualitative research here, I guess I have to question about whether Porter is as cutting-edge as he thinks he is.
And I don’t know about the rest of you, but one of my econometric instructors, a brilliant econometrician named Joel Horowitz, and I once had a really interesting discussion about whether Bayesian approaches are inherently qualitative, and that wasn’t one of your typical sniffy-snooty, looking-down, pissing-on-the-wall, I’m-ever-so-much-more-rigorous-than-thou academic discussions. People like Horowitz, who are genuinely secure in their work, don’t have to do that: it was just the two of us chatting about where ideas come from and how people use them to formulate theory, and getting into some pretty interesting epistemological waters as we went.
The part here that pains me to write: research and higher education do seem to be in process of changing, but it’s not strictly a data revolution where quantoids like Porter stand astride the earth while the silly dinosaurs die. Instead, the star economy of the academy means that there are global academic darlings, who get all the sunshine, and then the rest of us–the Help–who get whatever crumbs are left.
Funding, particularly that for social science, is consolidating and drying up for just about everybody, not just those dummies who tell “stories.”
And if I were a betting woman, I’d guess that the university Porter teaches at, NC State, stands a good chance of either being the only state university in North Carolina…or being closed in the next 30 years. And since UNC at least has a sports dynasty on their side, I’d bet the latter.
I hope I am wrong. But I don’t think I am. I think some aspects of higher education are, in fact dying, and lot of what I see in Porter’s argument is the anxiety that all of us have about the changes going on around us: I’M not the dinosaur or the Help. YOU OTHERS ARE.
Then he goes on to say that quantitative dominance is only going to get worse because:
1. Statistics is now prominent in the K-12 math curriculum; it was nonexistent when I was a kid. Students at a young age will now be learning quant methods, not qual methods.
This assumes that students don’t learn qual methods, and I don’t think he’s right about that. I agree that we are seeing more statistical literacy in K-12 (and thank heaven), but we also seem to be seeing things like expanded service learning, visual ethnography in addition to data literacy.
2. The media has gotten much more data savvy, and now regularly present charts and graphs based on quant data. This is creating a culture where we tend to talk and view issues in terms of what the quant data tell us.
Yes, but the media also show us word clouds, videography…text mining appears regularly in the media, etc. It’s not like you can’t graph various aspects of qualitative research.
And, um, “This is creating a culture where we tend to talk and view issues in terms of what the quant data tell us”…go read some media effects research before you make sweeping conclusions like this based on your impression. The tail can wag the dog in terms of what media shows us.
3. Number 2 is especially true for academic research. The Chronicle of Higher Ed and Inside Higher Ed report predominantly on quant studies. The major media outlets, like the NY Times, tend to report on work done by economists. When was the last time you read about an anthropological study in the national media?
And yet Matt Desmond got a six-figure book deal just telling stories, and we didn’t, with our big, giant, better-than-his data.
4. More and different quant datasets are continually collected, as we use more electronic devices and the cost of data storage continues to drop to almost nothing. So it’s becoming much easier to study a wide variety of topics using a quant lens than it was 20 or even 10 years ago.
5. Statistical and visualization software is easier to use every year, putting more tools in the hands of people who might normally never crack open R and run a regression analysis.
I’m currently doing a project with about 5,000 images from the web. Quant? But it’s coded images and text mining. Qual?
Porter’s approach seems to be “everything that is new and emerging is quant and everything old and lousy is qual”–and it’s an easy way to frame an argument you wish to win–but that doesn’t make you right about your basic definitions. Just because you have a lot of data doesn’t mean your approach isn’t qualitative. If I measure every single thing that happens every nanosecond of an individual’s life…I might have a lot of data, but not necessarily generalizable research conclusions. And it could still be interesting and useful as all hell.
This last point, to me, just suggests that old binaries like “quant” and “qual” are going away because they aren’t useful, not that Porter is right in his characterization of them.
The rest of the essay is academic posturing: my discipline does things in a rigorous way, education doesn’t, and so forth. Everybody knows that there are good studies out there and weak studies out there, and there are plenty of examples of weak quant and weak qual studies.
His link to an editorial targeted to qualitative researchers on how to get their work published strikes me as good advice for academic writers in general, though nothing here strikes me as particularly earth-shattering for those of us who get our work published. But here it is, for those out there who can use advice.
*Leontief and Galbraith were both nominated, but didn’t win. Always a bridesmaid.