Over at Five Thirty Eight, Nate Silver takes a swing at the Brookings study that I highlighted the other day. I wrote a rather lengthy response because Silver is just so off the mark that I responded (late at night, when I was tired, so I sound like an idiot) and I’ll respond here:
Given how little Silver knows about transit and transportation, the declaration that the Brookings study “”asks the wrong question” is way off the mark. There are a lot of mode choice studies in the world that have already asked the questions that Silver seems to think are the right and important ones. Just because he hasn’t read these studies doesn’t mean these questions haven’t already been asked and answered–a lot. Nor does it mean that Brookings was wrong for not replicating the hundreds of mode choice studies already out there. So the question “do people have a choice?” Is VERY well mined in the research. The subsidiary question “What are the characteristics of those choices” is also thoroughly mined.
And all of these studies find pretty much what Silver is banging on about–that NYC and these other regional big systems come out well in terms of individual mode choice. It’s just that Brookings is also right–the world doesn’t need another study that finds that lots of people in New York take public transit. We already know that.
Instead, what Brookings is trying to get at is the geographic supply of jobs and transit. They are looking for supply CONSTRAINTS, NOT demand. And that’s worth looking at. The question they ask is:assuming you have residential access to transit (a subset of the population), and you’d like to take it, could you feasibly get to a job? That’s a good question because it hasn’t been answered as much, and the answers give us some clues as to why transit is often less popular as a commute mode than it might be.
Nowhere in their report does it say that Modesto transit is “better” than NYC. Instead, the interpretation is, simply, that a higher percentage of people who have transit can get to a higher percentage of area jobs. With a measure of jobs and transit coverage like this, geographically small regions are BOUND to rank better than larger ones, which is unfortunate, but hardly impossible to interpret within the context of existing transit research.
God, people. Does EVERY study about transit have to WET ITS PANTS about NYC/SF/Boston to tell us a piece of the puzzle about transit service quality? How’s about if everybody who studies transit starts off each manuscript with:
New York City is the bestest of the bestest. Except for Tokyo, which kicks its ass. But New York is really best. Best best best. Nothing better in the whole land. Best, I tell you, BEST!
Now may I ask a question about public transit where the answer isn’t New York? Because there are other places in the world that aren’t New York, and transit is meant to work there, too.
Two things:
1. When you create a list in your report that ranks the cities you are implying that one place is better at something than another place. If you want to avoid those comparisons don’t include a ranked list and a table of the top and bottom 10 in your ranking.
2. You wrote:
“Nowhere in their report does it say that Modesto transit is “better” than NYC. Instead, the interpretation is, simply, that a higher percentage of people who have transit can get to a higher percentage of area jobs.”
Actually, it doesn’t measure that. It measures the average percent of metro area jobs that are accessible in 90 minutes. In Modesto the average block group resident can get to 38% of metro area jobs in 90 minutes on transit. What about the other 62%? Are they accessible from somewhere on transit? Or are they not accessible by transit at all? That’s a really important distinction when you are trying to compare different sized cities and it distorts the data when they limit themselves to the 90 minute radius.
Rank measures are reductive just like every other measure, and people should know that by now. How many worthless “Top fat American cities” or “Top Cities for Singles” lists do we have to sit through for people to stop acting like rankings matter outside of rank-sum tests?
So they publish their rankings–and the report I read for them had multiple sets of rankings– It doesn’t mean the rest of the world has to act like a bunch of wronged virgins because somebody dared suggest their favvy city didn’t come out on top. The cycle for this study was soooo typical: the study is released, the ONLY thing people bothered to read were the rankings, they got hysterical about the rankings, and now people are complaining that the rankings were “mere link bait.” So who did that? Brookings or all the people who reacted to the study in high dudgeon without reading anything other than the one set of rankings?
2. In the report I reviewed, there was more than one measure ranked. I could be wrong, so thanks for the correction if so, but I am pretty sure I raised my objection (shared in the post). Perhaps it isn’t–I review at lot of these studies, so it might have been a manuscript. However, I am pretty sure that there was more than one measure in that report, but I don’t have the time to run check now.
But are you suggesting that the 90 minute commute was too restrictive? In my review, I raised the opposite given average commutes? When I reviewed the study, I told them that 90 minutes was too generous based on average commutes in the US; they argue, I think convincingly, that it’s not too generous based on average transit commutes. I believe in the appendix they do relax the 90 minute constraint.
The problem is that by restricting it to a 90 minute radius you make large MSAs look terrible in terms of “job access,” even though all their jobs may be accessible by transit.
If you have one small MSA that takes 90 minutes to get across and every job in that MSA has transit access outside it, you now have 100% of the jobs as “accessible” under this metric. Now take an MSA with the same exact population density, the same awesome comprehensive transit access, and make it 3X as large. Now something like 50% of the jobs are “accessible” under this metric. Is that a fair or even useful comparison?
The point I’m making isn’t that the 90 minutes is not generous enough or too generous, it’s that it’s a useless restriction that limits the definition of accessibility. By limiting the radius of jobs you are including to those within 90 minutes, but then comparing that number as a percent of jobs in the entire MSA, you make small areas look better than big areas with better access.
Again I’ll point to the following example:
Modesto job access number: 38%
The other 62% of jobs, are they not accessible because they are too far from transit, or because they take more than 90 minutes to get to on transit? Probably the former.
New York job access number: 37%
The other 63% of jobs, are they not accessible because they are too far from transit, or because they take more than 90 minutes to get to on transit? Probably the latter.
You know that the job access numbers are an average of the % of metro area jobs accessible in 90 minutes from each block group, right? It’s not a measure of “SOMEONE can get to this job in 90 minutes on transit.” It’s a measure of “pick a point and someone can get to THIS PERCENT of jobs in 90 minutes.”
As soon as you use percentages, you take scale out entirely, regardless of what time measure you use. So 100 percent of the population in Lamont, IA has walking access everywhere, and 0 percent have transit access because there is no transit supplied. (Well, there is a county paratransit for the elderly, but you know what I mean.)
There are some advantages to taking scale out of the question, with abundant caveats for your reasoning. Think about all the census-based mode choice studies that have talked about Americans’ use of transit, and how heavily the sheer scale of NYCs’ raw numbers skew those types of analyses so that you are really talking about NYC rather than US.
You write: “The other 63% of jobs, are they not accessible because they are too far from transit, or because they take more than 90 minutes to get to on transit? Probably the latter.”
How does this distinction matter in your calculation? In my view, the distinction concerns land use versus transit service quality. If you live six miles away but the bus comes only once every two hours, that’s my fault as an operator. If you live 20 miles away from the job, then the question is land use, urban form. Is that what you are getting at?
And–I think one of the reasons we are talking past each other is that I really, seriously do NOT care about fair or useful comparisons here. I care about transit service. It’s entirely possible that there are some things that NYC transit could improve, objectively, even if it is #1 according to everybody who lives there or wishes to live there. The Brooking study gives me another piece of information about how transit operates (or doesn’t) with regard to ONE part of transit (we’re not even talking about the important role transit plays for a whole bunch of nonwork related travel.)
Yes, you’re right–the study has a boundary problem. But all are going to, since administrative boundaries aren’t functional anyway.
The other reason we may be talking past each other–I don’t expect one study to have a full measure of something like job access. I clearly found the study both useful and interesting. But it’s partial. Flawed. Like every study–they look at a few things, but now that they’ve kicked up some dirt. Far too many studies of access only look at the residential side. It was refreshing break to look at it from the job side. If people now want to get all up arms and study transit job access better than done here–then the study did something important–it got people working in a different area. (Whenever people start complaining, for example, about Richard Florida, I always respond with “Two of my favorite studies of all time are *responses* to Richard Florida”. ) Starting up a conversation, however imperfectly, has value in policy.
The distinction between the two matters because in one case (job has no transit available at all) NOBODY can commute to that job on transit, and in the other case (job too far away for some people) it’s just being left out of an average.
Again, the job number isn’t comparable to the coverage number. The job number doesn’t say “here’s what percentage of jobs in the metro area have transit access.” It cuts the number down with the 90 minute radius and then averages things out, making the number smaller. In a certain metro area maybe one person can only get to 30% of the jobs on transit, but someone living on the other side of the MSA can get to another 30%, and someone living in the middle can get to the other 40%. But when you average them out, then it looks like you can only get to 33% of the jobs in the area on transit. And it makes the transit job in that place look just as useful as the transit in a place where it only takes 90 minutes to traverse the whole MSA, but only 33% of the jobs have a transit stop near them at all. There’s no distinction made between the two. That’s a big problem.
You’re right, but I still don’t know how you fix that problem in analysis without travel diary data for individuals, which the Brookings people don’t have for the US. The new NHTS is out there–a place to start.
Instead of analyzing the percent of metro area jobs an average person can get to in 90 minutes they should have done the reverse and just measured what percent of the metro area jobs have transit access period.
Or they should have looked at a total of what percentage of metro-area jobs can SOMEONE get to in 90 (or 60 or 45) minutes on transit.
As long as we are Monday-morning quarterbacking:
” just measured what percent of the metro area jobs have transit access period.”
I’m not on board with that one, depending on what you mean by transit access, except as an additional measure. So there are bus stop all over Orange County. But they have hour-long frequencies in some locations. Accessible? I guess. Maybe an index measure.
“Or they should have looked at a total of what percentage of metro-area jobs can SOMEONE get to in 90 (or 60 or 45) minutes on transit.”
That’s actually a lot more complicated than it sounds, and I think it’s less enlightening than it sounds. So what if SOMEONE can get there? That doesn’t really make it more informative than the average–still doesn’t mean that there is a good match between housing and job supply and transit supply. The average may be too generous or restrictive, but opening it across the entire distribution doesn’t help us get closer to the answer unless we know the distribution.
I think if you wanted to do gold standard, you’d want either diary data with matching job/residential location matched with transit provider, or you’d want some matched measure of job concentration/dispersal, geographic coverage, and frequency.
And I’m going to go and pretend I have a life now, but many thanks for contributing, MD–