Fall 1998 Muni Metro Survey Results
by Andrew Sullivan, 28 October 1998
The performance of Muni Metro, San Francisco's streetcar system, has come under increased scrutiny this fall as the Municipal Railway has implemented major changes to its operations. In August, the railway turned on its long-delayed, $70 million Advanced Train Control System to automate the operations of streetcars in the tunnel, extended N-Judah service to CalTrain, and began proof-of-payment fare collection, all on the same day; the resulting "meltdown" caused major delays and brought Muni under an intense media spotlight. Since mid-September, however, Muni has claimed that the system is now more reliable than before ATCS was installed. Rescue Muni conducted a special Metro Survey this fall to find out if these claims are accurate.
The results of our survey were decidedly mixed. Muni Metro showed some improvement in on-time performance. The Metro system as a whole, and most streetcar lines, delayed our participants somewhat less often than in the spring. No line performed better than it did in our first survey in February 1997, however, and the grades assigned continue to describe a system that is quite unreliable. In addition, we found that riders experience unacceptable levels of crowding far too often, and that travel times are quite inconsistent, particularly in the tunnel.
Survey participants experienced delays 28% of the time, less than this spring's score of 35% but more than the 24% of riders delayed in 1997. This earned Muni Metro a score of C. Reliability was worse in rush hour, but again not as bad as it was in the spring, with 39% of PM rush and 31% of AM rush riders experiencing delays. As we have noted before, this means that riders who take Muni to work are delayed at least every other day, or every day if they transfer or use Muni for more than just commuting.
|
route |
Fall 98 % late |
Fall 98 grade |
change in |
98 % late |
97 % late |
| J | 33% | D | -10% | 42% | 22% |
| JKLMN* | 1% | A | N/A | ||
| K | 33% | D | -9% | 41% | 27% |
| KLM** | 22% | C | 6% | 14% | 7% |
| L | 47% | F | -6% | 53% | 22% |
| M | 38% | D | 6% | 31% | 30% |
| N | 35% | D | -7% | 42% | 33% |
| metro total | 28% | C | -8% | 35% | 24% |
*Riders traveling between Embarcadero and Van Ness
** Riders traveling between Embarcadero and West Portal
This survey attempts to measure Muni Metro's reliability from the rider's perspective. For two weeks in September and October, volunteers recorded how long they waited for the streetcars that they used every day, and a few watched vehicles go by and recorded the headways. 170 volunteers recorded 1,896 separate vehicles, over twice the number of Muni Metro responses in the spring and five times as many as in 1997.
For each ride, we calculated waiting time and compared it to the frequency advertised on Muni's street map posted at most stops. We calculated the percentage of riders delayed, the average waiting time, and the average normalized waiting time - waiting time over advertised frequency - for each line. For data collected by watching vehicles go by (537 observations), we used a system of weighted averages to calculate these metrics for a hypothetical rider arriving at random. We then assigned letter grades based on the percentage of riders delayed, and we compared these with the 1998 and 1997 Riders' Survey results.
To compare service in the tunnel with service on the street, we identified all rides taken just in the tunnel. Based on their origin and destination, we assigned over 800 trips to "JKLMN" (Van Ness-Embarcadero) and "KLM" (West Portal-Embarcadero) segments and compared wait times to the minimum posted frequency of the lines in question.
We also asked riders to record their destinations and the time they arrived there, and to measure maximum crowding on their ride based on a scale of 1 (empty) to 5 (crush-loaded). With the arrival data, we calculated average travel times for all trips taken, a selection of which are published here. We also compared the average travel time for all trips with the minimum recorded; the resulting ratio is a helpful if rather rough measure of the typical rider's expectation of enroute delays. The crowding data helped us identify routes and times of day that are overloaded and unreliable as a result.
In the fall of 1998, Muni Metro delayed passengers 28% of the time. Of the 1,896 vehicles tracked, 519 had waiting times longer than the frequency advertised on the system map. This represents an improvement since the spring, when 251 of 712 rides (35%) were delayed, but is worse than in 1997, when 92 of 375 (24%) were.
The survey showed a striking difference between on-time performance in the tunnel and on the streetcar lines throughout the city. While service between Van Ness and Embarcadero ran on time 99 percent of the time, earning a grade of A, the segment between West Portal and Embarcadero earned a C with 28% of riders delayed, and all above-ground lines were graded D or F. A major reason for this was out-of-order dispatching at Embarcadero, a very old problem noted by several volunteers.
| route | % late | grade | 2/98 % late | change in % late | 2/97 % late | Avg wait time | Norm wait time | % min travel time | Avg crowding *** | Total responses |
| F | 0% | * | 13% | * | 39% | 0:04 | 40% | 153% | 2.93 | 14 |
| J | 33% | D | 42% | -10% | 22% | 0:08 | 93% | 139% | 3.12 | 188 |
| JKLMN* | 1% | A | 1% | 0:03 | 20% | 207% | 2.95 | 236 | ||
| K | 33% | D | 41% | -8% | 27% | 0:12 | 84% | 125% | 3.22 | 84 |
| KLM** | 22% | C | 14% | 7% | 7% | 0:05 | 80% | 217% | 3.58 | 578 |
| L | 47% | F | 53% | -6% | 22% | 0:12 | 143% | 125% | 3.59 | 227 |
| M | 38% | D | 31% | 6% | 30% | 0:12 | 91% | 126% | 3.51 | 125 |
| N | 35% | D | 42% | -7% | 33% | 0:08 | 96% | 130% | 3.00 | 445 |
| metro total | 28% | C | 35% | -7% | 24% | 0:07 | 86% | 166% | 3.35 | 1,896 |
*Riders traveling between Embarcadero and Van Ness
** Riders traveling between Embarcadero and West Portal
*** 1=empty, 3=standing room only, 5=crush-loaded
As in the spring, the worst line in the Muni Metro system was the L-"Terrible", graded F with 47% of riders delayed and the system's worst rating for average crowding. Poor as this grade was, this did represent a small improvement (6 percentage points) from February. The most-improved line was the J-Church, graded D with riders delayed 33 percent of the time; this was 10 percentage points better than in February. Most lines on the street (J, K, L, N) did show improvements in on-time performance since the spring. The J-Church was the most improved, by 10 percentage points; other lines improved by smaller margins. the M-Ocean View was the exception, continuing a worsening trend that began in 1997.
Reliability also varied significantly by time of day. Muni Metro was much less reliable at rush hour, when it is the most crowded; 31% of morning and 39% of evening rush-hour riders experienced delays, earning those periods a score of D. These time slots were significantly improved from the spring, however; the score for PM rush in particular improved from 57% to 39% late, probably as a result of the implementation of ATCS. Half again as many riders were delayed in the evening rush as were delayed during midday periods (25% delayed, graded C) or evenings and weekends (both graded B with 19% and 18% delayed, respectively).
Participants also rated crowding on the streetcars and travel times. Crowding was particularly pronounced at rush hour and on the segments in the tunnel; in the evening rush, for example, almost half of streetcars were crush-loaded. All lines to and through West Portal suffered from extreme crowding; the K, L, M, and "KLM" segment all were rated "4" or "5" on our 5-point scale around half the time. Even on nights and weekends, the level of crowding was remarkably high, with cars crush-loaded 17 and 18 percent of the time, respectively.
In our view, this is an unacceptable level of crowding, one not experienced by commuters on any other transit system in the area. Many participants commented that they could not board one or several cars due to crowding. In many cases, the high level of crowding (which results primarily from insufficient capacity, but also from misallocation of resources) contributes to poor on-time performance. Many riders have noted to us that they have abandoned the Muni Metro because of crowding as much as late streetcars; the recent increase of Fast Pass use on BART by several thousand trips per day is surely one result of this problem.
Riders in the subway also experienced long and erratic travel times, with the average trip taking 1.6 times the shortest time recorded for the trip taken, and with average travel times in the subway (JKLMN and KLM segments) more than twice recorded minimums. Outside the tunnel, service was much more consistent, with the average trip under 1.4 times the minimum. (An ideal score would be 1.0, with all trips taking the same amount of time; this should happen in the subway.) This confirmed what we already knew anecdotally: one cannot count on Muni to get to one's destination on time, even if it arrives at the stop within the published interval.
As always, riders gave us ample comments on system reliability, operator and staff courtesy, comfort and convenience. Many riders reported trouble:
One reported a serious safety violation:
Others reported better service:
And one reported a remarkable episode of rider-operator cooperation:
This survey shows both the benefits and the limitations of Muni's Advanced Train Control System and the Muni Metro Extension and Turnback as a means of running Muni Metro on time. With ATCS, Muni is in fact running more trains per hour through the system, and they are in fact running closer to on-time than they did in the spring. But because they are still dispatched out of order and they are almost all one car trains, the level of service for individual lines has not improved much at all. Of course, service between Van Ness and Embarcadero is excellent, because for users in this area the specific line does not matter - but this is small consolation for commuters from outside the central city.
That crowding is an issue is itself not big news, but the severe level of crowding identified by our volunteers suggests that Muni's decision to take 20 cars out of service is adversely affecting service. Many riders noted severe, and in some cases unsafe, levels of crowding that make the Muni experience particularly frustrating. Crowding was most severe for riders in the central city, because cars on all lines were full by the time they got to their stop; this lack of sufficient capacity makes it very difficult to depend on Muni at rush hour.
RESCUE MUNI has proposed several "quick fixes" to Muni, that we believe would improve service in the short term. In brief, we recommend:
Muni Metro is getting better - and it isn't. Inside the tunnel, service is distinctly more reliable and frequent than it was in the spring - but crowding is such that riders often can't get on. Outside the tunnel, service is not as bad as it was in the spring, except on the M-Ocean View - but on no line is it even as "good" as it was in February 1997. Travel times can occasionally be quite fast in the tunnel - but one needs to plan on long travel times, due to the frequency of delays. Meanwhile, Muni boasts of improving trains per hour at Embarcadero while refusing to do the one thing, dispatching cars in order, that would most reduce the occurrence of delays.
And so the old questions remain. Can Muni reasonably claim to have improved service, having reduced available cars by 20 percent? Can a railway under such conflicting political and bureaucratic pressure operate effectively? Can we expect the railway to meet, under its "no-excuses" 1999 budget, real performance benchmarks based on the rider's experience? Does the Brown administration, and Emilio Cruz's leadership, have a rational plan for bringing Muni Metro service levels at least back to where they were when the Mayor took office? Will Muni remain the huge issue it is now in next fall's mayoral race? We shall see.
[ RM Home Page ]