PHASE I REPORT Submitted to Valley Metro / Regional Public Transportation Authority Efficiency and Effectiveness Study Project No. 064008S In association with Texas Transportation Institute (TTI) HDR | S.R. Beard & Associates March 15, 2007 EXECUTIVE SUMMARY This report documents results and recommendations from Phase I of the Valley Metro/Regional Public Transportation Authority Efficiency and Effectiveness Study. The study’s purpose was to develop a performance reporting framework in support of Proposition 400. The Arizona statutes concerning the Proposition 400 law call for performance audits starting in 2010. While the framework can (and should) be applied to all transit routes in the region, the audits only concern those routes funded by Proposition 400. The framework developed addresses four “modes” (Fixed Route Systemwide, Fixed Route, Route level, Paratransit, and Rail). Bus service categories include Local, Supergrid, Express/BRT, Rural, Paratransit, and Circulators. The framework assumes that the measurement first occurs at the contractor level, then is rolled up to higher levels of aggregration. Framework Exhibit ES 1 Fixed Route, Systemwide Measures Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings Complaints per "x" Boardings On-time Performance Miles between Mechanical Failures Customer Satisfaction Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report Exhibit ES 2 Fixed Route, Route Level Measures Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Boardings per Revenue Hours (Express Bus) On-time Performance Miles between Mechanical Failures i Exhibit ES 3 Paratransit Measures Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Operating Cost per Revenue Hour Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Hour Percent No Shows On-time Performance Miles between Mechanical Failures Customer Satisfaction Exhibit ES 4 Rail Measures Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings On-time Performance Miles between Failures Customer Satisfaction With exception of the rail system, which is not yet open to revenue service and which therefore has no historical data, all bus and paratransit measures above have been tested by the RPTA and member cities. Valley Metro/RPTA is also in the process of developing specific performance goals across each indicator. Targets Significantly, the framework also proposes draft performance targets, which establish a baseline of performance expectation for Fixed Route bus (systemwide); Fixed Route bus at the route level; Paratransit; and Rail. Establishing a baseline, high level performance level for transit is also an important part of fully regionalizing transit in the Valley to ensure that every citizen, no matter where he or she lives, has the same high level service. Proposed performance targets are presented on the next page. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report ii Exhibit ES 5 Fixed Route, Systemwide Targets1 FIXED ROUTE BUS, SYSTEMWIDE Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings Complaints per "x" Boardings On-time Performance Miles between Mechanical Failures Customer Satisfaction Exhibit ES 6 Fixed Route, Route Level Targets FIXED ROUTE BUS, ROUTE LEVEL TARGET 25% $2.32 $1.75 $4.96 $0.67 3%* 3%* 2.1 1.2 0 28 90% 23,400 89% Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Boardings per Revenue Hour (Express Bus) On-time Performance Miles between Mechanical Failures Exhibit ES 7 Paratransit Targets PARATRANSIT Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Operating Cost per Revenue Hour Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Hour Percent No Shows On-time Performance Miles between Mechanical Failures Customer Satisfaction TARGET 25% $2.32 $1.75 $4.96 3%* 3%* 2.1 TBD 90% 23,400 Exhibit ES 8 Rail Targets RAIL TARGET Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding 5% $28.55 $27.16 $50.30 TBD Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday Boardings Avg. Sat. Boardings Avg. Weekday Sun./Holiday Boardings per Vehicle Revenue Mile Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings On-time Performance Miles between Failures Customer Satisfaction 3%* 3%* 1.76 5% 90% TBD 90% TARGET 25% $2.64 $1.98 $26.26 $0.67 10,655,000 26,090 N/A N/A 3.94 8.04 N/A N/A 95% 25,000 89% Note that targets for Rail are preliminary. There is very little data available on which to base the targets until the system has gone through some testing and begins revenue service. Assumptions include the 2010 Financial Plan and operating assumptions 1 * Items: Financial Plan assumptions; subject to service level increases. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report iii reflected in the New Starts criteria. Proposition 400 audit requirements do not supersede nor replace the New Starts planning process. Action Summary and Timeframe This Phase I report provides a roadmap for Valley Metro/RPTA and member agencies including detailed recommendations for: • • • • • • • Definitions by mode New Route implementation guidelines Route maturation guidelines Data tool implementation Reporting timeframe Reporting procedures (e.g., reporting by audience) Roles and responsibilities (e.g., data collection, quality control). Implementation of the new performance measurement framework will require hard work and commitment on the part of RPTA staff and member agencies. To fully realize the potential of the initiative, changes to operating practices are needed. Examples include full accounting in the calculation of operating costs and changes in contractor reporting practices (e.g., route level costs). A summary of the major recommendations is provided in ES 9 below. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report iv Exhibit ES 9 Summary Implementation Plan CATEGORY RECOMMENDATION ACTION OWNER TIMEFRAME Framework Adoption Adopt performance measures framework RPTA and VMR Boards April-June 2007 Target Setting and Adoption Discuss target goals for each performance measure Discuss target goals for each performance measure Adopt draft targets for framework TMC, VMOCC RPTA and VMR Boards RPTA and VMR Boards April-July 2007 February-June 2007 June-August 2007 Guidelines Adoption Discuss route implementation and maturation guidelines Discuss route implementation and maturation guidelines Identify "lifeline" network Adopt route maturation guidelines TMC, VMOCC RPTA Planning Dept and Board RPTA Planning Dept, Members RPTA Planning Dept and Board April-June 2007 April-June 2007 April-Sept 2007 June-July 2007 Information Issue performance guidelines to all reporting agencies RPTA Planning Dept April-May 2007 Implement Data Tool Develop final reporting format Complete transition from PMAS Develop reporting platform Work with individual members to facilitate process RPTA Planning Dept RPTA Planning Dept RPTA Planning Dept RPTA Planning Dept April-Sept 2007 April-Sept 2007 April-Dec 2007 April-Dec 2007 Contract Negotiations Include all measures in reporting requirement (system wide and route level) Tempe, Phoenix, Glendale RPTA Operations April 2007 - Dec 2008 Preparation for New Routes Conduct density scale analysis for new route implementation RPTA Planning Dept Ongoing New Route Monitoring Monitor new routes according to route maturation guidelines RPTA Planning Dept Ongoing Reporting Begin monthly reporting with new system Begin quarterly, etc briefing to Audiences - Bus/DAR Begin quarterly, etc briefing to Audiences - Rail RPTA Planning Dept RPTA Operations Mgr VMR Operations Mgr July 2007 Oct 2007 (for July 1 - Sept 30) First quarter after startup Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report v TABLE OF CONTENTS 1. Project Scope 1.1 Proposition 400 Context 1.2 Efficiency/Effectiveness Study Scope 1 1 2 2. Work Approach 2.1 General Approach 2.2 Legislative Summary 2.3 Outreach 4 4 5 8 3. Framework Description 3.1 Modes and Service Categories 3.2 Performance Measures by Mode 3.3 Definitions by Mode 3.4 New Route Implementation Guidelines 3.4 Route Maturation Guidelines 9 9 11 12 22 23 4. Action Plan 4.1 Setting Performance Targets 4.2 Implement Data Tool 4.3 Reporting Timeframe and Procedures 4.4 Roles and Responsibilities 4.5 Summary of Actions and Timeframe 25 25 27 28 31 32 5. Appendices A. Technical Advisory Committee Composition B. Template to Calculate Indirect Operations Costs C. Technical Advisory Committee Presentations 1. TAC Presentation #1 (April 27, 2006) 2. TAC Presentation #2 (May 24, 2006) 3. TAC Presentation #3 (June 28, 2006) 4. TAC Presentation #3 (August 23, 2006) 5. TAC Presentation #3 (September 27, 2006) 6. TAC Presentation #3 (October 24, 2006) 7. TAC Presentation #3 (November 29, 2006) Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report A-1 B-1 C-1 1. PROJECT SCOPE 1.1 Proposition 400 Context In November 2004, Maricopa County voters approved Proposition 400. Largely focused on executing the road and transit elements of the 20-Year Regional Transportation Plan (RTP), Proposition 400 effectively boosts regional investment in transportation infrastructure and services in the County through continuation of the 1985 1/2 cent sales tax through 2025. Proposition 400 also expanded the share of the ½ cent sales tax revenue dedicated to transit. RPTA’s mission of developing and delivering an integrated regional transit system reaffirms the 1985 core mandate for the agency and its responsibilities for the transit elements of the RTP. The regional public transit projects in the RTP include: • Regional funding support for existing transit services • Improvements to service levels for local transit routes that operate on the arterial street grid system and serve a regional function including 32 new routes • Improvements to service levels for regional express and bus rapid transit (BRT) services including 33 new routes • Rural transit service • Regional funding for complementary paratransit service to meet requirements of the Americans with Disabilities Act (ADA) • Expanded vanpool program • Purchase of almost 4,900 buses and vanpools, including 1,135 vehicles for expansion and more than 3,750 vehicles for replacement • Vehicle management systems (VMS) for transit vehicles • 13 park-and-ride facilities • 10 new transit centers and improvements to three existing transit centers • Regional funding for passenger amenities for bus stops • Eight new operations and maintenance facilities (including four facilities for fixed route bus, two facilities for paratransit, one for rural transit, and one for vanpool) and renovation of two existing facilities. Arterial improvements for BRT and right-of-way for dedicated BRT • 27.7 miles of new light rail extensions The Proposition 400 law – which is spelled out in several Arizona Revised Statutes (ARS) and which directly addresses the RTP itself - stipulates performance monitoring requirements, including performance audits by the auditor general on five-year cycles. The first of these performance audits will occur in 2010. While the performance criteria for light rail transit investments are specifically identified in ARS 28-6313, and under the Federal New Starts Program, criteria for the bus program were not defined to this degree. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 1 Early in 2005, Valley Metro/RPTA conceived of an Efficiency and Effectiveness Study to develop quantifiable performance measures that could be used to assess the performance of new transit service as it is brought on line. This would serve two complementary purposes. First, it would serve the intent of the legislation and prepare the transit agencies for the upcoming audits. Second, the study would help integrate public transportation performance monitoring across the Valley. Integration of public transportation services is a key objective of Valley Metro/RPTA; this is one of the agency wide goals recently developed by the Board of Directors as part of the Valley Metro/RPTA Strategic Plan. Establishment of consistent performance targets across transit operations is an important element of moving to fully regionalize the provision of transit services valley-wide. 1.2 Efficiency / Effectiveness Study Scope As Valley Metro/RPTA refined the scope of the Efficiency and Effectiveness Study, it invited input from its members, including through the Transit Management Committee (TMC) and the Intergovernmental/Transit Manager as well as its Board meetings. New ideas added to the scope of services included elements relating to an industry review of the most applicable measures, analysis of legal requirements, comparison to peer systems, and testing the evolving framework. Given the concern that new routes do not always perform optimally when first introduced, members requested that recommendations regarding route maturation guidelines be included in the study. The Service Efficiency / Effectiveness Study objective was finalized to develop quantifiable measures to evaluate bus and rail transit service performance, at both the route and system-wide levels. Such indicators can be used to evaluate route-level service efficiency and effectiveness, both relative to a standard and to rank routes relative to each other. The scope of work included the following tasks: • • • • • • • • Refine scope of services Develop public and agency involvement plan Review prior and ongoing studies Evaluate current performance monitoring and evaluation criteria utilized by RPTA and its member agencies Conduct peer city review Develop performance measure recommendations Develop implementation strategies and action plan Conduct 18-month evaluation. The project includes two phases: Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 2 • • Phase I – the baseline phase, included the first seven tasks identified above, from March 2006 to January 2007. Phase II – the 18-month evaluation task that serves as a status check on the performance monitoring effort. This report summarizes Phase I - including approach, results and recommendations from setting up the performance measurement framework. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 3 2. WORK APPROACH 2.1 General Approach The project team devised its own series of project tasks, closely following RPTA’s original program. The tasks were parallel to the extent possible and leverage input from the Technical Advisory Committee (TAC) and stakeholder involvement. The resulting task flow is illustrated in Exhibit 1. Exhibit 1 Efficiency and Effectiveness Study Task Flow 1 Project Management 3 1 Kick-off 4 5 Previous Performance Measures / Studies Existing Criteria Define & Test Performance Measures 6 Recommended Measures 7 Implementation Strategies 8 18 Month Evaluation Peer Measures 2 Stakeholder Involvement Stakeholder Input Following the project kick off, the consultant team engaged three tasks in parallel: review of previous performance measures and studies; review of existing criteria; and review of peer measurement systems. The tasks were intended to build on the current RPTA Performance Measurement Analysis System (PMAS), while learning from best practices in place with peer systems. These three tasks took place over a three month period (March-July 2006), with three presentations to the TAC. Testing the performance measures, although not identified in the Exhibit 1 diagram, is key to ascertain whether the measures can readily be calculated and what potential data collection challenges may exist. The Booz Allen team requested that RPTA test the measures, and requested volunteers from the membership. The cities of Glendale, Phoenix and Tempe participated. The last Phase I steps consisted in presenting the draft list of recommended performance measures to a variety of stakeholders, and in developing implementation strategies. For example, Booz Allen staff members collected peer best practice information regarding route maturation guidelines, and facilitated a discussion with the TAC to determine a recommended strategy for Valley Metro/RPTA. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 4 2.2 Legislative Summary As identified earlier, Proposition 400 is the means by which voters approved and funded the Regional Transportation Plan (RTP). The RTP identifies specific transportation improvements and allocates funding from FY2005 through FY2026. In passing Proposition 400, the voters approved a ½ cent sales tax to create the Public Transportation Fund (PTF). Proposition 400 performance monitoring requirements are rooted primarily in State Statutes – namely in Title 28 (Transportation) and Title 48 (Special Taxing Districts) of the Arizona Revised Statutes. The ARS provide the main legal basis for agency responsibilities, and also establish the audit requirements. In addition, the project team examined other regional transportation processes, including Valley Metro/RPTA’s Transit Life Cycle Program (TLCP), and Federal Transit Administration criteria with respect to light rail systems (i.e., New Starts criteria and United States Code). RTP Implementation Plan implementation is divided into four phases (FY2005-FY2010, FY2011-FY2015, FY2016-FY2020, FY2021-FY2026) taking into consideration: ƒ ƒ ƒ ƒ ƒ ƒ ƒ ƒ Traffic demand and congestion System continuity, connectivity and efficiency Revenue availability Bonding capacity and strategies Cost (and cash flow requirements) Project development process Project readiness Concurrent progress on multiple projects. Phasing and bus transit projects and facilities costs are discussed in the RTP-Transit Program Reference Manual. Cost of service improvements by jurisdiction/phase and annual estimated regional funding are also provided in the RTP-Transit Program Reference Manual. Most Applicable Arizona Revised Statutes For the purposes of the Efficiency and Effectiveness study, the following statutes are identified as being the most relevant. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 5 Bus Rail Type of Performance ARS 28-505 ARS 28-6354 ARS 28-505 ARS 28-6313 ARS 28-6354 Oversight ARS 28-6356F (CTOC) ARS 48-5121 (Board) ARS 28-6356F (CTOC) Audit ARS 28-6313 ARS 28-6356F ARS 28-6313 USC 5309 (e)(1)(B) ARS 28-6356F Performance Criteria in the Statutes There are no specific performance measurement criteria identified in the legislation. However, ARS 28-505 identifies ten transportation system performance “factors” that need to be addressed through a Board presentation, as shown in Exhibit 2. Exhibit 2 ARS 28-505 Transportation System Performance Factors 1. System preservation. 2. Congestion relief. 3. Accessibility. 4. Integration and connectivity with other modes. 5. Economic benefits. 6. Safety. 7. Air quality and other environmental impacts. 8. Cost-effectiveness of a project or service. 9. Operational efficiency. 10. Project readiness. The performance factors apply to bus and light rail performance. ARS 28-6354 describes annual reporting requirements, including criteria to establish priority corridors and corridor segments. For light rail performance, ARS 28-6313 further stipulates consideration of Federal Transit Administration sections 49 USC 5309(e)(1)(B). For light rail systems, ARS 286313 also stipulates that the audits will consider service levels, capital costs, operations and maintenance costs, transit ridership, and farebox revenues. These, in turn, become system performance elements that need to be monitored over time. Oversight Responsibilities With respect to the transportation system performance only (not funding), the most directly applicable statute is ARS 28-6356F. It address the oversight role of the Citizens Transportation Oversight Committee (CTOC). According to statutes, the CTOC reviews and advises the board, the governor, the director, the governing body of the regional planning agency, and the board of directors Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 6 of the regional public transportation authority on matters relating to projects funded pursuant to section 42-6104 in the regional transportation plan. CTOC should also review and comment on the criteria developed by the regional planning agency. Clearly, the Board provides continuing oversight responsibilities over all Valley Metro/RPTA activities, particularly the high visibility reporting requirements regarding Proposition 400. ARS 48-5121 summarizes this responsibility. Audit Requirements ARS 28-6313 specifies five-year audit requirements in detail: A. Beginning in 2010 and every fifth year thereafter, the auditor general shall contract with a nationally recognized independent auditor with expertise in evaluating multimodal transportation systems and in regional transportation planning to conduct a performance audit, as defined in section 41-1278, of the regional transportation plan and projects scheduled for funding during the next five years. B. With respect to light rail systems, the audit shall consider the criteria used by the federal transit administration pursuant to 49 United State Code section 5309(e)(1)(B) and the interrelationship among the criteria to provide federal funding for light rail systems. For light rail systems, the audit shall also consider: 1. Service levels 2. Capital costs 3. Operation and maintenance costs 4. Transit ridership 5. Farebox revenues. C. The audit shall: 1. Examine the regional transportation plan and projects scheduled for funding within each transportation mode based on the performance factors established in section 28-505, subsection A, in the context of the transportation system. 2. Review past expenditures of the regional transportation plan and examine the performance of the system in relieving congestion and improving mobility. 3. Make recommendations regarding whether further implementation of a project or transportation system is warranted, warranted with modifications or not warranted. The Efficiency and Effectiveness Study provides a good opportunity to prepare the agency for the upcoming audits by ensuring that the transportation system performance factors and criteria are addressed in RPTA’s performance monitoring framework. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 7 2.3 Outreach It is important to note that the outreach for the Efficiency and Effectiveness Study began long before the project even started. The scope development and RFP development process involved all member agencies at the Transit Management, Intergovernmental, and Board levels. Consultant selection was determined by a large cross agency selection panel representing diverse interests and levels within the agencies. This depth of outreach continued into the Efficiency and Effectiveness Study itself, initially through the project Technical Advisory Committee (TAC) and by working oneon-one with individual member agencies. Due to the project’s highly technical nature, the TAC was comprised of transit planners and data analysts from each member agency. Membership included technical staff with monthly and annual transit performance reporting experience, in-depth knowledge of transit operations, transit data sources, awareness of potential data issues (e.g., weaknesses in contractor reporting statistics) and experience analyzing key performance indicators. RPTA planning staff worked with each member agency to identify the appropriate candidate staff, who could attend regular TAC meetings and would be active in helping the RPTA develop a regional measurement framework. RPTA decided to augment the committee by inviting the Arizona Department of Transportation, Maricopa Association of Governments, and Valley Metro Rail to join the TAC. About five months after the project started and initial results became available, RPTA enlarged the outreach to include regular presentations to the TMC, the Intergovs/Planning Managers, Citizens Transportation Oversight Committee (CTOC) – identified in the Statutes, and the Valley Metro Board. The project team continued this outreach through the end of Phase I. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 8 3. FRAMEWORK DESCRIPTION A set of performance measures provide the basis for evaluating system efficiency and effectiveness for each transit mode and service category. The TAC agreed upon the definition of each performance measure to ensure consistency and accuracy in data collection and reporting by all providers. The framework for performance evaluation also includes guidelines for new service implementation. The TAC agreed to use definitions of terms consistent with the National Transit Database (NTD) whenever appropriate. In some cases, this decision caused the revision of definitions previously adopted for PMAS reporting. In a few cases, the PMAS definition of a term or performance measure was adopted as the most appropriate for Valley Metro providers. 3.1 Modes and Service Categories The performance evaluation framework addresses three Valley Metro transit modes: fixed route bus, paratransit bus, and light rail. Fixed Route Bus The fixed route bus mode is defined according to NTD as transit services provided on a repetitive, fixed schedule basis along a specific route with vehicles stopping to pickup and deliver passengers to specific locations. Each fixed route trip serves the same origins and destinations. Valley Metro fixed route bus performance is evaluated at the route level using five service categories and at the system level for all fixed routes. The five fixed route bus service categories are defined by Valley Metro according to route design as follows: • Bus Category 1 – Local routes meet the local travel needs of transit riders in each city in the Valley Metro service area. Local routes may operate on either arterial or local collector streets. Funds to operate local routes generally come from fares and local government revenues. A local route is not limited to only one city. Many Valley Metro local routes serve two or more jurisdictions; however, the route design is intended to serve localized trip patterns within those cities. • Bus Category 2 – Supergrid routes are arterial grid routes that provide a regional connection function. Regional funding of this service ensures consistent (and in some cases higher) service levels across jurisdictions that would not be possible if the routes had to depend on varying local funding levels from the jurisdictions served. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 9 • Bus Category 3 – Express/Bus Rapid Transit (BRT) routes operate as overlays on corridors served by local fixed route service, but provide higher speed services by operating with limited stops and with other enhancements, such as bus-only lanes, queue-jumpers or signal priority systems. Express/BRT routes may operate on major arterials or along regional freeways. Arterial express/BRT routes may operate during peak and off-peak periods. Freeway express/BRT routes are often designed to use high occupancy vehicle (HOV) facilities to connect remote park-and- ride lots with major activity centers, including core downtown areas. These routes can also provide suburb-to-suburb connections using the regional freeway system and intermediate stops. Freeway express/BRT route are typically peak period only. Existing express/BRT routes may be locally funded. For example, RAPID routes are funded by the City of Phoenix. However, most existing express/BRT routes will transition to regional funding and new express/BRT routes will be implemented with resources from the PTF regional fund. • Bus Category 4 – Circulator routes are local fixed routes that operate in a limited service area and often using smaller vehicles. Neighborhood circulator routes operate in less dense residential areas to provide service to communities that otherwise may be difficult to serve with local routes. Activity center circulators operate in dense, high activity areas such as a downtown core. Circulator routes are designed for short trips and operate with frequent stops to provide internal circulation for an activity center. Ridership per revenue mile operated on neighborhood circulators is typically lower than local routes, while ridership per revenue mile operated on activity center circulators may be higher than local routes. • Bus Category 5 – Rural routes address the need to provide connections between the urban and rural communities of Maricopa County. The urban area is that portion of the metropolitan area served by local fixed routes. Rural routes provide connections between remote communities and urban transit nodes and address a range of trip needs for rural area residents. Paratransit Bus The definition for the paratransit bus mode is found in Valley Metro PMAS. Paratransit service is defined as specialized transportation by car, van, or bus completing trips as the result of passenger requests of specific origin and destination, either with advanced reservations or through same day call requests. Paratransit includes all complementary paratransit services to meet the ADA requirements for persons with disabilities. Valley Metro paratransit also includes specialized transportation for seniors and, in some jurisdictions, the general public. User side subsidy programs (e.g., taxi vouchers) are Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 10 included in paratransit services. Seven percent of the PTF fund for the bus transit program is dedicated to funding regional ADA paratransit. Light Rail The fourth Valley Metro transit mode is light rail. The system is now under construction and currently scheduled for opening in late 2008. As defined by NTD, light rail transit is an electric railway that is characterized by passenger rail cars operating singly or in short, usually two car, trains on fixed rails in shared or exclusive right-ofway. Valley Metro light rail vehicles will draw power from an overhead electric line via a pantograph. 3.2 Performance Measures by Mode The set of performance measures are defined for each of the three Valley Metro transit modes: fixed route bus, paratransit bus, and light rail. Performance measures for fixed route are established for route level reporting as well as for systemwide reporting. Performance measures are selected to report cost efficiency, cost effectiveness, and service effectiveness. Cost efficiency measures evaluate the amount of money spent to produce a unit of service (e.g., cost per mile). Cost effectiveness measures are used to evaluate what is achieved for the amount of money spent (e.g., cost per passenger boarding). Service effectiveness measures evaluate what is achieved for the unit of service delivered. An example of a service effectiveness measure is passenger boardings per revenue mile. There are numerous performance measures that can be used to evaluate efficiency and effectiveness. At the same time, selecting too many measures to report on is overly burdensome for preparers. In selecting the most appropriate performance measures, the Booz-Allen team and TAC members considered several factors: • • • • • • • Compliance with the appropriate legislation (as discussed in Section 2) Performance measures already used in practice by Valley Metro member jurisdictions, in particular through the existing PMAS Peer transit system performance measures for similar modes and service categories Statistics already reported for NTD Performance expectations Ability of the member agencies to efficiently collect the data required for different performance measures at the desired level of detail and frequency (monthly, quarterly, annually) Ability to ensure accuracy and consistency of the data collected. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 11 The final set of recommended performance measures is provided in Exhibit 3. Several performance measures are applicable for more than one mode. Exhibit 3 provides a cross reference for each performance measure by mode and for each reporting level for fixed route bus. Exhibit 3 Performance Measure Framework Performance Measure Fixed Route, Systemwide Fixed Route, Route Level ● ● ● ● ● ● ● ● ● Paratransit Light Rail ● ● ● ● ● ● ● Cost Efficiency/Effectiveness Farebox recovery ratio Operating cost per boarding Subsidy (net operating cost) per boarding Cost per revenue mile Cost per revenue hour Average fare ● ● ● ● Service Effectiveness Total boardings Boardings avg. weekday, Saturday, Sunday Boardings per revenue mile ● ● ● Boardings per revenue hour Safety Incidents per 100,000 Vehicle Miles Security incidents per “x” boardings Complaints per “x” boardings On-time performance Percent no shows ● ● ● ● Miles between mechanical failure3 Customer satisfaction ● ● 3.3 ● ● ● ●2 ● ● ● ● ● ● ● ● ● ● Definitions by Mode The TAC discussed and agreed upon definitions of each performance measure by mode. The following section documents the definitions of performance measures and formula for calculation, if applicable. First, the modes applicable to the performance measure are identified in a small table. The definitions of terms for data required to calculate performance measures are also presented, as well as the source for the definition. For example, farebox recovery ratio is defined as are the definitions of passenger fares and operating expenses and the data required to calculate the farebox recovery ratio. 2 For bus, recommended for Express Bus/BRT only 3 The Rail mode reports Miles between Failures. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 12 ● ● ● ● ● ● Measures of Cost Efficiency/Effectiveness Performance Measure Farebox recovery ratio Fixed Route, Systemwide Fixed Route, Route Level Paratransit Light Rail ● ● ● ● Farebox recovery ratio Percent of operating expenses that is recovered from passenger fares. Formula: Passenger fares / Operating expenses Passenger fares The revenue earned from carrying passengers on regularly scheduled and demand response services, including base fares, zone/distance premiums, express service premiums, extra cost transfers, quantity purchase discounts applicable to a passenger’s ride, and special transit fares. Passenger fares do not include advertising revenue or other operating revenue types. Source: NTD Operating expenses Total costs associated with the operation of revenue vehicles, including maintenance costs. Direct operating costs include costs incurred to provide the service, including administrative and overhead costs associated with the direct operation of the service. For cities that operate Paratransit services directly (i.e., do not subcontract the operation to a private provider), direct costs include agency costs for activities such as dispatch, reservations, scheduling, service supervisors, and other direct costs of providing the service. For contracted service, operating expenses include all of the contractor’s costs of providing service, including operations, maintenance, and Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 13 administrative cost. Operating expenses also include the documented administrative costs of the agency administering the contracted service. Operating expenses include the time of individuals responsible for the management and supervision of the contracted service plus any related direct expenses. Where costs are tracked by cost center, costs include those reported for the transit cost center (i.e., direct expenditures plus indirect expenditures). Where costs are not tracked by cost center, these are indirect costs that need to be documented and allocated to the transit program. 4 Source: PMAS, modified by the TAC Performance Measure Operating cost per boarding Fixed Route, Systemwide Fixed Route, Route Level Paratransit Light Rail ● ● ● ● Operating cost per boarding Operating expenses / Unlinked passengers trips Operating expenses Defined above. Unlinked passenger trips The number of passengers who board public transportation vehicles. Passengers are counted each time they board vehicles no matter how many vehicles they use to travel from their origin to their destination. Source: NTD Also referenced as “passenger boardings” or simply “boardings”. Source: TAC For Paratransit, data should be tracked and reported by program type (e.g. ADA paratransit, 4 Administrative costs are expected to be documented in reports to NTD. For agencies that are not required to report to NTD, a form and instructions for documenting agency administrative costs are included in Appendix B. The consultant team developed this template for agencies to use in preparation of the required performance reports. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 14 non-ADA paratransit, taxi voucher, or mileage reimbursement). Performance Measure Fixed Route, Systemwide Fixed Route, Route Level Paratransit Light Rail Subsidy (net operating cost) per boarding ● ● ● ● Subsidy per boarding Also referenced to as Net operating cost per boarding Subsidy / Unlinked passenger trip Subsidy Financial assistance from any federal, state or local government source. (Operating expenses less passenger fares) / Unlinked passenger trips Source: NTD Operating expenses Defined above. Passenger fares Defined above. Unlinked passenger trips Defined above. Performance Measure Cost per revenue mile Fixed Route, Systemwide Fixed Route, Route Level ● ● Paratransit Light Rail ● Cost per revenue mile Operating expenses / Revenue miles Operating expenses Defined above. Revenue miles The miles operated when a vehicle is available to the general public and there is an expectation of carrying passengers who pay fares, are subsidized by public policy, or provide payment through a contractual arrangement. Revenue service excludes deadhead, vehicle maintenance testing, school bus service, and charter service Source: NTD Revenue miles for Paratransit Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report The miles that vehicles are in revenue service, from the first passenger pick-up to the last drop15 off, excluding any travel during scheduled breaks. Source: NTD and TAC Discussion Performance Measure Fixed Route, Systemwide Cost per revenue hour Fixed Route, Route Level Paratransit ● ● Light Rail Cost per revenue hour Operating expenses / Revenue hours Operating expenses Defined above. Revenue hours The time when a vehicle is available to the general public and there is an expectation of carrying passengers. Vehicles operated in fare free service are considered in revenue service. Revenue service includes layover and recovery time. Revenue service excludes deadhead, vehicle maintenance testing, school bus service, and charter service. This measure should only be applied to Express Bus/BRT. Source: NTD Revenue hours for Paratransit The hours that vehicles are in revenue service, from the first passenger pick-up to last drop-off, excluding schedule breaks. Source: NTD Performance Measure Fixed Route, Systemwide Average fare ● Fixed Route, Route Level Paratransit Light Rail ● ● Average fare Passenger fares / Unlinked passenger trips Passenger fares Defined above. Unlinked passenger trips Defined above. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 16 Measures of Service Effectiveness Performance Measure Fixed Route, Systemwide Fixed Route, Route Level Paratransit Light Rail ● ● ● ● Total boardings Total boardings Unlinked passenger trips Source: TAC discussion Unlinked passenger trips Defined above. Performance Measure Fixed Route, Systemwide Fixed Route, Route Level Paratransit Light Rail Boardings avg. weekday, Saturday, Sunday ● ● ● ● Boardings average weekday The number of passengers who board public transportation vehicles on an average weekday. Source: NTD Average weekday A typical representative weekday in the operation of the transit system. Average Saturday A typical representative Saturday in the operation of the transit system. Average Sunday A typical representative Sunday in the operation of the transit system. Source: NTD Performance Measure Fixed Route, Systemwide Fixed Route, Route Level ● ● Boardings per revenue mile Paratransit Light Rail Boardings per revenue mile Unlinked passenger trips / Revenue miles Unlinked passengers trips Defined above. Revenue miles Defined above. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 17 ● Performance Measure Fixed Route, Systemwide Fixed Route, Route Level Paratransit ● ● Boardings per revenue hour Light Rail Boardings per revenue hour Unlinked passenger trips / Revenue hours. This measure should only be reported for Express Bus/BRT (Category 3 Bus service). Unlinked passengers trips Defined above. Revenue hours Defined above. Performance Measure Fixed Route, Systemwide Safety incidents per 100,000 vehicle miles Safety incidents per 100,000 vehicle miles Safety incidents ● Fixed Route, Route Level Paratransit Light Rail ● Safety incidents / Total vehicle miles / 100,000 Safety incidents involve a transit vehicle or occur on transit-controlled property and meet one or more of the conditions described below. Safety incidents include only “major” safety incidents. Major Safety Incidents include one or more of the following conditions: • A fatality other than a suicide • Injuries requiring immediate medical attention away from the scene for two or more persons • Property damage equal to or exceeding $25,000 • An evacuation due to life safety reasons, • A collision at a grade crossing resulting in at least one injury requiring immediate medical attention away from the scene or property damage equal to or exceeding $7,500 • A mainline derailment [on rail] • A collision with person(s) on a rail right-ofway resulting in injuries that require Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 18 immediate medical attention away from the scene for at least one person. Source: NTD Vehicle miles The miles that a vehicle travels from the time it pulls out from its garage to go into revenue service to the time it pulls in from revenue service. Total vehicle miles include deadhead, vehicle maintenance testing, school bus service, and charter service. Source: NTD and TAC Discussion Performance Measure Fixed Route, Systemwide Security incidents per “x” boardings Security incidents per “x” boardings ● Fixed Route, Route Level Paratransit Light Rail ● Security incidents / Unlinked passenger trips The “x” should be determined based on the magnitude of the boardings (generally such as the measure be in the single digits). Security incidents Security incidents are crimes such as injuries or deaths resulting from assaults, arson, or homicide and the consequences of security incidents. Security incidents should not be reported as safety incidents. Security incidents only include “major” security incidents. Major security incidents (i.e., crimes) produce the threshold values for major incident reporting (a fatality, two or more injuries, property damage over $25,000). Security incident types include aggravated assault, arson, bombing, bomb threat, burglary, chemical or biological release, hijacking, homicide, larceny/theft, motor vehicle theft, robbery, sabotage, and vandalism. Source: NTD Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 19 Performance Measure Fixed Route, Systemwide Complaints per “x” boardings Fixed Route, Route Level Paratransit Light Rail ● Complaints per “x” boardings Complaints / Unlinked passenger trips / “x” The “x” should be determined based on the magnitude of the boardings. Complaints Not defined. Valley Metro Customer Service issues reports on complaints. Customer Service is in the process of purchasing a new customer contact software program. Source: TAC discussion Unlinked passengers trips Performance Measure Defined above. Fixed Route, Systemwide Fixed Route, Route Level Paratransit Light Rail ● ● ● ● On-time performance On-time performance for fixed route Percent of all fixed route trips that operate no more than 0.0 minutes early and 5.0 minutes late, compared to scheduled arrival/departure times at published time points. On-time performance is reported by the Valley Metro automated Vehicle Management System (VMS). Source: PMAS On-time performance for paratransit For ADA service, on-time performance is the percent of all ADA trips that are picked up within the 30 minute ready window. For non-ADA service, on-time performance is the percent of non-ADA trips that are picked up within the ready window. This measure does not apply to user-side subsidy services. Source: TAC discussion and Paratransit providers Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 20 On-time performance for light rail Percent of all rail trips that operate no more than 0.0 minutes early and 5.0 minutes late, compared to scheduled arrival/departure times at each station. Source: Question for VMR Performance Measure Fixed Route, Systemwide Fixed Route, Route Level Percent no shows Paratransit Light Rail ● Percent no shows No shows/ Total paratransit boardings No shows Paratransit patrons who do not call to cancel a reservation but do not appear to board (“no show”) when the vehicle arrives within the ready window. No shows include paratransit patrons who cancel when the vehicle arrives at the curb. Source: TAC discussion and Paratransit providers Performance Measure Fixed Route, Systemwide Miles between mechanical failure Fixed Route, Route Level ● Paratransit Light Rail ● ● Miles between mechanical failure Vehicle miles / Mechanical system failure Vehicle miles Defined above. Mechanical system failure A major mechanical system failure is a failure of some mechanical element of the revenue vehicle that prevents the vehicle from completing a scheduled revenue trip or from starting the next scheduled revenue trip because actual movement is limited or because of safety concerns. Other mechanical system failures are failures of some other mechanical element of the revenue vehicle that, because of local agency policy, prevents the vehicle from completing a schedule revenue trip or from starting the next Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 21 revenue trip even though the vehicle is physically able to continue in revenue service. Source: NTD Performance Measure Fixed Route, Systemwide Customer satisfaction Customer satisfaction Fixed Route, Route Level ● Paratransit Light Rail ● ● Index representing customer satisfaction for a particular mode, measured through a survey. Valley Metro and member agencies conduct periodic market surveys to evaluate customer satisfaction. Survey results are published and provide data for historical trend analysis for Fixed Route and Paratransit modes. Valley Metro Rail intends to conduct similar market surveys to measure customer satisfaction. Source: TAC discussion 3.4 New Route Implementation Guidelines Reasonable expectations need to be made in implementing new routes. In areas where population and employment density are significantly low or supporting passenger facilities are unavailable, the affected jurisdiction should consider a phased service approach. A population and employment density approach to service phasing could be accomplished by establishing a baseline point system (i.e., a “Density Scale” analysis). A planned route would be credited or debited a point for each percentage point of difference between the population and employment density within 0.25 miles of the planned route and the regional system averages. The points would be generated by RPTA planning staff using adopted MAG population and employment projections. The results could serve as a reference for affected jurisdictions to determine if they would prefer to fully implement service on opening day or implement it in phases. Due to the region’s rapid growth, it wouldn’t be reasonable to create the Density Scale for all planned routes of the 20-year program at the same time. It is recommended that the Density Scale points be calculated no more than two years in advance of a route’s implementation. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 22 A phased implementation could include reduced headway, span of service, or geographic coverage. Phasing service implementation should not effect reduced funding for the affected community. Reduced service from planned levels should provide affected communities with proportional jurisdictional savings. If for a given route, demand materializes faster than expected and more service is required, then the jurisdiction can reallocate dollars from other services within the limits of their jurisdictional allocation or local funds. The methodology to potentially change service plans is driven by the Proposition 400 and the Transit Life Cycle Program. First, the approved proposition limits the transfer of funds between the three primary modes: freeways, streets and transit. However, funds within a mode can be moved from one project to another if a review and approval process is followed consistent with the jurisdictional equity process identified in the TLCP). Second, plan elements can be adjusted based upon periodic reviews of projected revenues and expenditures. Third, the voter approved plan is amendable, but consideration must be given to alternative projects in the same corridor by the same mode. 3.5 Route Maturation Guidelines When new routes are implemented, passengers boardings will not be expected to achieve the same levels during the initial months of service as when the route matures. Typically route maturation guidelines exempt new routes from performance standards for two years, giving them time for a higher ridership base to develop, and for the route to become more productive and efficient. The TAC endorsed several recommendations related to route maturation guidelines. • The first recommendation is to use a two year period for the maturation period. The individual route performance would be calculated from time of inception, but its performance would not be included for two years. • The second recommendation is to track subsidy per boarding (i.e., net operating cost per unlinked passenger trip) as primary measure of performance, with the intent of focusing possible corrective action for the routes requiring the highest subsidy per boarding in the first two years. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 23 • The third recommendation is for RPTA staff, working with partner agencies, to identify a lifeline network. This network would be maintained over time to serve Title VI markets or other policy requirements even if its performance fell below the systemwide average. After the two year maturation period, all routes would be part of the performance measurement calculations. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 24 4. ACTION PLAN The action plan includes development of a reporting tool and guidelines for performance data reporting, plus agency roles and responsibilities involved in reporting performance data and measures. RPTA staff will have primary responsibility for developing the reporting tool and guidelines, but it is recommended that they involve the members of the Effectiveness and Efficiency Study TAC in reviewing and commenting on draft documents and procedures. Transparency and availability of regional transit performance information are key drivers to this Action Plan. 4.1 Setting Performance Targets Performance targets, or benchmarks, are an important tool in gauging transit performance. During the final stages of the study, the consultant team worked with Valley Metro/RPTA and member agencies to develop targets. For the most part, these are conservative, based on known historical performance. In other words, the targets identified reflect known past historical performance, not an arbitrary “stretch” target. It is anticipated the Valley Metro/RPTA and VMR Boards will weigh in the targets over time as a matter of policy. The targets presented here are a starting point. Exhibit 4 – Fixed Route Bus, Systemwide Targets FIXED ROUTE BUS, SYSTEMWIDE Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings Complaints per "x" Boardings On-time Performance Miles between Mechanical Failures Customer Satisfaction TARGET 25% $2.32 $1.75 $4.96 $0.67 3%* 3%* 2.1 1.2 0 28 90% 23,400 89% Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report ASSUMPTIONS Regional Fare Policy recommendation to Board Baseline from FY05-06 PMAS FR average Baseline from FY05-06 PMAS FR average Baseline from FY05-06 PMAS FR average Five year timeframe starting in FY08 Matches 3% CPI increase; Subject to service increases Matches 3% CPI increase; Subject to service increases Baseline from FY05-06 PMAS FR average Baseline from FY05-06 PMAS FR average Ultimate goal, 0 was achieved by RPTA Veolia for entire 2006 Low end of current RPTA "C" range; meets Phoenix current complaints Meets both RPTA and Phoenix current performance Baseline from CY06 RPTA data (weighted average across fleets). Phoenix OK with target. Need to move to Miles between Mechanical Failures over time with next contract negotiations. Number will go down some, bcs there are more Mech. Failures than Roadcalls. Customer satisfaction index based on March survey - Combination of "very satisfied" and "somewhat satisfied" with overall service 25 Exhibit 5 – Fixed Route Bus, Route Level Targets FIXED ROUTE BUS, ROUTE LEVEL Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Boardings per Revenue Hour (Express Bus) On-time Performance Miles between Mechanical Failures TARGET 25% $2.32 $1.75 $4.96 3%* 3%* 2.1 TBD 90% 23,400 ASSUMPTIONS Regional Fare Policy recommendation to Board Baseline from FY05-06 PMAS FR average Baseline from FY05-06 PMAS FR average Baseline from FY05-06 PMAS FR average Matches 3% CPI increase; Subject to service increases Matches 3% CPI increase; Subject to service increases Baseline from FY05-06 PMAS FR average; peer average Challenge: Phoenix does not currently break out this data. Meets both RPTA and Phoenix current performance Baseline from CY06 RPTA data (weighted average across fleets). Need to move to Miles between Mechanical Failures over time with next contract negotiations. Number will go down some, bcs there are more Mech. Failures than Roadcalls. Exhibit 6 – Paratransit Targets PARATRANSIT Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Operating Cost per Revenue Hour Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Hour TARGET 5% $28.55 $27.16 $50.30 TBD 3%* 3%* 1.76 Percent No Shows 5% On-time Performance Miles between Mechanical Failures Customer Satisfaction 90% TBD 90% Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report ASSUMPTIONS Baseline from FY05-06 PMAS DAR system average Baseline from FY05-06 PMAS DAR system average Baseline from FY05-06 PMAS DAR system average Baseline from FY05-06 PMAS DAR system average No goal currently. Set by each agency Matches 3% CPI increase; Subject to service increases Matches 3% CPI increase; Subject to service increases Baseline from FY05-06 PMAS DAR system average; close to low of peers Phoenix does not have a target for "no shows", but has seen 5.3% from July-Dec 2006. Paratransit consultant/RPTA recommended 5%. The Glendale target is 10%. Glendale target. Phoenix's performance standard is also 90% or above Set by each agency (Phoenix tracks miles per 100,000 service miles) Glendale and Phoenix (Satisfied and Very Satisfied; every 2-5 yrs). 26 Exhibit 7 – Rail Targets RAIL Cost Efficiency/Effectiveness Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare Service Effectiveness Total Boardings Boardings Avg. Weekday Boardings Avg. Sat. Boardings Avg. Weekday Sun./Holiday Boardings per Vehicle Revenue Mile Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings On-time Performance Miles between Failures Customer Satisfaction TARGET 25% $2.64 $1.98 $26.26 $0.67 10,655,000 26,090 N/A N/A 3.94 8.04 N/A N/A 95% 25,000 89% ASSUMPTIONS Regional Fare Policy recommendation to Board Booz Allen estimate calculated for 2010. Close to the peer average of $2.18 Booz Allen estimate based on farebox recovery ratio and operating cost per boarding assumptions Booz Allen estimate based on $28M operating cost and 1,071,000 revenue car miles in Financial Plan. 2010 numbers. Assumption behind VMR fare revenue numbers (2010) VMR Estimate for 2010, first full year of operation Based on FFGA New Starts for Opening Year (Annual approx. 7.8M) Not available Not available Rail. Based on 2,184,000 car miles in 2010 (Financial Plan). Rail. Based on 1,071,000 car miles in 2010 (Financial Plan). To be provided in Spring, 2008 as part of VMR Budget process To be provided in Spring, 2008 as part of VMR Budget process Rail (Peer data is 98%) Rail (Peer data shows 35,000 with outliers excluded) Estimate index number based on bus targets The rail targets are preliminary. Very little data is available on which to base targets until the system is in revenue service. Current assumptions include the Financial Plan and operating assumptions reflected in the New Starts criteria. Note that the Proposition 400 requirement does not supersede, nor replace, the New Starts process. New Starts project measures will continue to be tracked and reported to the FTA in conformance to the New Starts process. 4.2 Implement Data Tool Using the performance indicators and data definitions detailed in Section 3, RPTA staff will develop a reporting format, software platform (e.g., Excel, Access) and guidelines to be used by the agencies responsible for reporting route and mode level performance data. It is recommended that the reporting format and platform minimize data re-entry by RPTA staff and facilitate data reporting, manipulation and accumulation. The reporting format could be customized for each reporting entity, to minimize the likelihood of incomplete reporting. It is recommended that the reporting format include performance indicators as well as the performance data required to calculate them, plus the ability to review prior quarters’ data, to facilitate consistency checks. A sample reporting format is provided in Exhibit 8. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 27 Exhibit 8 – Sample Performance Reporting Format Performance Data Fare Revenue Operating Costs Boardings Weekday Saturday Sunday/Holiday Revenue Miles Rte 1 1st Qtr (Jul-Sep) Rte 2 Rte 3 Total FY 2008 4th Qtr 2nd Qtr 3rd Qtr (Oct-Dec) (Jan-Mar) (Apr-Jun) Rte 1 Rte 2 Rte 3 Total Rte 1 Rte 2 Rte 3 Total Rte 1 Rte 2 Rte 3 Total Rte 1 Year to Date Rte 2 Rte 3 Total Performance Indicators Farebox Recovery Operating Cost per Boarding Operating Cost per Revenue Mile Boardings per Revenue Mile Performance guidelines should be issued to all reporting agencies and should include performance data definitions; identify data sources to promote consistency; stipulate reporting formats, platforms and timeframes; and provide directions for updating data. RPTA staff should consider opportunities to facilitate the reporting process, both for the reporting agencies and for the RPTA staff who will be responsible for using and consolidating the data. Web-based reporting could provide such an opportunity. 4.3 Reporting Timeframe and Procedures Frequency of reporting Following the guidelines, reporting format and timelines provided by RPTA, reporting agencies (as defined in Section 4.3 Roles and Responsibilities) will be responsible for submitting quarterly and annual performance data. RPTA staff will compile the data, develop route level and modal performance indicators, and prepare reports for dissemination to the audiences identified below. RPTA staff will be responsible for computational accuracy, and for defining report formats and establishing reporting timeframes. Reports will be prepared on a quarterly basis and will cumulate in an annual report at the end of each fiscal year. The first quarter should be from July 1 through September 30. The last quarter should therefore be from April 1 through June 30. Reporting timelines will recognize the need at year-end to reconcile and update previouslyreported data to report audited year-end data. RPTA staff should develop an annual reporting schedule and timeline, including a process for reminding reporting agencies when reports are due. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 28 Reporting by audience Throughout the performance measurement framework development, the study team recognized the need to include interests and reporting requirements of different audiences. Four key audiences were identified, as discussed below. Performance indicators identified for each audience are shown in Exhibit 9. The list is a starting point and can be modified or updated over time as appropriate. The important aspect is the need to officially report system performance to stakeholder groups on a quarterly basis. • Member Agency/RPTA Practitioners – those individuals at member agencies (including Valley Metro Rail) who, given their responsibilities for monitoring performance and performance trends, are interested access to all performance measures, for all modes and at both the system-wide and route level. In many cases, the same individuals will be responsible for reporting the performance data to RPTA/Valley Metro. • CTOC, the Legislature, the State Auditor and the Performance Auditor(s) who will be retained to conduct the five-year performance audits are expected to have an interest almost all of performance measures, by mode and at both the systemwide and route levels. It was generally agreed that they would want to see boardings at the mode, system-wide, and route level, but that boardings by dayof-week would be of limited interest to this audience. • Member Agency Councils, who are responsible for funding the services operated by or for the member agencies, are likely to be interested in most, but not all of the performance measures. It was acknowledged that individual Councils had, in the past, requested information on specific performance indicators and that there would continue to be a need to report additional performance measures for these audiences on a City-specific basis. • RPTA Board members and the Public are also likely to be interested in a portion of the performance measures identified to meet Proposition 400 needs. Like the Member Agency Councils, RPTA Board members are likely to be most interested in indicators that measure cost efficiency and the aspects of performance that are most likely to impact the public (e.g., safety, on-time performance, customer satisfaction). Performance reporting to the Board should occur no less than once per year. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 29 Exhibit 9 – Performance Measure Audiences FIXED ROUTE BUS, SYSTEMWIDE Member Agency / CTOC / Auditor(s) / RPTA Practitioners Legislature Cost Efficiency/Effectiveness D D D D Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Member Agency Councils RPTA Board D D D D D D D D D D D D D D D D D D D D D Service Effectiveness D D D D D D D D D Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings Complaints per "x" Boardings On-time Performance Miles between Mechanical Failures Customer Satisfaction D D D D D D D D Others Measures, specific to individual cities FIXED ROUTE BUS, ROUTE LEVEL Member Agency / CTOC / Auditor(s) / RPTA Practitioners Legislature Cost Efficiency/Effectiveness D D D D Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Member Agency Councils RPTA Board D D D D Service Effectiveness D D D D D Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile On-time Performance Miles between Mechanical Failures D D D D Others Measures, specific to individual cities PARATRANSIT Member Agency / CTOC / Auditor(s) / RPTA Practitioners Legislature Cost Efficiency/Effectiveness Member Agency Councils RPTA Board D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D D Farebox Recovery Ratio Operating Cost per Boarding Subsidy (Net Opg Cost) per Boarding Operating Cost per Revenue Hour Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile D Boardings per Revenue Hour Percent No Shows On-time Performance Miles between Mechanical Failures Customer Satisfaction D Others Measures, specific to individual cities RAIL Member Agency / CTOC / Auditor(s) / RPTA Practitioners Legislature Cost Efficiency/Effectiveness Farebox Recovery Ratio Subsidy (Net Opg Cost) per Boarding Cost per Revenue Mile Average Fare D D D D Member Agency Councils RPTA Board D D D D D D D D D D D D D D D D D D D Service Effectiveness Total Boardings Boardings Avg. Weekday, Sat., Sun. Boardings per Revenue Mile Safety Incidents per 100,000 Vehicle Miles Security Incidents per "x" Boardings On-time Performance Miles between Failures Customer Satisfaction D D D D D D D D D D D D D D D Others Measures, specific to individual cities Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 30 4.4 Roles and Responsibilities This section defines agency roles and responsibilities for collecting, compiling and reporting performance data, indicators and trends. Data Collection Those member agencies that directly operate or contract for transit services will be responsible for collecting and reporting data to RPTA staff, using the reporting formats and following reporting guidelines and timelines defined by RPTA staff. Data will be collected and reported by mode and at the route level for fixed route bus services, consistent with the data definitions provided in Section 3.3 Definitions by Mode. Data will be collected from primary sources such as contractor invoices and reports, internal sources used to collect data for National Transit Database (NTD) reporting and other audited reports. Member agencies and Valley Metro/RPTA are encouraged to scrutinize contractor data for accuracy and completion. Operating costs need to be captured as per the definitions in Section 3.3. In some cases, the current reporting of the contractor will have to be changed. For example, vehicle reliability is measured in miles between mechanical failures, not miles between road calls. It is important to measure road calls for the day to day management of road supervision, but the reporting of mechanical failures is required to calculate the vehicle reliability measure. RPTA staff will be responsible for collecting data from reporting agencies and compiling them to provide the route level and mode level performance indicators defined in Section 3. If the agencies report directly on-line, than RPTA staff will be responsible for ensuring that the submittals are timely and complete. RPTA staff will also review data and performance indicators and trends for consistency and work with reporting agencies to review and resolve data issues. Report Preparation RPTA staff will be responsible for formatting, compiling, and reporting performance data, indicators and trends. Reports will be prepared on a quarterly basis for briefing to the TMC, Intergovs, and Board. The reports will include year-to-date data, cumulating to an annual report at the end of a fiscal year. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 31 Quality Control One challenge with compiling data and reporting indicators from a variety of sources is ensuring quality control. Of particular concern is the need to maintain an up-to-date database, in part by requiring revisions to previously-reported data. As agencies audit data and prepare year-end reports, it will be necessary to ensure that data maintained in the RPTA performance database are updated to reflect final, year-end, audited numbers. As noted in the section on Data Collection, RPTA staff should also be responsible for reviewing and validating the consistency of performance data and indicators. Rather than revising previously-issued reports, data should cumulate so that the most recent report includes the most current data. For this reason, RPTA staff may choose to report each quarter in each report and to cumulate the data to report year-to-date results. 4.5 Action Summary and Timeframe Implementation of the new performance measurement framework will require hard work and commitment on the part of RPTA staff and its member agencies. To fully realize the potential of the initiative, changes to operating practices are needed. Examples include full accounting in the calculation of operating costs and slight changes in contractor reporting practices, such as route level costs. A summary of the major recommendations is provided in Exhibit 10 below. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 32 Exhibit 10 Summary Implementation Plan CATEGORY RECOMMENDATION ACTION OWNER TIMEFRAME Framework Adoption Adopt performance measures framework RPTA and VMR Boards April-June 2007 Target Setting and Adoption Discuss target goals for each performance measure Discuss target goals for each performance measure Adopt draft targets for framework TMC, VMOCC RPTA and VMR Boards RPTA and VMR Boards April-July 2007 February-June 2007 June-August 2007 Guidelines Adoption Discuss route implementation and maturation guidelines Discuss route implementation and maturation guidelines Identify "lifeline" network Adopt route maturation guidelines TMC, VMOCC RPTA Planning Dept and Board RPTA Planning Dept, Members RPTA Planning Dept and Board April-June 2007 April-June 2007 April-Sept 2007 June-July 2007 Information Issue performance guidelines to all reporting agencies RPTA Planning Dept April-May 2007 Implement Data Tool Develop final reporting format Complete transition from PMAS Develop reporting platform Work with individual members to facilitate process RPTA Planning Dept RPTA Planning Dept RPTA Planning Dept RPTA Planning Dept April-Sept 2007 April-Sept 2007 April-Dec 2007 April-Dec 2007 Contract Negotiations Include all measures in reporting requirement (system wide and route level) Tempe, Phoenix, Glendale RPTA Operations April 2007 - Dec 2008 Preparation for New Routes Conduct density scale analysis for new route implementation RPTA Planning Dept Ongoing New Route Monitoring Monitor new routes according to route maturation guidelines RPTA Planning Dept Ongoing Reporting Begin monthly reporting with new system Begin quarterly, etc briefing to Audiences - Bus/DAR Begin quarterly, etc briefing to Audiences - Rail RPTA Planning Dept RPTA Operations Mgr VMR Operations Mgr July 2007 Oct 2007 (for July 1 - Sept 30) First quarter after startup Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 33 APPENDICES Valley Metro/RPTA Efficiency and Effectiveness Study Report Appendix A Appendix A Technical Advisors Committee – Service Effectiveness & Efficiency Study Monique de la Rios-Urban Senior Performance Manager Maricopa Association of Governments (602) 254-6300 mdelos@mag.maricopa.gov Janeen Gaskins Grants Administrator City of Avondale (623)478-3025 jgaskins@avondale.org Rogene Hill Assistant City Manager City of Avondale (623)478-3012 rhill@avondale.org Greg Jordan Transit Administrator City of Tempe 20 East 6th Street Tempe, AZ 85281 Office: 480.858.2094 Mobile: 480.296-5285 greg_jordan@tempe.gov Tami Ryall Deputy Manager Town of Gilbert 1025 South Gilbert Road Gilbert, AZ 85296-3401 (480) 503-6765 tamir@ci.gilbert.az.us Valley Metro/RPTA Efficiency and Effectiveness Study Report Joseph F. Marie Director, Operations and Maintenance 411 North Central Avenue, Suite 200 Phoenix, Arizona 85004 Work Phone: 602-744-5575 Cell Phone: 602-702-2114 jmarie@valleymetro.org Matt Carpenter Public Transportation Division Arizona Department of Transportation 206 South 17th Ave., Mail Drop 310B Phoenix, AZ 85007 602-712-6790 Mcarpenter@azdot.gov Mike Normand, Transit Planning Manager City of Chandler Public Works/Transit Services Mail Stop 412, PO Box 4008 Chandler, AZ 85244-4008 (480) 782-3440 mike.normand@ci.chandler.az.us Mike Sabatini Planning Division Manager MCDOT 2901 West Durango St. Phoenix, AZ 85009 (602) 506-8628 mikesabatini@mail.maricopa.gov Matthew Dudley Transit Supervisor City of Glendale (623)930-3507 Mdudley@GLENDALEAZ.com A-1 Appendix A Technical Advisors Committee – Service Effectiveness & Efficiency Study (continued) Debra Astin Transit Manager Transportation Department 7447 East Indian School Rd. #205 Scottsdale, AZ 85251 (480) 312-2526 dastin@scottsdaleaz.gov Jim Huling Assistant to the City Manager City of Mesa Box 1466 20 East Main Street #750 Mesa, AZ 85211 (480) 644-5796 jim.huling@cityofmesa.org Dale Hardy Transit Planning Manager Phoenix Transit Department 302 North 1st Avenue, Suite 900 Phoenix, AZ 85003 (602) 262-1611 dale.hardy@phoenix.gov Anne MacCracken Transit Planner Valley Metro/RPTA 302 North 1st Avenue, Suite 700 Phoenix, AZ 85003 (602) 495-0352 amaccracken@valleymetro.org Dan Lundberg Community Initiatives Director City of Surprise 15832 North Holyhock Surprise, AZ 85374 (623) 583-1688 daniel.lundberg@surpriseaz.com Stuart Boggs Manager of Transit Planning Valley Metro/RPTA 302 North 1st Avenue, Suite 700 Phoenix, AZ 85003 (602) 534-5474 sboggs@valleymetro.org Bryan Jungwirth Assistant to the General Manager Valley Metro/RPTA 302 North 1st Avenue, Suite 700 Phoenix, AZ 85003 (602) 534-1803 bjungwirth@valleymetro.org Valley Metro/RPTA Efficiency and Effectiveness Study Report A-2 Appendix B Appendix B Template for Calculating Total Operating Cost This template provides a useful tool for a city/agency to step through what are typical expenses in support of Valley Metro that might be incurred but not typically reported. Highlights • Assumption is that the primary cost is Salaries and Wages for personnel. • There are a number of categories to capture other Direct Costs. • There is also a category to apply the city/agency’s Administrative Cost Rate to capture city administration overhead. Steps The city/agency should fill in all of the bright yellow blanks for the template to calculate annual contribution. There is one choice to enter percent of time (bright yellow) OR actual hours of time (light blue/green area). • Employee Name and Title. The city/agency can enter as many names and titles for staff that devotes time to transit as appropriate (if uncomfortable about the use of names, use title only). • Pay rate per hour for each employee. • Time as Percent of Effort OR Time as Hours Contributed. An agency can enter percent of time by person (bright yellow); the template will automatically calculate hours. Or the agency may chose to enter number of hours directly (light blue/green area). Note: the template is intended to be a spreadsheet to assist a local entity to identify and document expenses in support of Valley Metro. When printed, the names of individuals will not appear on the printed document (only the position title or other reference in column C, rows 4-17 will appear in the printed document). Summary data can also be inserted directly into the master spreadsheet by line item. If the city/agency judges that stating employee pay data is too revealing, then average rates or mid-points of salary ranges can be substituted. • City or Agency Reporting. This is self explanatory. • Reporting Period and Contact Information. This is self explanatory. • Direct Personnel Costs. These will calculate automatically. • Fringe Benefits. Enter applicable payroll benefit rate. There is an option to have two different rates for different classification for personnel contributing. There is also the ability to separate health insurance costs per unit from payroll benefits that are a percent of salaries/wages. • Travel and Per Diem. Include if there is a need to capture expenses including miles to/from meetings or travel to other cities/states on Valley Metro related business. • Capital Equipment. The intent is to provide a place to report cash outlay for a capital expense in support of transit that is not reported elsewhere. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report B-1 Appendix B • • • • • • Expendable Goods or Supplies. These should be limited to a purchase of any expendable supplies that can be directly linked to a meeting or project in support of a Valley Metro service or function. Professional Services. Intended to capture local efforts for planning or technical support for Valley Metro. Again, this would need to be work that is directly linked to a project in support of a Valley Metro service or function and not reported elsewhere. Direct Transit Services. Intended to capture local direct expenses to deliver transit or paratransit service. For example, the user-side subsidy for a taxi-cab voucher program. Other Operating Expenses directly linked to a meeting or project in support of a Valley Metro service or function. Other Operating Expense (Not included when applying Indirect Cost in next step). This category primarily recovers cost for computer services, etc. However, it might be used to capture utilities or rent, etc. Administrative (Indirect) Cost Rate. Should be limited to a city or agency rate that is established for other government funding services and is supported by an approved indirect cost rate for administrative expenses. If the administrative indirect cost rate does not apply, leave blank. Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report B-2 Appendix B CITY or AGENCY Reporting Agency Costs Contributing to Valley Metro Transit Services Estimated Contribution for Period: Name of Person Preparing Report Email contact and phone number Est. Hours Description PERSONNEL Direct Costs Salaries and Wages - Regular Time Manager Planner Planner Planner Planner Planner Analyst Analyst Clerical Clerical Total Salary and Wages Fringe Benefits 16.100% Payroll Benefit Rate (describe benefit) 16.100% Payroll Benefit Rate (describe benefit) $459.00 Health Insurance per staff month $459.00 Health Insurance per staff month Total Fringe Benefits Travel and Per Diem Miles Rental Car Meals Lodging Airfare Total Travel and Per Diem Capital Equipment (Provide Explanation) Total Capital Equipment Expendable Goods/Supplies Maps, Publications Computer Supplies Materials for Meetings Total Expendable Goods/Supplies Professional Services Consultant - insert description of services performed. Est. Staff Mo. 0.20 0.30 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.50 416 624 0 0 0 0 0 0 0 0 Hourly Rates $50.00 $40.00 $35.00 $24.00 $20.00 $45.00 $15.00 $14.00 $20.00 $18.00 Line Item Total $ $ $ $ $ $ $ $ $ $ 20,800 24,960 - $7,597 No. 2150 2 8 8 2 Rate $0.445 $50.00 $36.00 $85.00 $220.00 $957 $100 $288 $680 $440 $2,465 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 $0 Quantity 0.50 0 0 0 Rate $225 $0 $0 $0.00 $113 $0 $0 $0 $113 $ Adjusted Total Direct Cost* $ 55,822 $113 55,935 $10,048 $10,048 TOTAL CONTRIBUTION TO VALLEY METRO TRANSIT $ * Clerical Staff are directly charged as the support required is significantly greater than the routine level of services ** Excludes Capital Equipment, Professional Services, Other Operating Costs that do not qualify for Indirect Cost Rate *** Administrative Indirect Cost Rate represents city/agency overhead for administrative expenses such as legal, purchasing, and human resources that are available to all departments and allocated as a percent of labor and direct expenses Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report 45,760 $7,367 $0 $230 $0 Subtotal Other Operating Expenses (NO INDIRECT) Total Other Operating Expenses TOTAL DIRECT COSTS Administrative Indirect Cost Rate*** 18.0% TOTAL ADMINISTRATIVE INDIRECT COSTS Total Est. Cost $ Consultant - insert description of services performed. Total Subcontract/Consultant Direct Transit Service Expenses Transit program - Insert description of operating expense Example, Cab Connection User-Side Subsidy Total Total Direct Transit Serivce Expenses Other Operating Expenses Telephone (Long Distance Charges) Reproduction/Printing Air Courier/Express Mail Other (Specify) Subtotal Other Operating Expenses Other Operating Costs (NO INDIRECT will be charged) Computer Operations Category Subtotals B-3 65,983 Appendix C Appendix C Technical Advisory Committee Presentations This Appendix contains all seven Technical Advisory Committee presentations carried out during 2006. They are shown in chronological order as follows. • • • • • • • TAC Presentation #1 (April 27, 2006) TAC Presentation #2 (May 24, 2006) TAC Presentation #3 (June 28, 2006) TAC Presentation #3 (August 23, 2006) TAC Presentation #3 (September 27, 2006) TAC Presentation #3 (October 24, 2006) TAC Presentation #3 (November 29, 2006) Valley Metro/RPTA Efficiency and Effectiveness Study Phase I Report C-1