A REPORT TO THE ARIZONA LEGISLATURE Performance Audit Division Special Report Arizona Department of Economic Security—Children Support Services—Transportation Services March • 2014 REPORT NO. 14-101 Debra K. Davenport Auditor General The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the operations of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Representative John Allen, Chair Senator Chester Crandell, Vice Chair Representative Paul Boyer Representative Andrea Dalessandro Representative Martin Quezada Representative Kelly Townsend Representative Andy Tobin (ex officio) Senator Judy Burges Senator David Farnsworth Senator Steve Gallardo Senator Katie Hobbs Senator Andy Biggs (ex officio) Audit Staff Dale Chapman, Director Jeremy Weber, Manager and Contact Person Marc Owen, Team Leader Brandi Melancon Brenda Quintero The Auditor General’s reports are available on our Web site at: www.azauditor.gov Printed copies of our reports may be requested by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 STATE OF ARIZONA OFFICE OF THE DEBRA K. DAVENPORT, CPA AUDITOR GENERAL MELANIE M. CHESNEY AUDITOR GENERAL DEPUTY AUDITOR GENERAL March 14, 2014 Members of the Arizona Legislature The Honorable Janice K. Brewer, Governor Mr. Clarence Carter, Director Department of Economic Security Mr. Charles Flanagan, Director Division of Child Safety and Family Services Transmitted herewith is a report of the Auditor General, A Special Report of the Arizona Department of Economic Security—Children Support Services—Transportation Services. This special report is in response to Laws 2013, 1st S.S., Ch. 10, §33, and was conducted under the authority vested in the Auditor General by Arizona Revised Statutes §41-1279.03. I am also transmitting within this report a copy of the Report Highlights for this report to provide a quick summary for your convenience. As outlined in its response, the Arizona Department of Economic Security’s Division of Child Safety and Family Services agrees with all of the findings and plans to implement or implement in a different manner all of the recommendations. My staff and I will be pleased to discuss or clarify items in the report. Sincerely, Debbie Davenport Auditor General Attachment 2910 NORTH 44 th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051 Arizona Department of Economic Security—Children Support Services—Transportation Services REPORT HIGHLIGHTS SPECIAL REPORT Our Conclusion The Office of the Auditor General has completed a special report on the Arizona Department of Economic Security’s (Department) provision of transportation services for children support services. In fiscal year 2013, the Department paid at least $14.6 million for transportation services for child protective services (CPS) clients. However, the Department does not use performance measurement to manage and evaluate these services, including overseeing contracted transportation providers. The Department should develop and implement a performance measurement system to evaluate these services and ensure that this system provides the necessary data to evaluate the appropriateness and cost-effectiveness of contracting for these services. In addition, although the Department has implemented some procedures to help ensure proper payments to transportation providers for these services, additional steps would help address internal control deficiencies. 2014 March • Report No. 14-101 At least $14.6 million in transportation services provided to CPS clients in fiscal year 2013 The Department provides transportation services to children and families (clients) in the CPS system throughout the State, using contracted transportation providers, department staff, and foster parents. Transportation services include transporting clients to and from family visits, medical/counseling appointments, school, daycare, or other required appointments. Although some of the Department’s transportation costs for children support services are embedded within other department expenses, the Department paid providers at least $14.6 million for transportation costs in fiscal year 2013, an increase of nearly $6.7 million since fiscal year 2011. The increase corresponds to an increase of children in out-of-home care from 10,974 as of June 2011 to 14,494 as of June 2013. We asked other western states about their rates for contracted transportation services for CPS clients. Four of the five states that responded reported that they did not contract for such services. These states reported that they provided transportation through state workers, volunteers, and/or foster parents. Department should use performance measurement to manage transportation services Department does not use performance measurement to manage and evaluate transportation services—The Department’s transportation contracts include performance expectations regarding the timeliness of pickups, acceptance of trip referrals, and driver no-shows. However, the Department does not require providers to report whether these expectations are met. In addition, although the Department collects some information through monthly activity reports, such as the number of trips per client, the number of cancellations by the provider, and the number of late pickups, department staff reported using this information for billing purposes only and not for contractor oversight. Department should implement performance measurement system—An effective performance measurement system collects and assesses performance data against a performance standard. Our report lists examples of common performance measures used by the transit industry for demand-response transportation, although the usefulness of some of these measures to a contracting agency may vary. The specific measures used to evaluate the Department’s program should be developed based on its program goals and objectives with input from relevant stakeholders. Given the increasing costs of providing transportation services, the Department should develop and implement a performance measurement system in conjunction with revising and rebidding its transportation contracts, which expire in October 2014. Specifically, the Department should: • Review its existing performance measures and assess whether additional measures could be useful for managing and evaluating its transportation services; Ensure that such performance measures are clearly defined with regard to what is • being measured and the data that will be used for the measure; • Include performance data-reporting requirements and guidelines in its contracts; • Implement policies and procedures for using performance data to evaluate the transportation services and consider actions for improved performance; and • Ensure that the measurement system provides the necessary data to evaluate the appropriateness and cost-effectiveness of contracting for these services. Recommendations The Department should: Develop and implement a performance measurement system to better manage and evaluate its transportation services; and Ensure that the measurement system provides the necessary data to evaluate the appropriateness and cost-effectiveness of contracting for these services. • • Department should increase safeguards to reduce the risk of improper payments The Department has implemented some safeguards to help ensure proper payments for transportation services. These safeguards include: • Segregating the authorization and approval of invoices from the processing of payments for invoiced services; • Using an automated system to ensure that payments are for services and at rates specified in contracts and that invoices are not duplicated; A • post-payment comparison of payment amounts to invoice amounts by a person not involved in processing; and • A post-payment review process that verifies on a sample basis that billed services are supported. However, the Department lacks some standard controls that would help ensure that provider payments are made only for valid services. Specifically: • Written policies and procedures—Although the Department has some written instructions, department • • staff reported that it lacks written policies and procedures for processing transportation payments, which increases the risk of improper or inconsistent payment processing. Prepayment review—The State of Arizona Accounting Manual requires that invoices be reviewed and approved before payment. However, department staff do not adequately review invoiced services before they are paid such as verifying transportation dates, mileage, supporting mileage logs, and transportation requests. Our review of payment transactions showed that staff at one payment processing unit performed some of these types of verifications, while staff at another unit did not. Although the Department performs some post-payment reviews, which can help detect improper payments, these reviews do not prevent improper payments from occurring. In addition, only a few transportation providers are selected for postpayment review. Supervisory review—According to department staff, supervisors do not review and approve transportation payments unless the invoice appears to be a duplicate. However, supervisory review is a standard procedure that helps to ensure compliance with policies and procedures, detect errors, and further ensure payments are made only for valid services. Recommendation The Department should develop and implement formal written payment policies and procedures that address the Department’s current procedures and that require some level of prepayment review and supervisory review based on the Department’s available resources. Arizona Department of Economic Security—Children Support Services— Transportation Services A copy of the full report is available at: www.azauditor.gov REPORT HIGHLIGHTS Contact person: March 2014 • Report No. 14-101 Jeremy Weber (602) 553-0333 PERFORMANCE AUDIT TABLE OF CONTENTS Introduction Chapter 1: Department should use performance measurement to manage transportation services 1 5 Performance measurement an essential management tool 5 Department does not use performance measurement to manage transportation services 6 Recommendations 8 Chapter 2: Department should increase safeguards to reduce the risk of improper payments 11 Department has implemented some safeguards for processing transportation payments 11 Additional steps needed to address deficiencies in internal controls 12 Recommendations 14 Appendix A: Methodology a-1 Agency Response continued Office of the Auditor General page i TABLE OF CONTENTS Tables 1 2 concluded State of Arizona page ii Transportation service payments for children support services Fiscal years 2011 through 2013 (Unaudited) 2 Contracted providers’ service areas, sample rates and fees, and total payments received Fiscal year 2013 (Unaudited) 3 INTRODUCTION Scope and Objectives The Office of the Auditor General has completed a special report of the Arizona Department of Economic Security’s (Department) provision of transportation services for children support services. This report is the second in a series of three reports required by Laws 2013, 1st S.S., Ch. 10, §33, related to the Department’s expenditures for children support services. As required by law, this report addresses (1) transportation expenditures for children support services by type of funded service (see Introduction, pages 1 through 4), and (2) best performance measures to evaluate the effectiveness of these services (see Chapter 1, pages 5 through 9). This report also addresses the Department’s internal controls for processing transportation payments (see Chapter 2, pages 11 through 14). The first report addressed the Department’s contracts for foster home recruitmentrelated services (see Report No. CPS-1301), and the third report will address emergency and residential placements of children. This audit was conducted under the authority vested in the Auditor General by Arizona Revised Statutes §41-1279.03. Transportation services Department provides transportation services The Department provides transportation services to children and families (clients) in the child protective services (CPS) system. According to department staff, these services include transporting clients to and from family visits, medical/counseling appointments, school, daycare, or other appointments, as court-ordered or determined in clients’ case plans. The Department also provides bus passes to clients and purchases car seats for department staff to transport children. The Department transports clients using contracted transportation providers, department staff, and foster parents. In addition, the Department reported that it may use noncontracted providers as a last resource if contracted providers are unavailable. Clients may be transported by themselves or with department staff or an adult escort. As of October 2013, the Department contracted with 12 providers to transport clients within various areas of the State specified in their contracts.1,2 Contracted providers are required to meet several requirements, including completing background checks through the Central Registry and either submitting their fingerprints to the Department for criminal records checks or obtaining a fingerprint clearance card.3 These checks help ensure that individuals transporting clients have not been identified as the subject of a substantiated report of child abuse or neglect and have not committed certain criminal offenses. Drivers are also responsible for ensuring clients are safely and properly transported, which includes escorting children to their final destination and obtaining a signature from the receiving party. Requests for transportation by contracted providers are typically initiated by CPS case managers. Once a need for transportation is identified, a case manager will select a contracted provider and submit a request directly to the provider. Although the Department does not have formal guidance for selecting a contracted provider, some department staff indicated that case managers tend to select providers based on prior experience. The requests include the information necessary to complete the transport, such as the client name, the date and time of the trip, and pickup and drop-off locations. Upon receipt of the transportation request, the provider uses this information to schedule and perform the trip. Providers are required to submit invoices for completed trips 1 The contracts include transportation services for clients in various department divisions and not just CPS clients. 2 The Department also has an intergovernmental agreement with the Northern Arizona Intergovernmental Public Transportation Authority to purchase bus passes for clients in the northern part of the State. 3 The Central Registry is a confidential database of substantiated reports of child abuse and neglect. Office of the Auditor General page 1 on a monthly basis. Payments to providers are processed by various department staff throughout the State, but most invoices are processed by the Department’s payment processing units located in Phoenix and Tucson.1 Department paid at least $14.6 million for transportation services in fiscal year 2013 Although some of the Department’s transportation costs for children support services are embedded within other department expenses (see discussion on page 4), it paid providers approximately $14.6 million for transportation costs in fiscal year 2013. As shown in Table 1, the majority of these costs were for contracted taxi and shuttle services. The costs also included payments for ambulance transportation, reimbursements to noncontracted providers and foster parents for transporting clients, and bus pass and car seat purchases. Transportation costs increased by nearly $6.7 million Table 1: Transportation service payments from fiscal year 2011 to fiscal year 2013. According for children support services to department staff, a major factor in the increase in Fiscal years 2011 through 2013 transportation costs is the growth in the number of (Unaudited) children in out-of-home care, which increased from 10,974 children as of June 2011 to 14,494 children Payment type 2011 2012 2013 as of June 2013. Department staff also reported that increased court-ordered supervised visitation Transport1 $7,205,459 $ 9,984,388 $13,459,321 requirements for children in CPS care contributed to Ambulance2 437,088 638,253 774,028 the transportation cost increases. Bus pass purchases3 Reimbursements 4 Car seat purchases5 Total payments 167,035 160,862 261,504 93,424 101,371 97,027 18,005 21,358 3,916 $7,921,011 $10,906,232 $14,595,796 1 Primarily payments for contracted taxi and shuttle services. 2 Payments to providers for ambulance transport. 3 Bus passes purchased in bulk and distributed through the local CPS offices. 4 Reimbursements for mileage, meals, and lodging to noncontracted providers and foster care parents. 5 Payments for car seats purchased for department staff to transport children. Source: Auditor General staff analysis of the Children’s Information Library and Data Source Report 87 for fiscal years 2011 through 2013. Although the expenditures reported in Table 1 included payments to both contracted and noncontracted providers, 96 percent or more of the total expenditures were paid to contracted providers in those years. As shown in Table 2 (see page 3), contracted provider rates and charges varied by contract, but all providers charged a mileage rate, and many providers charged a flag-drop or no-show fee. For example, the mileage rates for nonwheelchair transportation ranged from $1.55 to $3.90 per mile.2 Variation in these rates may be explained by differences in contractors’ vehicles, operating costs, and levels of service. Auditors contacted child protective agencies in other western states to determine how the Department’s 1 In addition, transportation claims for the northern part of the State are processed by department staff in that area. On occasion, some special transportation invoices, such as transportation reimbursement provided as part of court-ordered visitation, are processed by other administrative staff. 2 Department staff reported that contracted providers were allowed to offer rates that fell within a range that the Department established. State of Arizona page 2 Office of the Auditor General page 3 Gila, Maricopa, Pinal Provider 11 Provider 12 3.90 2.65 2.00 3.90 2.95 2.40 1.55 3.05 4.00 5.50 7.00 10.00 n/a n/a n/a n/a 7.50 n/a n/a $ 5.00 Nonwheelchair 28.50 35.00 28.00 10.00 n/a n/a n/a 30.00 25.00 n/a 29.95 $14.00 Wheelchair Flag-drop fee1 n/a n/a 28.00 n/a 20.00 20.00 n/a 15.00 25.00 10.00 $34.50 n/a - 4,498 19,589 50,526 209,081 247,530 1,213,775 1,417,555 1,839,966 2,299,113 3,017,201 $3,888,659 Total paid to provider Source: Auditor General staff analysis of transportation contract price sheets and the Children’s Information Library and Data Source Report 87 for fiscal year 2013. 4 This provider did not charge a flag-drop or no-show fee but charged a minimum-trip fee that ranged from $10 to $60 depending on the area of the State. 3 Rates and fees listed in the table apply only to Maricopa County and vary outside of this county. appointment time. 2 Fee charged when a client is not at the designated pickup location at the confirmed date and time or if an appointment is canceled less than 2 hours prior to the scheduled n/a n/a 7.00 n/a 20.00 20.00 n/a 15.00 25.00 10.00 $15.00 n/a Wheelchair No-show fee2 Nonwheelchair 1 Fee charged for the start of each trip after client(s) are picked up to be transported to the approved and referred destination. Provider 10 Maricopa, Yuma Apache, Coconino, Mohave, Navajo, Yavapai Maricopa, Pima Provider 9 1.55 2.98 Graham, Greenlee, Pima Provider 8 1.80 2.10 2.50 3.70 2.10 2.48 1.80 2.35 $2.75 Wheelchair 2.48 1.80 1.90 $1.96 1.80 Maricopa, Pima, Pinal Maricopa, Pima, Pinal, Yavapai Cochise, Maricopa, Pima, Pinal All counties All counties All counties Nonwheelchair Mohave 4 3 Arizona counties served Mileage rate Contracted providers’ service areas, sample rates and fees, and total payments received Fiscal year 2013 (Unaudited) Provider 7 Provider 6 Provider 5 Provider 4 Provider 3 Provider 2 Provider 1 Table 2: contracted transportation rates compared to those in other states. Of the five states that responded, four states reported that they do not contract for transportation services for their child protective services clients.1 The fifth state, Washington, reported that it contracts with individuals to supervise parent-child visitation, and these contracts include reimbursement for transporting clients to and from visits.2 States reported that they generally provide transportation through some combination of case workers, state workers, volunteers, and/or foster parents. The Department incurs other transportation costs for children support services in addition to the amounts paid to providers, but these costs are embedded within other department expenses. For example, department staff reported that the Department incurs transportation costs as part of foster home recruitment-related services, in-home services, parent-aide services, and when CPS staff transport clients as part of children support services.3 However, department staff reported that these costs are incorporated within the payments for these specific services and department operating costs, respectively. There are also indirect transportation costs for administrative support, such as department staff who coordinate transportation requests or process transportation invoices. As a result, auditors were not able to determine the extent to which the Department’s total transportation costs as part of children support services exceeded the amounts reported in Table 1 (see page 2). Auditors will continue to evaluate the Department’s children support services expenditures when conducting audit work on emergency and residential placements of children, and, in conjunction with this additional audit work, assess whether the Department could implement a cost-effective method to more fully track its costs. 1 These states were Nevada, New Mexico, Texas, Utah, and Washington. 2 Washington pays a fixed rate for this service of $24.49 per hour plus $0.56 per mile for travel in excess of 60 miles. 3 In-home services providers assist in supporting and enhancing the family unit, and preserving and reunifying the family. Parent-aide services provide a range of support services, instruction, and assistance to parents or caregivers to improve their skills and ability to fulfill parenting roles and responsibilities. State of Arizona page 4 CHAPTER 1 The Arizona Department of Economic Security (Department) should develop and implement a performance measurement system to better manage its transportation services. Performance measurement is an essential tool for improving the quality of transportation services and managing costs, and auditors identified several common performance measures used by the transit industry that the Department could potentially use to evaluate its transportation services. However, the Department does not use performance measurement to manage these services, such as using performance measures to evaluate its overall provision of transportation services or to monitor contracted transportation providers’ performance. Given the increasing costs of providing transportation services, which totaled at least $14.6 million in fiscal year 2013, the Department should develop and implement a performance measurement system in conjunction with its plans to revise and rebid its transportation contracts, which expire in October 2014. The Department should also ensure that this measurement system provides the necessary data to evaluate the appropriateness and costeffectiveness of contracting for these services. Department should use performance measurement to manage transportation services Performance measurement an essential management tool Performance measurement is an essential tool for managing the quality and costs of transportation services. An effective performance measurement system collects and assesses performance data and compares results against a performance standard, such as past performance or other providers’ performance. Performance measures should be clearly defined with regard to what is being measured, the data sources that will be used, and how often data will be collected.1 Data should be collected and used at regular intervals to assess performance, measure progress toward achieving program goals and objectives, and consider actions, such as policy or operational changes, for improved performance.2,3 Performance data thus can be used to improve program management, increase cost-effectiveness, and support requests for future funding.4 The Transit Cooperative Research Program (TCRP) identified several common performance measures used by the transit industry for demand-response transportation (see textbox, page 6).5,6 These measures can be used for a variety of management purposes. For example, measuring the ratio of on-time pickups to total pickups could be useful to a contracting agency for evaluating the overall timeliness of contracted transportation services.7 However, the usefulness of these performance measures to an agency that contracts for transportation services may vary. One expert auditors interviewed indicated that, because of potential variability among providers, measuring contracted 1 State of Arizona, Governor’s Office of Strategic Planning and Budgeting. (2011). 2011 Managing for results handbook. Phoenix, AZ: Author. 2 State of Arizona, Governor’s Office of Strategic Planning and Budgeting, 2011 3 Burkhardt, J.E., Garrity, R., McGehee, K., Hamme, S.S., Burkhardt, K., Johnson, C., & Koffman, D. (2011). Sharing the costs of human services transportation; Volume 2: Research report (TCRP Report 144). Washington, DC: Transportation Research Board. 4 Burkhardt et al., 2011 5 The TCRP provides a continuing program of applied research on transit issues. It is sponsored by the Federal Transit Administration. 6 As defined in TCRP Report 124, “demand-response is a transit mode comprised of passenger cars, vans, or small buses operating in response to calls from passengers or their agents to the transit operator, who then dispatches a vehicle to pick up the passengers and transport them to their destinations. A demand-response operation is characterized by the following: (a) the vehicles do not operate on a fixed route or schedule, and (b) the vehicle may pick up several passengers at different pickup points before taking them to their respective destinations.” 7 KFH Group, Inc., Urbitran Associates, Inc., McCollom Management Consulting, Inc., & Cambridge Systematics, Inc. (2008). Guidebook for measuring, assessing, and improving performance of demand-response transportation (TCRP Report 124). Washington, DC: Transportation Research Board. Office of the Auditor General page 5 providers’ operating cost per mile may be less useful for monitoring their performance than measures that focus on quality, such as safety and timeliness. Additionally, measures used to evaluate contractors’ performance should assess factors that are within the contractors’ control. Examples of common performance measures used by the transit industry for demand-response transportation • Accidents/safety incidents per 100,000 vehicle miles • • • • • • • • No-shows per scheduled trips On-time pickups to total pickups The specific performance measures used to evaluate a particular transportation Complaints per 1,000 passenger trips program should be developed based on Average trip length program goals and objectives with input from relevant stakeholders.1 Further, Average vehicle travel time selecting a reasonable number of Response time performance measures is important because gathering and analyzing Trip denials to trips requested performance measurement data is Operating cost per vehicle mile resource-intensive. Several factors should Source: KFH Group, Inc., Urbitran Associates, Inc., McCollom be considered in evaluating potential Management Consulting, Inc., & Cambridge measures, including whether they are Systematics, Inc. (2008). Guidebook for measuring, assessing, and improving performance of demandmeaningful to the program’s goals; valid in response transportation (TCRP Report 124). Washington, that they represent what is being measured; DC: Transportation Research Board. customer-focused; comprehensive in that the set of measures includes all key aspects of program performance; balanced in that several types of measures are included; credible by being based on accurate and reliable data; cost-effective in terms of data collection and processing costs; simple to calculate and interpret; and comparable.2 Department does not use performance measurement to manage transportation services Although performance measurement can help manage the quality and cost of transportation services, the Department has not implemented a performance measurement system for these services. Specifically, department staff reported that they do not use performance measurement to manage and evaluate transportation services, including overseeing contracted transportation providers. For example, although the Department includes performance expectations in its transportation contracts (see textbox, page 7), the contracts do not require providers to report whether these expectations are being met, and providers that auditors contacted indicated that they were not reporting this specific information to the Department. However, the Department requires providers to report some performance information through monthly activity reports, which department staff could use for performance measurement purposes. This information includes the number of 1 State of Arizona, Governor’s Office of Strategic Planning and Budgeting, 2011 2 State of Arizona, Governor’s Office of Strategic Planning and Budgeting, 2011 State of Arizona page 6 trips per client, number of trip cancellations made by the contracted provider, and number of late pickups made by the provider. As of October 2013, department staff reported that this information was used only for billing purposes and not for contractor oversight or other management purposes, such as evaluating whether the Department efficiently or effectively uses its contracted providers.1 Further, department staff reported that compiling and analyzing even the limited data included in the monthly activity reports would be time-consuming and resource-intensive because this information is not reported electronically. Given the increasing costs for providing transportation services, which totaled at least $14.6 million in fiscal year 2013, the Department should develop and implement a performance measurement system to better manage and evaluate the transportation services that are provided. This performance measurement system should be developed in conjunction with the Department’s plans to develop new transportation contracts. The Department reported that it plans to revise and rebid its transportation service contracts, which have been extended through October 2014 in order to consider this report’s recommendations. Specifically, the Department should: Department performance expectations for contracted transportation providers Outcome Goals • Ninety percent of all individuals shall be transported within 15 minutes of scheduled pickup and delivery. • Ninety-five percent of all referrals, with exception of same-day requests, will be accepted, scheduled, and completed unless the contractor demonstrates all service slots have been previously scheduled. Performance Standards • Ensure that there are no more than 5 percent late trips per month. • Ensure that there are no more than 5 percent driver no-shows per month. A driver no-show for an authorized written referral from the Department counts as a late trip. • Ensure that the performance standard is met for 2 months in a row or any 3 months total during any contract term. Failure to attain the standards may be cause for cancellation of the contract. • Ensure that all complaints are forwarded to department staff for resolution. • Source: Auditor General staff review of the Department’s transportation Review its existing performance measures for service contracts. transportation services—The Department should review its existing performance measures and assess whether additional measures could be useful for managing and evaluating its transportation services, including overseeing the contracted providers. Selected measures should reflect key management goals and objectives for these services. In addition, the measures should be meaningful, valid, customer-focused, comprehensive, balanced, credible, cost-effective, simple, and comparable. The Department could select measures from the list of common performance measures identified by the TCRP (see textbox, page 6) and/or develop other measures, as appropriate. In addition, the Department should solicit input from stakeholders, as appropriate. For example, it should solicit input from providers in developing measures related to contract oversight or that require performance data to be reported by providers (see next bullet). • Clearly define performance measurements—Any performance measures developed should be clearly defined with regard to what is being measured and what data will be used for the measure. If contractors will be required to provide this data, those requirements should be 1 The Department reported that, beginning in January 2014, it has expanded the scope of its post-payment reviews of transportation providers to include an assessment of whether the providers have met some of the performance expectations outlined in its transportation contracts. However, post-payment reviews of transportation providers are not routinely conducted (see Chapter 2, page 12, for additional information about post-payment reviews). Office of the Auditor General page 7 included in the transportation contracts. The contracts should also include guidelines for reporting the data, such as developing and requiring the use of templates, electronic submission of data, or other standardized reporting methods. • Develop policies and procedures for using performance measurement data—The Department also should develop and implement policies and procedures for using performance measurement data to evaluate the transportation services and consider actions, such as policy or operational changes, for improved performance. Further, once this performance measurement system has been implemented, the Department should use the performance data it collects to help assess whether contracting is the most appropriate and cost-effective means of providing transportation services in the future. Although the scope of this report did not include assessing whether contracting was the most appropriate means of providing these services, other western states that responded to auditors’ inquiries generally reported that they do not use contractors to transport their child protective services clients (see Introduction, page 4). Therefore, in developing a performance measurement system for its transportation services, the Department should ensure that the measurement system provides the necessary data to evaluate the appropriateness and cost-effectiveness of contracting for these services. For example, collecting data to analyze the average cost per trip for using contractors versus department staff or foster parents might be helpful in determining the most cost-effective option. Recommendations: 1. State of Arizona page 8 In conjunction with its plans to develop new transportation contracts, the Department should develop and implement a performance measurement system to better manage and evaluate the provided transportation services. Specifically, the Department should: a. Review its existing performance measures and assess whether additional measures could be useful for managing and evaluating its transportation services, including overseeing the contracted providers. Selected measures should reflect key management goals and objectives for these services. In addition, the measures should be meaningful, valid, customer-focused, comprehensive, balanced, credible, cost-effective, simple, and comparable. The Department should solicit input from stakeholders, such as providers, in developing these performance measures, as appropriate. b. Clearly define selected measures with regard to what is being measured and what data will be used for the measure. If contractors will be required to provide this data, those requirements should be included in the transportation contracts. The contracts should also include guidelines for reporting the data, such as developing and requiring the use of templates, electronic submission of data, or other standardized reporting methods. c. 2. Develop and implement policies and procedures for using performance measurement data to evaluate the transportation services and consider actions, such as policy or operational changes, for improved performance. As part of developing a performance measurement system for its transportation services, the Department should ensure that the measurement system provides the necessary data to evaluate the appropriateness and cost-effectiveness of contracting for these services. Office of the Auditor General page 9 State of Arizona page 10 CHAPTER 2 Although the Arizona Department of Economic Security (Department) has implemented some safeguards to ensure proper payments to both contracted and noncontracted transportation providers, it should increase its safeguards to reduce the risk of improper payments. Specifically, the Department should develop and implement formal written policies and procedures for payment processing. Additionally, based on the Department’s level of available resources, these policies and procedures should require (1) some level of prepayment verification of invoices to ensure payment for only valid services, and (2) supervisory review, using a random or risk-based approach, to help ensure compliance with policies and procedures and to detect processing errors. Department should increase safeguards to reduce the risk of improper payments Department has implemented some safeguards for processing transportation payments Adequate internal controls help to ensure the proper disbursement of public funds, such as preventing duplicate payments and payments for unallowable costs. The Department has implemented some safeguards to help ensure proper payments to transportation providers. Although these safeguards do not address all aspects of ensuring the appropriateness and accuracy of invoiced services, they help to segregate functions that should be performed separately, identify duplicate invoice payments, ensure payments are made based on contract terms, and verify the validity of some service payments that have already been made. Specifically: • Segregation of duties—The Department has appropriately segregated payment processing duties. Specifically, different staff are responsible for (1) authorizing and approving the services for which providers can be paid and (2) processing provider payments. Segregation of duties is necessary to help ensure an adequate internal control environment and prevent any one employee from having control over an entire transaction. • Logging invoices—The Phoenix and Tucson payment processing units reported that they have procedures to log and date stamp each provider invoice received prior to assigning it for processing. Department staff at the Phoenix unit reported that this procedure helps staff track invoices, gauge providers’ compliance with submitting timely invoices, and identify duplicate invoices. For example, the Phoenix unit invoice log tracks the provider name, service month, invoiced amount, date received, and payment status. Keeping a log of the invoices’ status helps to reduce the likelihood of duplicate invoices being paid. However, these procedures would not detect duplicate services billed on more than one invoice. • Automated system checks—The Children’s Information Library and Data Source (CHILDS) system, which is used by department staff to process payments, including transportation payments, has automated system checks to (1) ensure that providers receive payment only for services and at rates specified in their contracts, and (2) suspend any potentially duplicate provider invoice payments for further supervisory review. Supervisors and other specific staff are responsible for investigating and clearing suspended payments in CHILDS, as appropriate. Auditors conducted limited test work and examined supporting documentation for 17 payment transactions and did not identify any duplicate payments. Office of the Auditor General page 11 • Final warrant reviews—According to department staff, a designated staff person, separate from staff processing the invoices, is responsible for comparing the warrants issued to providers to the warrant register and the providers’ invoices.1 However, department staff reported that this review is only done after the payment processing staff have completed payment. This control helps to detect processing errors and identify potential overpayments due to department staff error. • Post-payment reviews—Based on recommendations from a March 2011 Auditor General report (Report No. CPS-1101), the Department has established a control to perform postpayment reviews. Post-payment reviews are performed by department internal audit staff and generally entail the review of several invoices from a specific provider to identify duplicate service payments and verify that billed services are supported. This type of review helps ensure that the Department has paid for valid transportation claims. According to department staff, all providers paid through CHILDS are subject to post-payment reviews, such as providers of parent-aide, foster home recruitment, and group care facility services. Staff reported that providers are selected using a methodology that considers the type of service, total payment amount, and other risk factors. Under this methodology, however, transportation providers have not been frequently selected for review. Specifically, as of February 2014, only 2 of the 34 post-payment reviews the Department completed have involved a transportation provider. The first review was completed in September 2011 and identified an overpayment of $6,043, approximately 1.7 percent of total payment amounts tested, made to the provider because the provider invoiced more miles than could be verified. The second review, completed in February 2014, also identified an overpayment of $2,350, or 0.3 percent of the total payment amounts tested, because the provider billed for more miles than driven. In addition to reviewing payments, the February 2014 post-payment review included an assessment of whether the provider met the two performance outcome goals outlined in the transportation contract and found the provider to be in compliance. Department staff reported that they plan to include performance assessment in future post-payment reviews. As of February 2014, the Department was in the process of reviewing an additional transportation provider and planned to review four other transportation providers, although it had not yet determined a time frame for conducting these reviews. Additional steps needed to address deficiencies in internal controls Despite these safeguards, the Department lacks some standard controls that would help ensure that transportation provider payments are made only for valid services. These deficiencies increase the risk for improper payments, and the Department should add additional safeguards to improve its payment-processing internal controls. Specifically: • Written policies and procedures for greater consistency in payment processing—Although the Department has some written instructions for processing transportation invoices,department staff reported that it lacks formal written policies and procedures. Not having written policies 1 A warrant is a written order directing payment of a specified amount to a designated payee. State of Arizona page 12 and procedures increases the risk of improper or inconsistent payment processing. For example, the Department has standard procedures for correcting overpayments based on improper billing, but staff indicated that it does not have standard procedures for correcting errors in payments to vendors that are made by department staff. Department staff reported that they address these types of errors either by requiring a provider to remit the overpaid amount to the Department or giving the provider the option to have overpayments reduced from future invoices. However, neither method for addressing this type of error is reflected in written policies or procedures. • Requiring prepayment review to ensure accuracy and appropriateness—Department staff do not adequately verify the accuracy and appropriateness of invoiced services before payment is made, such as verifying that transportation dates and invoice calculations are accurate; trip distances are supported by mileage logs or mapping software; and services charged were actually requested by case managers and provided to department clients. Specifically, auditors’ review of payment transactions from one payment processing unit showed that staff verify some information, including transportation dates, mileage, and total calculations prior to payment, and this unit’s payment supervisor reported that these verifications are performed on a random basis. However, staff from a different payment processing unit did not perform these types of verifications based on auditors’ review of payment transactions and an observation of one payment processing staff. Verifying invoiced services is important for ensuring proper payments. For example, some department staff stated they have identified instances of providers billing for nonreferred clients through prepayment verifications. Additionally, as mentioned previously, the Department’s two completed post-payment reviews of transportation providers identified overpayments because the providers had billed and been paid for more miles than were supported. Although the Department’s post-payment review process has helped identify improper payments, it does not prevent improper payments from occurring, and as discussed previously, post-payment reviews have not been frequently conducted for transportation providers. Further, the State of Arizona Accounting Manual requires that invoices be reviewed and approved prior to payment. It also states that, for invoiced services, there should be an approval and verification that services were rendered by an authorized person and calculations on the invoice should be reviewed. Department staff indicated that prepayment verification of every invoice might not be feasible at current staffing levels and due to the manual nature of the payment process. Still, department staff should conduct some level of prepayment review until additional staff resources and/or technology allow the Department to implement a more comprehensive prepayment review process. For example, department staff could review invoice totals and verify a small sample of invoiced services prior to payment, either randomly or using a risk-based approach. • Requiring supervisory review of payments—In addition, department staff reported that supervisors do not review and approve transportation payments unless the invoice appears to be duplicative, as discussed previously. However, supervisory review is a standard procedure intended to help ensure staff compliance with policies and procedures, detect processing errors, and further ensure that payments are made only for valid services. For example, auditors noted an instance where the amount recorded and paid to a provider was more than the billed invoice amount due to department staff error. Although this error was detected by the final warrant review, which occurred after the payment was sent to the provider, supervisory review Office of the Auditor General page 13 and approval could have detected the error before payment was completed. In addition, the State of Arizona Accounting Manual states that invoices should include a final approval for payment, and there should be some indication on the invoice that all the checks and/or approvals were made. Department staff indicated that requiring supervisory review prior to payment would affect the Department’s ability to process payments in a timely manner based on its available staffing levels. However, supervisory review can include a higher level, lessdetailed review, such as ensuring that key payment information (e.g., service types, dates, and amount) entered by staff into CHILDS matches the invoices prior to payment. Implementing adequate safeguards to ensure proper payment to providers is especially important given the increasing costs of providing transportation services. Therefore, the Department should develop and implement formal written payment policies and procedures to ensure consistency across the different payment units. In addition to addressing current procedures and controls, these policies and procedures should require prepayment review and verification of invoiced services, such as ensuring that transportation dates and invoice calculations are accurate; trip distances are supported by mileage logs or mapping software; and services charged were actually requested by case managers and provided to department clients. If verifying all invoiced services prior to payment is not feasible due to the Department’s available staff resources and technology, the Department should require department staff to conduct some level of prepayment review until additional staff resources and/or technology allow the Department to implement a more comprehensive prepayment review process. Further, the policies and procedures should require some level of supervisory review prior to payment, using a random or risk-based approach, to help ensure compliance with policies and procedures and detect processing errors. The Department should then expand its supervisory review of payments as resources allow. Recommendations: 1. State of Arizona page 14 The Department should develop and implement formal written payment policies and procedures to ensure consistency across the different payment units. These policies and procedures should: a. Address the Department’s current procedures and controls; b. Require prepayment review and verification of invoiced services, such as ensuring that transportation dates and invoice calculations are accurate; trip distances are supported by mileage logs or mapping software; and services charged were actually requested by case managers and provided to department clients. If verifying all invoiced services prior to payment is not feasible due to the Department’s available staff resources and technology, the Department should require department staff to conduct some level of prepayment review until additional staff resources and/or technology allow the Department to implement a more comprehensive prepayment review process; and c. Require some level of supervisory review prior to payment, using a random or risk-based approach, to help ensure compliance with policies and procedures and detect processing errors. The Department should then expand its supervisory review of payments as resources allow. APPENDIX A This appendix provides information on the methods auditors used to meet the report objectives. The Auditor General and staff express appreciation to the Arizona Department of Economic Security (Department) Director and staff for their cooperation and assistance throughout the engagement. Methodology Auditors used the following methods to meet the report objectives: • Auditors interviewed department officials and staff and reviewed all of the Department’s transportation services contracts, including contractor price sheets; applicable policies and procedures; and other department information. In addition, auditors performed an observation of payment claims and examined supporting documentation for two payment transactions. • Auditors obtained and analyzed financial expenditure information for transportation services from the Department’s Children’s Information Library and Data Source Report 87 for fiscal years 2011 through 2013. Additionally, auditors analyzed this information to identify potential duplicate payments and examined supporting documentation for 17 of those payment transactions. • Auditors reviewed literature on performance measurement for transportation services, including reports from the Transit Cooperative Research Program.1 In addition, auditors contacted experts on contracted transportation services from the Georgia Institute of Technology and KFH Group, Inc. to obtain information related to performance measurement for transportation services. • Auditors interviewed four of the Department’s contracted transportation providers regarding the transportation services contracts. • Auditors interviewed child protective services agency administrative and contract staff in five western states (Nevada, New Mexico, Texas, Utah, and Washington) to obtain information about whether they contract for transportation services and to review their contracted transportation rates. 1 The Transit Cooperative Research Program provides a continuing program of applied research on transit issues. It is sponsored by the Federal Transit Administration. Office of the Auditor General page a-1 AGENCY RESPONSE AGENCY RESPONSE Division of Child Safety and Family Services Janice K. Brewer Governor Charles Flanagan Director March 13, 2014 Ms. Debra K. Davenport Office of the Auditor General 2910 North 44th Street, Suite 410 Phoenix, Arizona 85018 Re: Auditor General Transportation Services Audit Response Dear Ms. Davenport: The Division of Child Safety and Family Services (Division) and the Arizona Department of Economic Security (Department) appreciate the opportunity to provide this response to the Auditor General's report on Transportation Services. The Department values the collaborative effort of the Auditor General's staff throughout this audit. In general, the information presented is constructive in helping the Division of Child Safety and Family Services develop stronger contract terms and internal controls related to transportation services. The report notes the Department has established performance measures for contracted transportation services. The Department agrees that collecting data for the performance measures and holding contractors accountable is needed to adequately manage transportation services. Currently, there are over 15,000 children in out of home care in Arizona. The Division workforce must be able to access high quality transportation services in a timely manner in order to meet the needs of the children and families it serves. Transportation is a critical service that helps improve outcomes for children and families. The auditors identified two areas where the Department could improve transportation services. The Department has addressed the auditors' recommendations below: RECOMMENDATIONS – CHAPTER 1 1. In conjunction with its plan to develop new transportation contracts, the Department should develop and implement a performance measurement system to better manage and evaluate the provided transportation services. Specifically, the Department should: 1717 W. Jefferson, S/C 005, Phoenix, AZ 85007 Telephone (602) 542-5844 Auditor General Transportation Services Audit Response Page 2 a. Review its existing performance measures and assess whether additional measures could be useful for managing and evaluating its transportation services, including overseeing the contracted providers. Selected measures should reflect key management goals and objectives for these services. In addition, the measures should be meaningful, valid, customer-focused, comprehensive, balanced, credible, cost-effective, simple, and comparable. The Department should solicit input from stakeholders, such as providers, in developing these performance measures, as appropriate. b. Clearly define selected measures with regard to what is being measured and what data will be used for the measure. If contractors will be required to provide this data, those requirements should be included in the transportation contracts. The contractors should also include guidelines for reporting the data, such as developing and requiring the use of templates, electronic submission of data, or other standardized reporting methods. c. Develop and implement policies and procedures for using performance measurement data to evaluate the transportation services and consider actions, such as policy or operational changes, for improved performance. 2. As part of developing a performance measurement system for its transportation services, the Department should ensure that the measurement system provides the necessary data to evaluate the appropriateness and cost effectiveness of contracting for these services. Response: The finding of the Auditor General is agreed to and the recommendations will be implemented. Comment: As noted by the Auditor General, the Division currently maintains performance measures for contracted transportation providers. The Division will continue to evaluate all performance measures prior to issuing any future contract solicitations for transportation services. Prior to this audit, the Division began engaging stakeholders to gather information for other service solicitations. The Division has contacted with other states to begin assessing current performance measures. The Division will continue to seek input from stakeholders, as allowed by procurement rules and regulations, to ensure that all methods used to collect data result in the fair treatment of present and future contractors. Currently, auditors review contractors' for compliance with contracted performance measures. For the most recently issued audit, Department's auditors determined the contracted providers exceeded the required measures that are currently in the contract. The Division will improve the management of contracted transportation services by implementing a system that will include holding the Division and contractors accountable for quality and timely delivery of services. The system will include collecting Auditor General Transportation Services Audit Response Page 3 performance measurement data, analyzing the data collected, validating the accuracy of the data, and taking corrective actions when needed. In addition, the Division will develop policies and procedures for on-going desk reviews of performance measurement data. The Division recognizes transported clients face varied circumstances that may negatively impact some of the required performance measures. As a result, the Division must ensure that exceptions are considered when necessary to ensure all of our clients receive the essential services to help facilitate reunification or permanency. RECOMMENDATIONS - CHAPTER 2 The Department should develop and implement formal written policies and procedures to ensure consistency across the different payment units. These policies and procedures should: a. Address the Department's current procedures and controls; b. Require prepayment reviews and verification of invoiced services, such as ensuring that transportation dates and invoice calculations are accurate; trip distances are supported by mileage logs or mapping software; and services charged were actually requested by case managers and provided to department clients. If verifying all invoiced services prior to payment is not feasible, the Department should require department staff to conduct some level of prepayment review until additional staff resources and/or technology allows the Department to implement a more comprehensive prepayment review process; and c. Require some level of supervisory review prior to payment, using a random risk-based approach, to help ensure compliance with policies and procedures and detect processing errors. The Department should then expand its supervisory review of payments as resources allow. Response: The finding of the Auditor General is agreed to and a different method of dealing with the finding will be implemented. Comment: The Division strongly agrees with the need to have adequate internal controls to help ensure public funds are properly expended. As recommended, the Division will develop and implement formal written policies and procedures to ensure consistency across different payment units. As noted by the auditors, the Division currently maintains the following internal controls when processing payments:  The Division's Children's Information Library and Data Source (CHILDS) system is used to process payments. CHILDS includes system edits to ensure the Auditor General Transportation Services Audit Response Page 4   contractor has an active contract, the accurate rate was billed, and the service was contracted in the billed region. CHILDS suspends payment of claims that appear duplicated which then requires a second review before payment is processed. Contractors can only bill up to a maximum ‘allowable’ for miles entered into CHILDS. While the extent of reviews varies amongst payment processors, invoices are reviewed prior to payment. The reviews include verifying contract numbers, invoice totals, and in some instances, summing miles listed in contractors' reports to ensure the totals agree to invoiced amounts. As noted in the first bullet, CHILDS suspends payments that appear duplicated. A second review of high risk payments is required prior to releasing the payment. Instead of conducting a detailed pre-payment review and increasing the number of supervisory reviews with every billing line and document, the Division will reallocate audit resources to increase the number of post-payment reviews of transportation services. The Division is committed to creating an agency focused on its core mission of child safety. The Division will revisit the recommendation to add additional staff for the processing of payments after addressing the need for additional staff whose purpose is to protect the children of Arizona. Furthermore, this recommendation will be revisited only after the Division's audits indicate that an issue has been identified. The auditors did not identify overpayments. The Department's audit team identified overpayments that were 1.7 and 0.3 percent of the amount paid to the respective contractors. A risk-based audit methodology is used to determine the contractors to be audited. The risk factors used when developing a risk-based audit plan include the fiscal impact of the contracted service, likelihood of overpayments and complexity of the service. This recommended audit methodology resulted in a limited selection of transportation audits. However, the Division will reallocate existing audit resources to increase the number of transportation audits. Thank you again for the opportunity to respond. We value the time, effort, and diligence of the Auditor General's staff in producing this report. Sincerely, Charles Flanagan Director Performance Audit Division reports issued within the last 24 months CPS-1201 12-01 12-02 12-03 12-04 12-05 12-06 12-07 13-01 13-02 13-03 13-04 Department of Economic Security—Division of Children, Youth and Families—Child Protective Services—In-Home Services Program Arizona Health Care Cost Containment System— Coordination of Benefits Arizona Health Care Cost Containment System— Medicaid Eligibility Determination Arizona Board of Behavioral Health Examiners Arizona State Parks Board Arizona State Schools for the Deaf and the Blind Arizona Health Care Cost Containment System—Medicaid Fraud and Abuse Prevention, Detection, Investigation, and Recovery Processes Arizona Health Care Cost Containment System— Sunset Factors Department of Environmental Quality— Compliance Management Arizona Board of Appraisal Arizona State Board of Physical Therapy Registrar of Contractors Future Performance Audit Division reports 13-05 13-06 13-07 13-08 13-09 13-10 13-11 13-12 13-13 CPS-1301 13-14 13-15 Arizona Department of Financial Institutions Department of Environmental Quality— Underground Storage Tanks Financial Responsibility Arizona State Board of Pharmacy Water Infrastructure Finance Authority Arizona State Board of Cosmetology Department of Environmental Quality— Sunset Factors Arizona State Board of Funeral Directors and Embalmers Arizona State Board for Charter Schools Arizona Historical Society Arizona Department of Economic Security— Children Support Services—Foster-Home Recruitment-Related Services Contracts Review of Selected State Practices for Information Technology Procurement Arizona Game and Fish Commission, Department, and Director