A REPORT TO THE ARIZONA LEGISLATURE Performance Audit Division Performance Audit and Sunset Review Registrar of Contractors July • 2013 REPORT NO. 13-04 Debra K. Davenport Auditor General The Auditor General is appointed by the Joint Legislative Audit Committee, a bipartisan committee composed of five senators and five representatives. Her mission is to provide independent and impartial information and specific recommendations to improve the operations of state and local government entities. To this end, she provides financial audits and accounting services to the State and political subdivisions, investigates possible misuse of public monies, and conducts performance audits of school districts, state agencies, and the programs they administer. The Joint Legislative Audit Committee Senator Chester Crandell, Chair Representative John Allen, Vice Chair Senator Judy Burges Senator Rich Crandall Senator Steve Gallardo Senator Katie Hobbs Senator Andy Biggs (ex officio) Representative Paul Boyer Representative Andrea Dalessandro Representative Martin Quezada Representative Kelly Townsend Representative Andy Tobin (ex officio) Audit Staff Dale Chapman, Director Shan Hays, Manager and Contact Person Cheya Wilson, Team Leader Katherine Grzybowski Phil Hanus Genny Wilson Copies of the Auditor General’s reports are free. You may request them by contacting us at: Office of the Auditor General 2910 N. 44th Street, Suite 410 • Phoenix, AZ 85018 • (602) 553-0333 Additionally, many of our reports can be found in electronic format at: www.azauditor.gov STATE OF ARIZONA OFFICE OF THE DEBRA K. DAVENPORT, CPA AUDITOR GENERAL MELANIE M. CHESNEY AUDITOR GENERAL DEPUTY AUDITOR GENERAL July 26, 2013 Members of the Arizona Legislature The Honorable Janice K. Brewer, Governor Mr. William A. Mundell, Director Registrar of Contractors Transmitted herewith is a report of the Auditor General, A Performance Audit and Sunset Review of the Registrar of Contractors. This report is in response to an October 26, 2010, resolution of the Joint Legislative Audit Committee. The performance audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes §41-2951 et seq. I am also transmitting within this report a copy of the Report Highlights for this audit to provide a quick summary for your convenience. As outlined in its response, the Registrar of Contractors agrees with all of the findings and plans to implement all of the recommendations. My staff and I will be pleased to discuss or clarify items in the report. Sincerely, Debbie Davenport Auditor General Attachment 2910 NORTH 44 th STREET • SUITE 410 • PHOENIX, ARIZONA 85018 • (602) 553-0333 • FAX (602) 553-0051 Registrar of Contractors REPORT HIGHLIGHTS PERFORMANCE AUDIT Our Conclusion The Registrar of Contractors (ROC) licenses and regulates residential and commercial contractors. The ROC can improve the regulation of contractors and better protect the public by adequately addressing construction complaints and disciplining contractors when warranted. The ROC should promote earlier resolution of construction complaints by monitoring complaints as they move through its complaint-handling process and by adding time frames to this process, and it should ensure that discipline is imposed in a timely manner. Also, regardless of whether the ROC replaces its current data system, the ROC should take key actions to enable its data system to perform agency functions efficiently and effectively. This report also provides information about the Residential Contractors’ Recovery Fund (Recovery Fund). ROC should consistently ensure that complaints are adequately resolved As of August 2012, the ROC licensed almost 40,000 contractors. In fiscal year 2012, the ROC closed 3,597 complaints against licensed contractors without issuing a citation, which is a legal document listing alleged statutory violations; issued 1,498 citations; disciplined 1,139 licenses; and also received 1,566 complaints regarding unlicensed contractors. The ROC also oversees the Recovery Fund, which pays claims of up to $30,000 to homeowners who have been financially harmed by a licensed contractor. The Recovery Fund is financed mainly by a contractor-paid assessment when they obtain or renew their licenses. The ROC resolves complaints through a process that is intended to assist consumers by resolving issues of poor workmanship or abandoned work. However, the process does not consistently protect the public because problems are not always resolved before the ROC closes the complaint. We reviewed ten complaints that were closed in fiscal year 2011 prior to the issuance of a citation and confirmed that the ROC closed six complaints without ensuring workmanship problems had been addressed. For example, in January 2010, the ROC received a complaint regarding a pool where the tile was cracked, the pool leaked, and pipes stuck out too far. The ROC inspected the pool and then directed the contractor to fix the pool in 15 days. The ROC closed the complaint in March 2010 without verifying that the pool was fixed. In December 2010, the homeowner resubmitted the complaint because the problems were not fixed. Finally, in February 2011, the contractor signed a settlement agreement stating that the contractor would repay the complainant nearly $3,000. Recommendation To better protect the public, the ROC should modify its complaint-resolution process to ensure that problems are adequately addressed before closing complaints. ROC should streamline complaint-resolution process 2013 July • Report No. 13-04 Although the complaint-handling process ROC uses allows for fast complaint resolution by providing contractors opportunities to correct problems and not go through any remaining steps in this process once the problem is resolved, the complaint-handling process is lengthy for complaints that receive a citation. This process is especially long if a complaint goes through all possible steps, including going to a hearing at the Office of Administrative Hearings, and all parties take the full amount of time allotted. For example, a February 2009 complaint about a poorly refurbished pool that resulted in a license suspension took 18 months to resolve. This lengthy process has negative consequences in addition to the impact on the homeowners for whom relief is delayed. These include the ability of contractors who violate statute to continue working without discipline and potentially continue performing poor workmanship. In addition, there is a lack of up-to-date license information on the ROC’s Web site, which can negatively affect consumers’ hiring choices. Recommendations The ROC should: • Seek authority to incentivize contractors to resolve problems more quickly by charging complaint-processing fees when appropriate; Develop a process to monitor complaints to ensure they are moving through the process; and • Develop time frames for key steps in the process, such as issuing citations. • Problems with data system hamper ROC’s ability to perform core functions In March 2010, the ROC replaced an aging computer system with a new system, ROCIMS, to perform its core functions. This system was selected by the Governor’s Information Technology Agency (GITA) as one that could be used by several agencies. Responsibility for system implementation was shared between the ROC, GITA, and the State’s Web portal contractor. However, implementation of the new information management system was unsuccessful for many reasons. For example, although modification of some business practices is important to implementing this type of system, the ROC’s business practices were not evaluated and modified before the system was implemented. Also, a data migration plan was not used to ensure successful transfer of data from the old system to the new system. In addition, system testing was inadequate. As a result, as of August 2012, about 112,000 of ROC’s 544,000 records were duplicated or missing, while another 40,000 records contained inaccurate information. The ROC has experienced many difficulties because of its new system, including inefficiencies and the inability to comply with some statutory requirements. For example, ROCIMS is unable to identify the number of licenses an individual is associated with. This is important to know because if any one of those licenses is suspended, according to statute, all licenses with that individual’s name must be suspended. The ROC reported that it intends to replace ROCIMS with a new information management system. Whether or not this happens, it should take steps to fix the inaccurate data and take other steps to address system-related problems, which would also aid in the development and implementation of a new system. Recommendations The ROC should continue its efforts to correct the system’s data problems, analyze its business practices and redesign them as appropriate, and create processes for managing its information management system. Other pertinent information about the Recovery Fund The Legislature established the Recovery Fund to help homeowners who suffer financial losses because of a licensed residential contractor’s poor workmanship. Before a homeowner can access the Recovery Fund, he/ she must first either go through the ROC’s complaint process or obtain a civil court judgment against the contractor. Although the ROC has implemented new practices to more quickly process Recovery Fund claims, as of January 2013, homeowners were not receiving payments until approximately 12 to 13 months after the ROC approved the claim because of insufficient Recovery Fund monies. As of July 2012, the ROC had nearly $3.9 million in approved Recovery Fund claims that it could not pay because the Recovery Fund had not recovered from a total of $8.5 million in required transfers to the State General Fund in fiscal years 2009 through 2012. Registrar of Contractors A copy of the full report is available at: www.azauditor.gov Contact person: Shan Hays (602) 553-0333 REPORT HIGHLIGHTS PERFORMANCE AUDIT July 2013 • Report No. 13-04 TABLE OF CONTENTS Introduction Finding 1: ROC should consistently ensure that complaints are adequately resolved ROC handles complaints about contractor workmanship 1 9 9 ROC closed complaints without adequate resolution 12 ROC’s complaint-handling process differs from approach used by other Arizona regulatory agencies 16 ROC should modify its approach to resolving complaints 17 ROC should consistently close cases based on 2-year limitation 19 Recommendations 20 Finding 2: ROC should streamline complaint-resolution process 23 Lengthy complaint-handling process does not protect public 23 ROC should take additional steps to resolve complaints more quickly 26 Recommendations 29 Finding 3: Problems with data system hamper ROC’s ability to perform core functions 31 ROC installed new data system in 2010 31 ROCIMS was not properly implemented, which has affected the ROC’s efficiency 32 continued Office of the Auditor General page i TABLE OF CONTENTS ROC unable to use some system functionality and cannot effectively perform some statutorily required functions 38 ROC plans to replace ROCIMS but still needs to address ROCIMS-related implementation problems 41 Recommendations 44 Other Pertinent Information: Residential Contractors’ Recovery Fund protects homeowners harmed by licensed contractors Recovery Fund created to protect homeowners 47 ROC reviews Recovery Fund claims 48 Process for approving Recovery Fund claims has improved 49 Recovery Fund payments delayed 50 ROC expects Recovery Fund deficit to improve 51 Sunset factor analysis Appendix A: Methodology Agency Response continued State of Arizona page ii 47 53 a-1 TABLE OF CONTENTS Tables 1 Fees for and number of licenses by contractor type As of August 23, 2012 2 2 Residential Contractors’ Recovery Fund Schedule of revenues, expenditures, and changes in fund balance Fiscal years 2008 through 2012 (Unaudited) 6 Operating funds Schedule of revenues, expenditures, and changes in fund balance Fiscal years 2011 through 2013 (Unaudited) 8 3 4 Recovery Fund key facts during fiscal year 2012 48 5 Recovery Fund estimated payout schedule As of January 2013 50 Figures 1 Summary of formal complaint process 10 2 Recovery Fund cash balance and unpaid claims As of July 31, 2012 (Unaudited) 51 concluded Office of the Auditor General page iii INTRODUCTION Scope and Objectives The Office of the Auditor General has conducted a performance audit and sunset review of the Registrar of Contractors (ROC) pursuant to an October 26, 2010, resolution of the Joint Legislative Audit Committee. This audit was conducted as part of the sunset review process prescribed in Arizona Revised Statutes (A.R.S.) §41-2951 et seq. This performance audit and sunset review focused on determining whether the ROC consistently ensures that complaints are appropriately resolved, the length of the formal complaint process, and whether the ROC is able to perform its core functions using its data management system. In addition, this report provides information about the Residential Contractors’ Recovery Fund and includes responses to the sunset factors specified in A.R.S. §41-2954. Registrar of Contractors Purpose The Legislature created the Registrar of Contractors in 1931 (Laws 1931, Ch. 102) to license and regulate residential and commercial contractors (see textbox). Arizona is one of ten states that has a central regulatory body that regulates nearly The ROC’s Mission all types of construction trades. Other states leave this regulation to local government or To promote quality share the regulatory responsibilities among construction by Arizona multiple state agencies. Further, the extent of contractors through a regulation varies greatly among the states. licensing and regulatory For example, some states limit regulation to system designed to certain trades, such as electrical or plumbing protect the health, safety, work, while other states, like Arizona, regulate and welfare of the public. all types of construction trades. Responsibilities The Governor appoints a Registrar who oversees the ROC’s day-to-day operations and the administration of the Residential Contractors’ Recovery Fund (Recovery Fund). The ROC’s primary responsibility is regulating contractors, who are individuals and firms who perform construction services. The ROC regulates contractors by: Licensing—Statutes and agency rules establish licensing requirements for contractors. All contractors must be licensed with some exceptions, such as “handymen” who must limit themselves to jobs valued at less than $1,000, including the cost of labor and materials, and that do not require a permit. The ROC issues both commercial and residential contracting licenses and a dual license that allows a contractor to work on both commercial and residential properties. There are 238 different types of license classifications, including a general residential contractor, general dual swimming pool contractor, commercial engineering contracting for asphalt paving, and a residential specialty license for acoustical systems. As of August 2012, the ROC reported nearly 40,000 active licensed contractors in the State (see Table 1, page 2). Contractors’ licenses are issued to contracting entities, such as construction corporations, limited liability companies, partnerships, and individuals who own contracting businesses as sole proprietors. However, to receive a license, a contracting entity must have a qualifying individual who supervises the work of the licensed entity. The “qualifying party” does Office of the Auditor General page 1 Table 1: Fees for and number of licenses by contractor type As of August 23, 2012 Contractor type General Residential Commercial Dual2 Subtotal Engineering Residential Commercial Dual2 Subtotal Specialty Residential Commercial Dual2 Subtotal Total Biennial fees and assessments Initial Renewal Initial Recovery Renewal Recovery application Fund1 application Fund1 $445 890 1,105 Number of licenses $450 None 450 $290 580 860 $300 None 300 5,191 3,487 2,565 11,243 445 890 1,105 450 None 450 290 580 860 300 None 300 127 2,886 303 3,316 320 645 815 450 None 450 240 490 730 300 None 300 9,365 7,275 8,565 25,205 39,764 1 Residential and dual contractors contribute to consumer protection either by paying the Recovery Fund assessment or by posting a $200,000 surety bond or cash deposit. Contractors receiving their initial license and who choose to pay the Recovery Fund assessment pay $300 for the first year and $150 for the second year for a total of $450 for a 2-year license. Each subsequent time the contractor renews his/her license, the Recovery Fund assessment is $300—$150 for each of the 2 years of the biennial renewal period. 2 Contractors who hold a dual license are authorized to work on both commercial and residential projects. Source: Auditor General staff analysis of information posted on ROC’s Web site. not have to be the owner of the company, but may be someone who is employed regularly by the licensee and who is able to supervise the work performed by the employees of the company. For example, in a small painting company with only five employees, any one of those employees can be the designated “qualifying party” as long as that employee meets the licensing requirements listed below. The licensure requirements include: State of Arizona page 2 • Experience—The qualifying party must have worked in the trade for which the license is sought for between 6 months to 4 years, depending on the trade; • Examination—The qualifying party must pass a business management examination and a trade-specific examination, where applicable; • License bond—The applicant must secure a surety bond, cash bond, or certificate of deposit. The amount of the bond or certificate of deposit depends on the applicant’s license classification and expected volume of annual revenue; • License fee—The applicant must pay an initial biennial license fee (see Table 1, page 2, for fee amounts); • Other business requirements—The applicant must comply with state requirements associated with operating a business, such as having an Arizona Transaction Privilege Tax Number and workers’ compensation insurance; and • Public posting—A.R.S. §32-1104(C) requires the ROC to publicly post the names of first-time applicants for 20 days. The ROC may waive part of the posting period for previously licensed applicants. The ROC posts these names on its Web site as well as in the Licensing Department of its Phoenix office. Regulating workmanship standards by addressing complaints—In addition to licensing contractors, the ROC inspects and investigates complaints against contractors. The ROC’s complaint-handling function includes multiple departments and programs. Specifically: • • Building Confidence program—A.R.S. §32-1104 authorizes the ROC to establish a program to assist in the resolution of disputes in an informal process before a reportable written complaint is filed. The Building Confidence program is an informal program designed to help both contractors and property owners resolve concerns about the quality of workmanship. If a property owner is concerned that a contractor’s workmanship is substandard, the property owner or the contractor can request a Building Confidence visit. During the visit, a ROC inspector conducts an informal inspection to address concerns related to the quality of the workmanship. After the inspection, the results are provided to the property owner and the contractor. If the property owner and the contractor cannot resolve the issue after a Building Confidence visit, the property owner still has the option of filing a formal complaint against the contractor (see next bullet). In fiscal year 2012, the ROC reported receiving 165 requests for Building Confidence visits. A ROC official estimated that more than 90 percent of Building Confidence visits result in a resolution and do not proceed to the formal complaint process. The Building Confidence program is designed to help resolve concerns about workmanship. Formal complaint process—Consistent with statute, the ROC has established a formal complaint process to resolve issues of contractor work abandonment, failure to pay a subcontractor if the subcontractor’s services exceed $750, and quality of workmanship issues without the property owner or subcontractor having to go through a court process (see Finding 1, pages 9 through 21, for more information on the formal complaint process). Specifically, A.R.S. §§321154 and 1155, as well as other statutes that apply to all regulatory agencies in the State, establish major components of the ROC’s formal complaint process. Property owners or subcontractors can file formal complaints with the ROC against licensed contractors. After a complaint is received about poor workmanship, a ROC inspector conducts an onsite inspection to determine if the work meets standards. If the inspector finds problems with the licensed contractor’s workmanship, the contractor is given an opportunity to resolve the Office of the Auditor General page 3 problem. If the complaint is about work abandonment or failure to pay a subcontractor, the inspector sends the contractor a letter giving him time to resolve the issue. If the complainant is not satisfied that the licensed contractor has adequately resolved the problem within the time allotted by the ROC—10 days for abandonment and subcontractor payment complaints or 15 days for workmanship complaints—the complainant may request that the ROC issue a citation to the contractor (see textbox). Citation—A legal document from the ROC to a contractor listing the alleged violations of statute and potential disciplinary actions based on the violations. Source: Auditor General staff analysis of the ROC’s files and summary of A.R.S. §32-1155. If the problem is still not resolved after the ROC issues a citation, the complaint may then be sent to the Office of Administrative Hearings (OAH) for a hearing. According to the OAH’s fiscal year 2012 annual report, it received 779 cases for hearing from the ROC during that fiscal year. After the hearing, if the administrative law judge finds the contractor at fault, the ROC determines what discipline to impose on the contractor, if any, based upon the administrative law judge’s finding(s) and recommendation. A.R.S. §32-1154 provides three disciplinary options the ROC can take: suspending the license, revoking the license, or requiring the licensee to pay a civil penalty in order to retain his/her license. Depending on the violation, the ROC can also combine imposing a civil penalty on the licensed contractor with other disciplinary actions such as suspension. The contractor may have another opportunity to resolve the problem if the ROC issues a provisional order, which states the discipline that will be imposed if the contractor has not resolved the problem by the deadline established in the order. The ROC does not take disciplinary action in such cases if the problems are resolved. If the complaint results in the suspension or revocation of a contractor’s license, the complainant may be eligible to apply for monies from the Recovery Fund (see bullet on pages 5 through 7). According to the ROC, in fiscal year 2012, it closed 3,597 complaints without issuing a citation; issued 1,498 citations; and took disciplinary action against 1,139 licenses. Of the 1,139 licenses that received disciplinary actions, the ROC reported that 457 were suspensions and 682 were revocations. • The ROC reported receiving 1,566 complaints against unlicensed contractors in fiscal year 2012. State of Arizona page 4 Investigations of unlicensed contracting—The ROC also investigates statutory violations involving allegations of unlicensed contracting, aiding and abetting an unlicensed contractor, unlawful advertising, and providing false information on a license application. According to the ROC, in fiscal year 2012, it received 1,566 complaints against unlicensed contractors. The ROC takes an aggressive approach to protecting the public from unlicensed contractors through a quick investigation process. Auditors reviewed five cases of unlicensed contracting received between calendar years 2008 and 2012 and found that the ROC took between 11 and 99 days to handle these cases from the time that the ROC’s investigation unit received the complaint to the time the ROC referred the case to the courts for criminal prosecution. In addition, in accordance with A.R.S. §32-1151.02(A), the ROC protects the public by posting on its Web site the names of individuals criminally convicted or administratively adjudicated of contracting without a license. The ROC also includes information about how to protect oneself from unlicensed contractors on its Web site. According to the ROC, it also participates in undercover sting operations with other states’ contracting regulatory agencies. • Administering the Recovery Fund—The ROC administers the Recovery Fund, which provides financial relief to homeowners who have suffered a loss due to the actions of a licensed residential contractor if the homeowner first receives a court judgment against the contractor or if the contractor’s license has been suspended or revoked as a result of the homeowner filing a formal complaint with the ROC. In addition to paying new or renewal licensing fees, residential contractors and contractors with a license that allows them to work on both commercial and residential projects—a dual license—must either participate in the Recovery Fund by paying an assessment that is deposited into the Recovery Fund or post a $200,000 surety bond or cash deposit. These assessments are $450 for the initial biennial licensing period and $300 for biennial renewal periods (see Table 1, page 2). A.R.S. §32-1132(B) limits the assessment to $600 per biennial license period. As shown in Table 2 (see page 6), these assessment revenues represent the Recovery Fund’s primary revenue source, totaling more than $4.3 million of the approximately $4.5 million in net revenues the Recovery Fund received in fiscal year 2012. During fiscal year 2012, the ROC’s expenditures to operate the Recovery Fund were approximately $475,000, which was within the limit established by Laws 2011, Ch. 35, §8, and it paid judgments to homeowners totaling approximately $4 million. A homeowner who wishes to file a claim to receive monies from the Recovery Fund must either file a complaint with the ROC or go to court within 2 years after the date of occupancy or completion of the project. The homeowner may recover the amount of the actual damages up to $30,000. However, total claims against any one licensed contractor may not exceed $200,000. If total claims against the contractor exceed this amount, all pending claims payments are prorated. If Recovery Fund monies are paid to a homeowner for a claim, the ROC is required to suspend every license the contractor has until the contractor repays the Recovery Fund the full amount of the claim plus interest at the rate of 10 percent per year. The ROC’s staff determine a homeowner’s eligibility to receive money from the Recovery Fund and how much money will be paid. Pursuant to its authority to represent the ROC, Attorney General representatives provide legal assistance in cases where claim eligibility or amounts are in dispute. To obtain reimbursement for claims paid by the Recovery Fund for damages done by a contractor, the ROC sends the cases to the Attorney General’s Office for collection. Homeowners can recover up to $30,000 in actual damages from the Recovery Fund. A.R.S. §32-1134 requires the ROC to establish assessments and maintain the Recovery Fund’s fund balance at a level sufficient to pay operating costs and anticipated claims. However, the ROC’s ability to do so was affected by legislatively required transfers from the Recovery Fund to the State General Fund in fiscal years 2009 through 2012 (see Table 2, page 6). These transfers, which totaled approximately $8.5 million, were made in order to provide Office of the Auditor General page 5 page State of Arizona 6 73,709 11,926,852 $ 12,000,561 6,983,927 813,876 158,903 145,825 98,108 1,216,712 5,767,215 6,097,126 395,682 5,760 578,690 48 7,077,306 (19,670) 7,057,636 (7,886,283) 12,000,561 $ 4,114,278 827,575 10,335 103,148 13,072 954,130 6,352,530 6,600,000 13,906,660 6,042,719 (22,342) 6,020,377 $ 5,472,755 414,397 3,975 151,592 2009 506,578 423 81,826 32 588,859 5,355,888 1,837,300 7,782,047 5,206,279 (19,928) 5,186,351 4,956,029 216,442 3,190 30,618 (2,595,696) 4,114,278 $ 1,518,582 $ 2010 (1,414,631) 1,518,582 $ 103,951 571,526 5,588,872 23,500 6,183,898 425,757 96,034 49,735 4,792,042 (22,775) 4,769,267 $ 4,714,927 71,410 1,120 4,585 2011 $ 39,689 103,951 143,640 380,378 33,220 58,161 2,608 474,367 3,976,590 4,000 4,454,957 $ 4,348,860 115,574 55,700 279 1,188 4,521,601 (26,955) 4,494,646 2012 Source: Auditor General staff analysis of the Arizona Financial Information System (AFIS) Accounting Event Transaction File for fiscal years 2008 through 2012 and the AFIS Management Information System Status of General Ledger-Trial Balance screen for fiscal years 2010 through 2012. to provide support for state agencies. 3 Amount consists of transfers to the State General Fund in accordance with Laws 2009, Ch. 12, §44; Laws 2010, 7th S.S., Ch. 1, §148; and Laws 2011, Ch. 24, §§129 and 138 include any adjustments for amounts approved but unpaid or reported but unapproved during the applicable fiscal year. Prior to fiscal year 2011, the ROC reported that it had sufficient resources to pay claims as they were approved; however, toward the end of fiscal year 2011, resources were not sufficient to pay approved claims. Consequently, the ROC began calculating the amount of claims approved but unpaid at the end of the fiscal year and no adjustments were included in the table. At June 30, 2012, these claims totaled approximately $3.9 million. In addition, the ROC does not quantify claims that have been reported until they are approved; therefore, no adjustments were made for these potential claims. (See Other Pertinent Information, pages 47 to 52, for additional information.) 2 Amount represents payments made to claimants who were eligible and had suffered compensable actual damages as defined by the Recovery Fund statutes, but it does not 8). 1 The table includes the financial activity of the Recovery Fund. It excludes the financial activity of the ROC’s operating funds, which is reported separately in Table 3 (see page Net change in fund balance Fund balance, beginning of year Fund balance, end of year Expenditures and transfers Operating expenditures: Personal services and related benefits Professional and outside services Other operating Furniture, equipment, and leasehold improvements Total operating expenditures 2 Judgments Transfers to the State General Fund3 Total expenditures and transfers $ 2008 Residential Contractors’ Recovery Fund1 Schedule of revenues, expenditures, and changes in fund balance Fiscal years 2008 through 2012 (Unaudited) Revenues Initial and renewal fees Recoveries Fines, forfeitures, and penalties Interest on investments Other Gross revenues Credit card and online transaction fees Net revenues Table 2: adequate support and maintenance for state agencies. According to the ROC, as a result of the transfers, since March 2011, it has delayed payments to eligible homeowners and has only been able to pay claims when sufficient monies were available in the Recovery Fund. Claims are paid based on the order in which the claims were approved (see Other Pertinent Information, pages 47 through 52, for additional information about the Recovery Fund). Staffing and budget The ROC operates eight locations state-wide with its main office located in Phoenix.1 For fiscal year 2013, the ROC was appropriated 105.6 full-time equivalent (FTE) positions and as of August 2012, the ROC had 90 employees according to its organization chart. Specifically, the ROC has organized its employees as follows: • Executive and administrative (12 FTE)—The Director, chief of staff, chief legal counsel, assistant director of administration, and administrative staff provide overall direction and administrative support to the ROC; • Information technology (11 FTE)—The chief information officer and other information technology staff manage the ROC’s data system; • Inspections and investigations (40 FTE)—Inspectors conduct Building Confidence visits as well as worksite inspections for formal complaints against licensed contractors, and investigators investigate unlicensed contracting; • Licensing (13 FTE)—Licensing staff review applications and issue licenses; • Legal (9 FTE)—Legal department staff issue citations and process cases through the OAH; and • Recovery Fund (5 FTE)—Employees paid by Recovery Fund monies determine homeowners’ eligibility and the amount to be awarded for each claim. The ROC does not receive any State General Fund monies. Rather, the ROC’s revenue comes mainly from initial and renewal application fees. These fees vary depending upon the type of license issued. For example, initial application fees range from $320 for a new specialty residential contractor to $1,105 for a general dual contractor (see Table 1, page 2). The ROC also remits its interest, civil penalties, and 10 percent of all other operating revenues to the State General Fund. As shown in Table 3 (see page 8), the ROC generated or estimated generating between $11.1 and $11.8 million in annual gross revenues for fiscal years 2011 through 2013 and remitted or estimated remitting between $1.1 million and $1.3 million annually to the State General Fund during this period. 1 The ROC plans to close its Lake Havasu City office in June 2013. The investigator in the office will continue investigating cases in the area. Other business conducted by the office will be reassigned to other office locations. See Sunset Factor 3, page 59, for a list of the ROC’s offices. Office of the Auditor General page 7 Table 3: Operating funds1 Schedule of revenues, expenditures, and changes in fund balance Fiscal years 2011 through 2013 (Unaudited) Revenues Initial and renewal licenses Fines, forfeitures, and penalties Publications and reproductions Interest on investments Other Gross revenues Credit card and online transaction fees Remittances to the State General Fund3 Net revenues 2011 (Actual) 2012 (Actual) 2013 (Budget)2 $ 11,399,132 337,532 8,004 9,095 28,090 11,781,853 (67,739) (1,294,466) 10,419,648 $ 10,794,557 251,840 14,658 10,756 30,586 11,102,397 (62,954) (1,169,463) 9,869,980 $ 10,937,000 251,800 13,200 Expenditures and transfers Personal services and related benefits Professional and outside services Travel Other operating Equipment Total expenditures Transfers to the State General Fund4 Transfers to the Office of Administrative Hearings Total expenditures and transfers 5,576,281 102,488 171,215 1,440,054 79,697 7,369,735 247,500 534,445 8,151,680 5,959,728 148,303 158,702 1,201,118 55,437 7,523,288 52,900 471,917 8,048,105 Net change in fund balance Fund balance, beginning of year Fund balance, end of year 2,267,968 5,692,795 7,960,763 1,821,875 7,960,763 9,782,638 $ $ 36,300 11,238,300 (56,100) (1,174,300) 10,007,900 7,854,600 405,300 516,900 2,165,100 100,300 11,042,200 1,017,600 12,059,800 (2,051,900) 9,782,638 $ 7,730,738 1 The table includes the ROC’s operating funds. It does not include financial activity of the Cash Bond or Contractors Prompt Pay Complaint Funds since the registrar is only a custodian of these monies. In addition, it excludes the financial activity of the Recovery Fund, which is reported separately in Table 2 (see page 6). 2 Amounts were based on budgeted amounts submitted to the State’s budget offices in September 2012. The fiscal year 2013 budgeted expenditure amounts were similar to the fiscal year 2012 budgeted amounts; however, the ROC’s actual fiscal year 2012 expenditures were much lower than the budget. According to the ROC, the lower expenditures resulted from agency cost-cutting measures, including strictly complying with the state-wide hiring freeze, closing and consolidating offices, reducing vehicles, decreasing in-state travel, eliminating out-of-state travel, cutting cell phones, eliminating mailings, reducing printing, eliminating paper, and reducing staffing. Similarly, the ROC expects the fiscal year 2013 actual expenditures to be less than the budgeted amount. 3 The ROC remits all of its interest, civil penalties, and 10 percent of all other revenues to the State General Fund. 4 Amount consists of transfers to the State General Fund in accordance with Laws 2010, 7th S.S., Ch. 1, §148; and Laws 2011, Ch. 24, §§129 and 138, to provide support and maintenance for state agencies. Source: Auditor General staff analysis of the AFIS Accounting Event Transaction File and the AFIS Management Information System Status of General Ledger-Trial Balance screen for fiscal years 2011 and 2012; and the ROC’s fiscal year 2013 budget submission. The ROC’s operating expenditures in fiscal year 2012 were more than $7.5 million, with approximately 79 percent of this amount spent for personnel costs, including employee-related costs. Although Table 3 shows fiscal year 2013 budgeted expenditure amounts totaling approximately $11 million, the ROC expects actual expenditures to be less. Finally, the ROC ended fiscal year 2012 with a fund balance of nearly $9.8 million, but estimates ending fiscal year 2013 with a fund balance of approximately $7.7 million. State of Arizona page 8 FINDING 1 The Registrar of Contractors (ROC) should consistently ensure that complaints against licensed contractors are adequately resolved. The ROC handles complaints about workmanship through a process that provides opportunities for contractors to resolve problems and allows for citations, hearings, and disciplinary action if problems are not resolved. This process is intended to assist consumers by resolving issues of poor workmanship or abandoned work, but it does not consistently ensure that these problems are resolved before the ROC closes the complaint. As a result, the ROC does not consistently ensure that the public is adequately protected. The ROC’s approach to resolving complaints, which differs from the regulatory stance adopted by other Arizona regulatory agencies, focuses on benefiting both the consumer and the contractor. However, to better protect the public, the ROC should modify its complaint-resolution process to ensure that problems are adequately addressed and strengthen its processes for disciplining licensed contractors in those cases that warrant such action. Finally, the ROC does not consistently apply the 2-year statutory time limit for filing a complaint and should develop guidance for its staff to better ensure consistency. ROC should consistently ensure that complaints are adequately resolved ROC handles complaints about contractor workmanship Consistent with statute, the ROC has established a process to address complaints involving licensed contractors who violate statutes by performing poor workmanship, abandoning contracted work, or failing to pay a subcontractor (see Figure 1, page 10). Specifically, A.R.S. §§32-1154 and 32-1155, as well as other statutes that apply to all regulatory agencies in the State, establish major components of the ROC’s formal complaint process. This process provides contractors multiple opportunities to resolve the problems alleged in the complaint and ordered to be corrected by a ROC inspector, without receiving discipline. Specifically: • Contractor given opportunities to fix problems—When the ROC receives a complaint, it notifies the licensed contractor who is the subject of the complaint and verifies the allegations of statutory violations in one of two ways, depending on the nature of the complaint: ◦ For workmanship complaints, the ROC schedules and conducts an inspection to determine whether the work meets standards. In its letter to the contractor and complainant that schedules the inspection, the ROC advises the contractor to resolve the problems prior to the inspection. If the inspection finds that the work meets standards, the ROC sends a letter to both parties informing them that the complaint has been closed. If the inspection finds that the work does not meet standards, the ROC issues a corrective work order to the licensed contractor and the complainant, requiring the contractor to correct the work within a specified time frame—the time frame must be at least 15 days according to A.R.S. §32-1154(A)(23)—and informing the complainant that if the work is not completed within the allotted time, the complainant has 10 additional days to request that the ROC issue a citation.1 If the ROC does not receive a request for a citation from the complainant, it sends a letter notifying the contractor that the complaint is closed, but that it could be reopened upon written request of the complainant. During the audit, the ROC developed a letter to send to the complainant. 1 A citation is a legal document from the ROC to a contractor listing the alleged violations of statute and potential disciplinary actions based on the violations. Office of the Auditor General page 9 Figure 1: Summary of formal complaint process Process steps Complaint outcome Complainant Files a formal complaint with ROC ROC Sends letter asking contractor and complainant to resolve Complaint closed if contractor and complainant resolve problems ROC Conducts worksite inspection for workmanship complaint1 ROC Issues corrective work order if inspection finds problems Complainant Asks ROC to issue a citation2 Complaint closed if contractor resolves problems identified in corrective work order Complaint closed if complainant does not request citation or if contractor resolves problems once notified that citation was requested ROC Issues a citation and refers complaint to Office of Administrative Hearings (OAH) if contractor responds to citation OAH Holds hearing and judge recommends decision ROC Issues decision order based on judge’s recommendation ROC takes disciplinary action if contractor does not respond to citation3 Hearing may be vacated if contractor and complainant reach a settlement agreement at OAH ROC takes disciplinary action3 or Complaint closed if order allows contractor to resolve the problems and the contractor resolves problems before decision effective date 1 The ROC does not conduct inspections for complaints alleging abandonment or failure to pay subcontractors. In such complaints, the initial letter gives the contractor 10 days to resolve the problem and gives the complainant another 10 days to ask the ROC to issue a citation. 2 A citation is a legal document from the ROC to a contractor listing the alleged violations of statute and potential disciplinary actions based on the violations. 3 The ROC can suspend the license, revoke the license, impose a civil penalty, or summarily suspend all licenses the licensee holds. Source: Auditor General staff summary of information obtained from the ROC and review of complaint-related statutes and rules. State of Arizona page 10 ◦ For complaints about abandoned work or failure to pay a subcontractor, the ROC sends the contractor and complainant a letter that notifies the licensed contractor of the complaint, encourages both parties to resolve the problems prior to further involvement by the ROC, and gives the licensed contractor 10 days from the date of the letter to remedy the situation. If the licensed contractor does not resolve the problems within 10 days, the complainant has an additional 10 days to request that the ROC issue a citation to the contractor. Again, if the ROC does not receive a request for citation from the complainant, it sends a letter notifying the contractor that the complaint is closed, but that it could be reopened upon written request of the complainant. During the audit, the ROC developed a letter to send to the complainant. • Complainant requests that the ROC issue a citation—When the ROC receives a request from a complainant to issue a citation to the licensed contractor, the ROC notifies the licensed contractor of the request for a citation and allows the licensee another opportunity to resolve the complaint. This notice does not specify a time limit for correcting the problems. At this point in the process, the ROC inspector transmits the complaint to the ROC’s legal department to prepare the citation. If the complaint has not been addressed by the time the legal department is ready to draft the citation, the ROC prepares and issues a citation to the contractor. The citation lists the alleged statutory violations and potential disciplinary actions. • Citation, default, or hearing—When the licensed contractor receives the citation, A.R.S. §32-1155(B) requires that the contractor respond within 10 days. Failure to respond is deemed an admission of guilt, and the ROC can proceed with any of the disciplinary actions established by A.R.S. §32-1154, including suspending or revoking the contractor’s license or issuing a civil penalty. Conversely, if a licensed contractor denies the allegations specified in the citation, then the ROC prepares the case for a hearing with the OAH. However, according to the ROC, if the licensed contractor addresses the complaint allegations at this point in the complaint process and the Settlement agreement—An agreement reached complainant notifies the ROC that the problems have by the complainant and contractor about been resolved, the ROC will close the case and it will additional work to be performed by the not be heard by the OAH. In addition, if the complainant contractor or an amount of money the contractor and the contractor reach a settlement agreement agrees to pay the homeowner to settle the case. prior to or during the hearing, the OAH may vacate Source: Auditor General staff analysis of the ROC’s Web site the hearing and the ROC will close the case (see and complaint process. textbox). • Decision finalized—If the complaint proceeds to an OAH hearing, A.R.S. §411092.08(A) requires the administrative law judge to issue a written decision to the ROC within 20 days after the hearing is concluded. The ROC then has 30 days to accept, modify, or reject the judge’s recommendation and issue an order to the contractor stating the discipline and the effective date of the discipline. The order may be provisional, which gives the contractor another opportunity to resolve the problem and states the discipline that will be imposed if the problem is not Office of the Auditor General page 11 resolved by the effective date of the order. According to ROC officials, after the effective date indicated in a provisional order, the ROC’s legal department reviews any evidence submitted by the contractor and/or complainant to determine whether the problem was resolved. Once this determination has been made for a provisional order, or on or after the effective date indicated in the order for nonprovisional orders, the ROC enters the information into its data system, thereby allowing the public to see the complaint’s disposition when they look up the contractor on the ROC’s Web site. ROC closed complaints without adequate resolution The ROC’s process does not ensure that complaints are adequately addressed prior to closing them. For example, the ROC closed complaints without verifying that issues of poor workmanship and/or abandoned projects have been remedied. As a result, when complaints are not adequately resolved, the public may not be protected from problem contractors. ROC closed complaints without verifying problems are fixed—The The ROC considers a complaint resolved if the complainant no longer wishes to pursue it. ROC’s complaint-handling process allows complaints against licensed contractors to be resolved in various ways, such as the licensed contractor correcting poor workmanship, both parties agreeing to a needed repair or monetary settlement for the complainant, or by the ROC taking disciplinary action. For example, the ROC considers poor workmanship and abandoned work complaints resolved if the work has been repaired to the complainant’s satisfaction or if the complainant no longer wishes to pursue a complaint. The ROC indicated that it instructs its staff to contact both the complainant and licensed contractor to determine if the complaint has been resolved prior to closing a complaint. However, the ROC does not typically verify that workmanship problems identified in a complaint have been adequately addressed to industry workmanship standards by performing followup inspections prior to closing a complaint. Auditors’ review of ten complaints that were closed in fiscal year 2011 prior to the issuance of a citation confirmed that the ROC closed six complaints without ensuring that the workmanship problems had been adequately addressed through a follow-up inspection.1 The other four complaints were closed because of the statutory requirement that complaints be filed within 2 years of the contractor’s work for the ROC to be able to take action on them (see pages 19 through 20 for information regarding the statutory time limit). Specifically: • Some complaints closed without verification if complainant did not request a citation—The ROC automatically closed complaints if the 1 Auditors reviewed complaint cases closed in fiscal year 2011 including 10 complaints that closed prior to a citation and 10 complaints that closed after receiving a citation in order to observe the results of the ROC’s complaint process in actual cases. In addition, auditors reviewed the ROC’s actions regarding one contractor whose case was reported in an April 2012 news article regarding the contractor’s payment to settle accusations of fraud. That contractor’s 25 complaints are discussed separately in the textbox on page 15 regarding a licensee who was not disciplined. State of Arizona page 12 complainant did not request that the ROC issue a citation to the licensed contractor within the time limits specified in the ROC’s initial notice letter or corrective work order. For these types of cases, the ROC did not verify that the problems identified in the complaint had been resolved before closing the complaint (see textbox). In three complaints auditors reviewed, including one of the ten that were closed prior to issuance of a citation, the ROC had automatically closed the complaints, but later reopened them because the problems identified in the original complaint had not been addressed, and the complainants later contacted the ROC to request it to reopen the complaints. The ROC’s former practice of notifying the contractor but not the complainant when it automatically closed cases increased the risk that problems identified in complaints had not been adequately addressed. Complaint closed without verifying problems had been rectified—In January 2010, the ROC received a complaint regarding poor workmanship on a pool, including cracked deck tile, leaks in the pool, and pipes sticking out too far. In February 2010, the ROC inspected the pool and issued a corrective work order directing the licensed contractor to fix the work within 15 days. Following the expiration of the 15 days to fix the pool, the ROC did not hear from the complainant to request a citation in the additional 10-day time period to do so. The ROC closed the complaint in March 2010 without verifying that the workmanship problems had been addressed and, as was the ROC’s practice, the ROC notified the contractor that the complaint was being closed but did not notify the complainant. In December 2010, the complainant resubmitted the complaint because the contractor had not adequately resolved the problems. In February 2011—more than 1 year after the original complaint was filed—the contractor and complainant signed a settlement agreement stating that the contractor would repay the complainant nearly $3,000. Source: Auditor General staff analysis of the ROC’s files. • Some complaints closed without verification if complainant agreed to close complaint—Closing complaints based on the complainant’s agreement does not ensure that the contractor met workmanship standards. For four of the ten complaints auditors reviewed, the ROC’s records indicated that the complainants agreed to close the complaints, but the case files lacked documentation that the problems were resolved. For example, one complaint involved the repair of a swimming pool where, after conducting an inspection and issuing a corrective work order, the ROC closed the complaint based on a telephone conversation with the complainant, who stated that the work had been completed. However, the complainant wrote to the ROC less than 2 months later to request that the ROC reopen the case and proceed to a hearing. The ROC closed the case again 10 days later when the complainant called to state that all the work had been properly completed. • Some complaints closed without verification if contractor promised to fix problems—Finally, the ROC closes some cases based on the licensed Office of the Auditor General page 13 contractor’s promise to fix the workmanship problems. For two of the ten complaints auditors reviewed, the ROC inspectors appear to have closed the complaints when the contractors agreed to complete the repairs, but again, without verifying that the workmanship problems were adequately addressed.1 One complaint was closed before the inspector completed the inspection because the contractor had promised to repair the peeling paint and the complainant had agreed that the ROC could close the complaint based on the contractor’s promise. However, the complainant subsequently contacted the ROC to report that the repairs were not satisfactory. The ROC’s inspector informed the complainant that the ROC could not reopen the complaint because this subsequent contact was more than 2 years after the initial work was completed. A.R.S. §32-1155(A) establishes a 2-year time limit for the ROC to be able to address complaints (see pages 19 through 20 for additional information regarding the 2-year time limit). Public at risk when complaints are not adequately resolved—When the ROC does not adequately resolve complaints or when appropriate, administer disciplinary action, the public is not protected. Specifically, complainants experience frustrations and delays or difficulties when attempting to remedy losses when complaints are not adequately resolved. For example, • Auditors contacted a complainant who had submitted a complaint to the ROC in March 2009 about multiple problems with poorly installed flooring. The ROC’s inspection found that the workmanship was poor, and the ROC issued a corrective work order to the licensed contractor listing eight items the contractor needed to address. The complainant eventually requested that the ROC issue the contractor a citation because the contractor had not responded to the corrective work order within the allotted time frame, and a hearing was held in November 2009. At the hearing, the contractor agreed to settle the complaint by paying the complainant $1,634, and the ROC closed the case as “settled.” However, the contractor did not pay the agreed-upon amount, so the complainant requested that the ROC reopen the case. A second hearing was scheduled for April 2011, but was canceled when the contractor promised to make four payments totaling $1,634 on an agreed-upon schedule. The complainant asked the ROC to not close the complaint until the payments were received. However, the ROC closed the case again in June 2011 without verifying that the money had been paid. Additionally, the ROC did not take disciplinary action against the contractor despite finding that the contractor’s work did not meet workmanship standards. The complainant reported that the contractor still did not make the payments on the agreed-upon schedule, and as a result, the complainant decided to pursue collecting the money on his own instead of reopening the complaint with the ROC. The complainant stated that the ROC’s process was frustrating, 1 One of these cases is also included in the number that the ROC closed because the complainant agreed to close the complaint. State of Arizona page 14 particularly because the contractor’s license remained in good standing after the original complaint was closed, despite the poor workmanship and failure to meet the terms of the settlement agreement. In addition, consumers who want to use the ROC’s Web site to make informed choices about hiring a licensed contractor may not receive adequate information (see Sunset Factor 2, pages 58 through 59, for additional information regarding the ROC’s Web site). The ROC recommends checking a contractor’s license status before signing any contract, but if previous complaints were not adequately resolved, the Web site listing may not accurately convey the contractor’s adherence to industry workmanship standards. Furthermore, the ROC does not take disciplinary action when licensed contractors address problems identified in a complaint at any point in the complaint process prior to and in some cases even after a formal hearing. According to the ROC, it approaches complaint handling similar to a mediation process and seeks a resolution to the problems identified in a complaint and eventual dismissal of a complaint once both parties are satisfied, rather than disciplining licensed contractors who have violated statutes even if the problems are serious or the contractor has multiple complaints. As a result, licensed contractors who do not meet workmanship standards or abandon work are allowed to continue working without penalty, and contractors with a history of poor workmanship as demonstrated by multiple complaints may be allowed to continue performing additional substandard work (see textbox for an example). The ROC does not take disciplinary action when licensed contractors address problems identified in a complaint. Licensee with multiple complaints not disciplined—Between calendar years 2009 and 2012, a licensed contractor received 25 complaints, including some with workmanship issues such as failing to replace drywall that was removed from a bathroom to perform repairs and failing to replace a back patio. Based on the limited information available in the ROC’s data system, at least 6 of these complaints were issued a citation or corrective work order. However, as of October 2012, the ROC’s Web site showed that 4 complaints had been resolved and 1 complaint was still open, and that there had been no discipline taken against this contractor’s license. These complaints were closed for reasons such as the complainant’s not pursuing the case, the work being repaired by another contractor, or the complaint’s alleging theft, which is criminal and outside of the ROC’s jurisdiction. Source: Auditor General staff analysis of the ROC’s files and Web site. The ROC also does not escalate discipline for contractors with a history of violating statutes multiple times. The ROC uses a matrix to help decide discipline for contractors who have multiple concurrent complaints, but the ROC’s process does not provide for escalated discipline for contractors with a history of violations. According to the ROC, although only a small percentage of licensed contractors receive complaints, the few contractors who receive complaints often receive Office of the Auditor General page 15 multiple complaints. The ROC’s matrix for disciplining contractors with concurrent complaints provides for penalties such as probation, suspension, and revocation based on the number and nature of the concurrent complaints. However, the ROC has not included guidance in this matrix to help ensure contractors with a history of substantiated violations also receive appropriate discipline. Finally, if the ROC does not take disciplinary action, homeowners cannot recover their losses from the fund that was established to help them. Specifically, the Residential Contractors’ Recovery Fund (Recovery Fund) is available to provide recourse to homeowners who suffer a financial loss due to the actions of a licensed residential contractor (see Introduction, pages 5 through 7). However, the Recovery Fund is available to the homeowner only when the contractor’s license has been suspended or revoked as a direct result of the homeowner’s complaint. Thus, if the ROC does not take appropriate disciplinary action against a license, the homeowner cannot access the Recovery Fund. Homeowners cannot recover losses from the Recovery Fund if the ROC does not take disciplinary action. ROC’s complaint-handling process differs from approach used by other Arizona regulatory agencies The ROC’s complaint-handling approach has been in place since at least 1988. According to the ROC, its approach was adopted as a way to benefit both the complainant and the licensed contractor. Specifically, the complainant benefits by getting the problem resolved, while the licensed contractor benefits by being able to avoid disciplinary action. In two main ways, this approach differs from the regulatory stance adopted by other Arizona regulatory agencies. • Other Arizona regulatory agencies pursue complaints on their own—Some Arizona regulatory agencies pursue complaint investigations without requiring periodic complainant intervention to continue investigating and eventually resolving a complaint. The Arizona Agency Handbook (Handbook), published by the Attorney General’s Office, describes how the ROC differs from other agencies in requiring the complainant to bear the burden of pursuing the complaint.1 According to the Handbook, many enforcement and disciplinary proceedings involve two parties, the State or state agency and the licensee; but the ROC’s proceedings are based on formal complaints filed by a complainant, who is then a party to the proceeding. According to ROC officials, the ROC becomes a party to some types of complaints, but as of September 2012, it had not developed policies and procedures explaining its practice of pursuing some types of statutory violations and not others. Specifically, the ROC stated that it becomes a party to complaints where the complainant may not have the knowledge and evidence that the ROC 1 Arizona Attorney General (2011). Arizona Agency Handbook. Phoenix, AZ: Author. State of Arizona page 16 has, such as complaints alleging workers’ compensation violations, conviction of a felony, and failure to comply with advertising requirements. These types of violations do not involve issues of poor workmanship or abandoned work. However, in 2011, the ROC also became a party to a complaint against a contractor who had received six workmanship complaints within an approximately 2-year period. The ROC citation and complaint stated that the contractor had performed work beyond the scope of his contractor’s license, used false or deceptive advertising, failed to comply with building codes, committed fraudulent acts, and failed to complete a project for the price stated in the contract. Additionally, a ROC official stated that the ROC may become a party to complaints against licensed contractors with a history of repeated statutory violations that did not result in disciplinary action. The ROC developed a list of these contractors during the audit. According to ROC officials, neither the ROC nor the Attorney General’s Office have enough resources to proceed to a formal hearing when necessary for all of the complaints the ROC receives if the ROC were to become a party to all complaints. According to the OAH’s fiscal year 2012 annual report, the ROC already had more cases filed with the OAH than any other state agency except for the Arizona Health Care Cost Containment System (AHCCCS). Specifically, the ROC accounted for 779 cases filed with the OAH in fiscal year 2012, compared to 472 cases for the Department of Weights and Measures, the agency with the next highest number of cases. AHCCCS accounted for a total of 3,469 cases. • The ROC accounted for more OAH cases in fiscal year 2012 than any other state agency except AHCCCS. Other agencies take disciplinary action if licensees violate statutes, even if problems reported in complaint are resolved—Other Arizona regulatory agencies have processes to better ensure that licensees who commit violations are disciplined, regardless of whether or not complaints have been resolved to the complainant’s satisfaction.1 For example, if the Arizona State Board of Nursing finds that a licensee violated statutes or regulations, regardless of whether the licensee has taken steps to satisfy the complainant, it may discipline the licensee in a variety of ways based upon the risk or harm involved. However, the ROC does not take disciplinary action if contractors fix the work that led to the complaint or otherwise resolve the problem with the complainant, even when problems are serious or the contractor has had multiple complaints. ROC should modify its approach to resolving complaints The ROC can better protect the public by making changes to its complaint-handling process. First, the ROC should strengthen its process for ensuring that problems identified in the complaint have been adequately addressed and that work has been completed or standards of workmanship have been met before closing the complaint. 1 Auditors reviewed practices at the Arizona State Board of Nursing, the Arizona Medical Board, the Arizona Board of Appraisal, the Arizona State Board of Physical Therapy, and the Arizona State Board of Pharmacy. Office of the Auditor General page 17 Second, the ROC should strengthen its process for disciplining licensed contractors in those cases that warrant such action. In determining the discipline to impose, one mitigating factor could be whether contractors adequately addressed the problems identified in the complaint. ROC should strengthen its processes for ensuring that workmanship standards are met before closing complaints—According to New Mexico performs re-inspections to ensure work is fixed. the National State Auditors Association’s (NSAA) best practices in carrying out a state regulatory program, an agency should follow up as needed to determine whether a problem has been corrected or whether additional enforcement is needed.1 Auditors found that New Mexico’s contracting regulatory agency performs this followup. Specifically, according to the New Mexico Construction Industries Division of the Regulation and Licensing Department, it performs re-inspections to ensure that the contractor fixed workmanship problems found in its first inspection. Further, the New Mexico Construction Industries Division does not rely on homeowners to request these re-inspections, but rather requires the contractor to request a re-inspection and pay a re-inspection fee. In contrast, the ROC relies on complainant satisfaction and sometimes on contractor promises to verify that problems have been resolved when deciding whether to close a complaint. ROC complaints reviewed by the OAH demonstrate another way to verify that problems identified in a complaint have been addressed. For two of the three complaints auditors reviewed that proceeded to the OAH, official orders issued by the administrative law judges required the licensed contractor to provide “written proof” of compliance with the ROC’s corrective work order. In the third complaint, the judge vacated the hearing because the parties had reached agreement on a financial settlement. Therefore, the ROC should develop and implement policies and procedures for verifying that licensed contractors adequately address poor workmanship and/or abandoned work issues prior to closing a complaint. These policies and procedures should specify under what circumstances written documentation from the contractor or complainant and/or verbal statements by the complainant would be sufficient to support that corrective action was taken. They should also require the ROC inspectors to conduct follow-up inspections when appropriate to verify that work has been properly completed. Finally, the policies and procedures should address what steps the ROC will take, including issuing a citation, if the verification of corrective action does not indicate that the corrective action was properly completed. ROC should strengthen its processes for disciplining licensed contractors when appropriate—The ROC should use consent agreements to take disciplinary action against licensed contractors when appropriate. Specifically, the ROC should: 1 National State Auditors Association. (2004). Carrying out a state regulatory program. Lexington, KY: Author. State of Arizona page 18 • Develop policies and procedures to guide its use of consent agreements to take disciplinary action against licensed contractors who violate statute when appropriate. Other Arizona regulatory agencies use consent agreements to impose disciplinary actions when licensees have verified violations of statute. For example, the Arizona Medical Board enters into consent agreements for discipline that cite the substantiated violations and deviations from the standard of care, along with the agreed-upon discipline. The Arizona State Board of Nursing and other health regulatory agencies also use consent agreements to impose discipline. The ROC similarly has authority to use consent agreements, and should develop policies and procedures that would guide its use of these agreements to impose discipline. Such policies and procedures would help ensure it disciplines licensed contractors when appropriate and also would help conserve resources by reducing the need for OAH hearings. These policies and procedures should consider not only the nature of the violation and/or the repeat nature of the violation, but also mitigating and aggravating factors, such as whether the licensed contractor addressed workmanship problems in a timely manner. If licensed contractors who have substantiated violations decide not to enter into a consent agreement, the ROC should proceed with its complaint-handling process by referring these cases to the OAH. • Ensure that licensed contractors with multiple, substantiated complaints receive uniform, escalated discipline. The NSAA best practices for carrying out a state regulatory program suggests that the number and severity of violations should trigger a graduated level of sanction. As discussed previously, the ROC has developed a matrix that guides staff in escalating discipline for contractors who are the subject of multiple complaints at the same time. However, the ROC has no mechanism for escalating discipline for contractors with prior complaints that resulted in substantiated violations. As a result, the ROC should develop and implement policies and procedures for escalating discipline for contractors with prior complaints that resulted in substantiated violations. The ROC’s matrix for escalated discipline covers contractors with multiple complaints at the same time, but not contractors with a history of prior violations. ROC should consistently close cases based on 2-year limitation Although statute requires that the ROC only take action on complaints that were received within 2 years of the contractor’s work, the ROC does not have a method for consistently determining whether complaints fall within this 2-year time period. As a result, the ROC should develop and implement policies for determining which complaints meet the 2-year time frame requirement. Time limit for eligibility applied inconsistently—According to A.R.S. §321155(A), complaints must be filed with the ROC within 2 years of the close of escrow or the completion of the contractor’s work for the ROC to be able to act on them. However, the ROC has not taken a consistent approach in determining whether a Office of the Auditor General page 19 complaint was received within the 2-year statutory time frame. For example, the ROC received a complaint against a licensed contractor for work costing more than $30,000, but dismissed the complaint because of the 2-year time limit, even though the complainant provided documentation suggesting that the complaint may have been filed within 2 years of the completion of the work. In contrast, the ROC received a complaint regarding a project that cost approximately $800 that was clearly outside of the 2-year time limit, but the ROC conducted an inspection before determining that the case was outside of its jurisdiction. According to the ROC, if the complaint is not clearly out of its jurisdiction, it may still conduct an inspection. Of the ten complaints auditors reviewed that were closed prior to the ROC’s issuing a citation to the contractor, four complaints were dismissed because the complaint was received after the 2-year limit. The ROC reported lacking written policies and procedures to guide its staff in appropriately applying the 2-year time limit for filing complaints. According to the ROC, there are no written policies or procedures to guide its staff in determining the appropriate applications of the 2-year time limit. Inconsistent application of the 2-year time limit is unfair to contractors and the public and could potentially prevent the ROC from investigating some complaints that it should investigate and could result in its expending some resources on complaints that are outside its jurisdiction. Guidance needed for applying time limit—According to the ROC, its inspectors are given discretion to analyze the facts of each complaint and apply the appropriate statutory factors in determining on a case-by-case basis whether a complaint is within the 2-year limit. NSAA best practices suggest that a regulatory agency’s complaint-handling process should include a step that screens out complaints that a regulatory agency does not have jurisdiction to investigate. Therefore, to ensure the ROC investigates only complaints it receives that are within the 2-year time limit, the ROC should establish and implement guidelines to help its staff appropriately determine whether complaints meet the time frame requirement and therefore should be investigated. Recommendations: 1.1 The ROC should develop and implement policies and procedures to better ensure that complaints are adequately addressed prior to closing them. These policies and procedures should: State of Arizona page 20 a. Specify under what circumstances complaints should be closed based on written documentation from the contractor or complainant and/or verbal statements by the complainant indicating that corrective action had been taken; b. Specify under what circumstances complaints should not be closed until ROC inspectors conduct follow-up inspections to verify that work has been properly completed; and c. Specify the steps the ROC will take if corrective action was not properly completed. 1.2 The ROC should develop and implement policies and procedures to guide its use of consent agreements to discipline licensed contractors when appropriate. These policies and procedures should: a. Consider not only the nature of the violation and/or the repeat nature of the violation, but also mitigating and aggravating factors, such as whether the licensed contractor addressed workmanship problems in a timely manner; and b. Specify that if licensed contractors who have substantiated violations decide not to enter into a consent agreement, the ROC should proceed with its complaint-handling process by referring these cases to OAH. 1.3 The ROC should develop and implement policies and procedures for escalating discipline for contractors with prior complaints that resulted in substantiated violations, to ensure that licensees with multiple substantiated complaints or a history of substantiated complaints receive appropriate discipline. 1.4 The ROC should develop and implement guidelines for determining whether complaints received fall within the statutory 2-year time limit for ROC jurisdiction. Office of the Auditor General page 21 State of Arizona page 22 FINDING 2 The Registrar of Contractors (ROC) should shorten its process for resolving complaints where it issues citations, which includes some complaints that proceed to formal hearings and/or receive disciplinary action. The ROC’s complaintresolution process for these complaints is lengthy and allows complaints to remain unresolved for long periods of time. Lengthy complaint handling can negatively affect the public by allowing contractors who violate statute to continue working when they should not, providing misleading information to consumers who may be seeking to hire contractors, and prolonging the period until property owners receive needed repairs. To better protect the public, the ROC should address factors that result in lengthy complaint processing. Specifically, the ROC should determine whether to close a complaint or issue a citation based on its verification of whether or not the contractor has resolved problems identified in the corrective work order, as recommended in Finding 1 (see pages 9 through 21). The ROC should also explore and implement ways to incentivize contractors to resolve problems before a citation is issued, monitor complaints as they move through the process, add time frames for key steps in the complaint-resolution process, and develop and implement policies and procedures to ensure that discipline is imposed on licensed contractors in a timely manner. ROC should streamline complaint-resolution process Lengthy complaint-handling process does not protect public The process the ROC uses to inspect and resolve complaints involving licensed contractors who perform poor workmanship, abandon contracted work, or fail to pay a subcontractor is discussed in detail in Finding 1 (see pages 9 through 21). This process provides contractors with multiple opportunities to resolve complaints, some of which are required by statute. These opportunities allow faster resolution of complaints because once the complaint is resolved, the contractor and complainant do not need to go through the entire complaint process (see Figure 1, page 10). However, if a complaint cannot be resolved without issuing citations, addressing matters in a formal hearing, and/or the ROC’s ultimately disciplining the contractor, complaints can remain open for 1 year or more. Lengthy complaint resolution may put the public at risk because contractors who do not meet workmanship standards or abandon work can continue to work undisciplined, and information available to the public about these licensees may be misleading. Complaints that result in discipline undergo lengthy process— Although the ROC closes some complaints automatically when the complainant does not pursue them, complaints that result in the ROC issuing a citation and eventually disciplining the contractor can remain open for a long time. The ROC has an involved process for handling complaints when the complainant requests that the ROC issue a citation. This process includes a number of steps required by statutes that govern administrative procedures. In such cases, these steps include sending the citation to the contractor and allowing the contractor time to respond; scheduling and conducting a formal hearing with the Office of Administrative Hearings (OAH); giving the administrative law judge time to issue a ruling and the ROC time to decide whether to accept, modify, or reject the judge’s order; and handling any requests for rehearing or appealing the decision. If a workmanship complaint undergoes all possible steps in the process including proceeding to a hearing, rehearing, and appeal, and all parties take advantage of the full number of days allowed, the complaint will remain open for 240 days—225 of which are set by statute—plus the time spent on steps that do not have time limits.1 In addition, for cases the ROC closes prior to a hearing, ROC 1 The 225 days established by statute include 60 days for OAH to hold a hearing, as required by statute. However, for fiscal year 2013, Laws 2012, Ch. 298, §4, requires the OAH to hold hearings as soon as reasonably possible. Office of the Auditor General page 23 policy allows homeowners an additional 180 days to reopen the complaint if the problems were not addressed. The resulting total 420 days that can elapse before a complaint is finally resolved does not include any time that would be required for the homeowner to apply for and receive payment from the Recovery Fund for a claim associated with the complaint. The complaint-handling process can take much longer than the time allowed by statute and policy. Although the full process can be long even in the best of circumstances, in actual practice, the process can take much longer than the time allowed by statute and policy. To understand how the ROC’s complaint process plays out in actual cases, auditors reviewed ten complaints that involved a citation and that were closed in fiscal year 2011.1 Based on this review, auditors identified many factors that affected the length of time to resolve these complaints. Some of these factors are outside the ROC’s control, such as contractor bankruptcy or the time a complaint is with the OAH for a formal hearing. However, other factors that lengthen the resolution of a complaint are a function of the ROC’s process. For the ten complaints auditors reviewed, the ROC quickly performed some of the initial parts of the process before issuance of the citation, such as inspecting allegations of poor workmanship, but once a citation was requested, individual steps in the complaint process from that point forward took a long time to complete. This resulted in an overall time frame that was excessive for handling some complaints. Here are two examples: • Complaint open for approximately 18 months—In February 2009, the ROC received a complaint about a contractor who refurbished a pool poorly. Within 16 days of receiving the complaint, the ROC had inspected the pool, identified instances where the contractor did not meet workmanship standards, and issued a corrective work order. In April 2009, the property owner requested that a formal citation be issued to the contractor because the work had still not been repaired. However, it was not until July 2009, 92 days later, that the ROC issued the citation to the contractor. The citation was sent to the OAH, and a formal hearing was held in November 2009, at which an administrative law judge issued a recommended order to (1) suspend the contractor’s license until the ROC received and accepted written proof that the contractor had met the terms of the corrective work order; (2) impose a civil penalty to be paid within 30 days of the effective date of the ROC’s order, with the license to be automatically revoked if the penalty was not paid by the deadline; (3) not suspend the license if the contractor properly accomplished the corrective action on or before the effective date of the ROC’s order; and (4) establish an effective date 40 days after the ROC’s order. The ROC adopted the order 29 days later. After the ROC adopted the OAH’s order, the complaint was further delayed nearly 4 months when the contractor requested a rehearing. Although the OAH denied the rehearing request, the ROC took more than 1 month to 1 Auditors reviewed complaint cases that closed in fiscal year 2011 in order to observe the results of the ROC’s complaint process in actual cases. Specifically, auditors reviewed ten cases that closed prior to a citation and ten cases that closed after receiving a citation. A citation is a legal document from the ROC to a contractor listing the alleged violations of statute and potential disciplinary actions based on the violations. State of Arizona page 24 adopt the OAH’s denial of the rehearing request. In its order adopting the OAH’s recommended denial, the ROC also adopted the other recommendations regarding the suspension, civil penalty, and 40-day period in which the contractor could correct the workmanship problems before the order would become effective. Finally, it was not until 32 days after the end of the additional 40 days, in August 2010—about 18 months after the complaint had been received—that the ROC recognized that the contractor had not provided proof of correcting the workmanship problems within the 40-day period, and entered the discipline into its data system. • Discipline delayed for nearly 11 months—In August 2009, the ROC received a complaint about a licensee who had accepted $1,000 from homeowners to install carpet, but had taken the money and abandoned the project. Because the complaint alleged abandonment, the ROC did not conduct an inspection. Consistent with its process, the ROC sent a letter encouraging the contractor to resolve the problem and informing the homeowners that if the problem was not resolved, the homeowners could, within 20 days of the letter, request the ROC to issue a citation to the contractor. After 15 days, on September 2, 2009, the homeowners notified the ROC that they had still not heard anything from the contractor and requested that the ROC issue a citation. The ROC cited the contractor for work abandonment at the end of October 2009. Although the contractor failed to respond to the citation, which according to A.R.S. §321155(B) is an admission of guilt, the ROC did not issue a decision and order until February 2010. The order stated that the license would be revoked in April 2010. However, even though the order was not provisional and therefore the ROC did not need to spend time determining whether the contractor had complied with the terms of the order, the ROC did not enter the discipline into its data system until 148 days later, in September 2010, which, according to the ROC’s practice, is the actual discipline effective date. The ROC did not enter discipline into its data system until 148 days after deciding to revoke the contractor’s license. Lengthy process negatively impacts the public—Delays in resolving complaints can negatively affect the public in several ways. Specifically: • Problem contractors continue working undisciplined—Having complaints remain open for a long time puts the public at risk because it allows licensees who are the subject of valid complaints to continue working for long periods of time without receiving any discipline. It also potentially allows problem contractors to continue performing poor workmanship. • Public information is inadequate to inform consumer and licensee choices—When up-to-date information on the status of a license is not available, consumers are at risk of making uninformed hiring decisions. For example, the information provided by the ROC’s call center and on its Web site reflects only discipline that has been entered into its data system and that the ROC considers effective. Therefore, the ROC’s delay in entering discipline into its data system can result in consumers not having access to up-to-date Office of the Auditor General page 25 complaint and discipline information on licensees. Also, because the ROC’s public information can inform hiring decisions of both consumers seeking to hire licensed contractors or contractors seeking to hire subcontractors, delays in updating its disciplinary information can allow a contractor whose license the ROC has recently decided to revoke or suspend to be hired because the information is not yet reflected on the Web site. • If a licensed contractor files for bankruptcy while a complaint is being processed, federal law prevents the ROC from citing the contractor. Homeowner relief is delayed—When a complaint involves a citation and hearing, a homeowner may wait for an excessive amount of time before the workmanship problems in the complaint are repaired or until he/she receives financial relief from the Recovery Fund to pay for repairs. For example, a homeowner receives Recovery Fund monies only after the complaint process is completed, the licensee is disciplined, and the Recovery Fund claim is processed (see Other Pertinent Information, pages 47 through 52). This time is in addition to the time that a homeowner may have spent working with the contractor to address problems prior to submitting a complaint. In addition, the lengthy complaint process may allow for case complications that arise to affect the complaint process and further prolong homeowner relief, such as when a licensed contractor files for bankruptcy. If a licensed contractor files for bankruptcy during the time a complaint is being processed, federal law prevents the ROC from citing the contractor and further complicates a property owner’s ability to access the Recovery Fund. In two of the ten complaints auditors reviewed that involved a citation, the contractor declared bankruptcy during the complaint process. In one of these complaints, the complainant obtained a court-ordered relief of stay, which allowed the ROC to continue with the process and discipline the contractor. Obtaining the court order took 14 months, considerably lengthening the time to resolve the complaint. The other complaint was one of 20 filed against a contractor during the 6 months prior to the contractor’s declaration of bankruptcy. In this case, the complainant did not obtain a court order and the complaint was closed without discipline. ROC should take additional steps to resolve complaints more quickly One of the improvements to the ROC’s process recommended in Finding 1 may help shorten the time it takes for the ROC to resolve complaints (see pages 9 through 21 for more information). Specifically, the ROC should determine whether to close a complaint or issue a citation based on its assessment of whether or not the contractor has resolved the workmanship issues identified in the corrective work order. This should help to reduce the overall time spent on complaints that are reopened because the licensed contractor did not resolve the problem. Auditors identified an additional four ways the ROC could resolve complaints more quickly: State of Arizona page 26 • • Seek authority to incentivize contractors to resolve problems more quickly— To help improve complaint resolution timeliness, the ROC should seek statutory authority to incentivize contractors to resolve workmanship problems identified in the corrective work order more quickly. As discussed in Finding 1 (see pages 9 through 21), the ROC’s complaint process provides multiple opportunities for contractors to resolve issues, such as poor workmanship, at any point in the complaint-resolution process, even after a formal hearing in some cases. Some states have created financial incentives for contractors with verified workmanship complaints to address complaints quickly. For example, according to the Nevada State Board of Contractors, if a contractor does not fix the verified workmanship issue(s) in a timely manner, a disciplinary hearing is scheduled and the contractor may then face fines and pay investigative costs, plus restitution to the homeowner. In addition, South Carolina may require licensees who have violated state laws to pay for complaint investigation and prosecution costs, in addition to other sanctions, unless the licensees resolve the complaints quickly. The ROC could similarly expedite complaint resolution if it could require contractors who fail to fix workmanship problems identified in the corrective work order prior to being issued a citation to pay fees to cover complaint resolution costs. For example, the OAH billed the ROC more than $400,000, or an average of $619 per complaint, according to the ROC’s calculation, for complaints sent to the OAH for a formal hearing from July 2011 through April 2012. The ROC should seek statutory authority to charge this and other complaint-handling costs to the contractor if workmanship problems identified in a corrective work order have not been addressed before a citation is issued. The ROC would also need to develop and implement a mechanism to identify and track costs associated with specific complaints. In Nevada, contractors who do not fix verified workmanship issues on time may be required to pay investigative costs. Establish a system for monitoring open complaints—The ROC should use its existing reports or develop other mechanisms, as appropriate, to monitor open complaints and ensure they move through the complaint process and do not remain inactive for long periods of time. For example, one complaint that auditors reviewed was apparently lost and was not worked on for 242 days after being reassigned to a different staff member, according to the case notes.1 According to the ROC, although it has experienced difficulty in obtaining adequate complaintmanagement reports from its data system (see Finding 3, pages 31 through 45, for additional information), it is able to create some reports that would allow it to track complaints. The ROC could use these reports for tracking purposes, similar to some other state agencies. For example, Arizona Medical Board officials reported that they have a variety of reports that they use to monitor the progress of open complaints, such as reviewing an open case report that identifies complaints that have not been actively investigated for 30 days or more. At their weekly investigator meeting, they then create a plan to move inactive cases forward. The ROC should better ensure that complaints are not lost or delayed by similarly tracking and monitoring the progress of its open complaints. 1 Auditors reviewed this case as one of four cases reviewed to assess the information the ROC provides to the public. Office of the Auditor General page 27 • Statutes establish some steps’ time frames, but the ROC does not have formal time frames for other key steps. Develop time frame goals for key steps in process—To further improve timeliness, the ROC should develop and implement written time frames for completing key steps in its complaint-handling process. Statutes establish time frames for some steps in the process, such as the time required for contractors to respond to a citation. However, the ROC does not have formal time frames for completing some additional key steps, such as issuing citations and issuing orders of suspension or revocation when contractors do not respond to citations. Formal, written goals might help the ROC ensure that key steps in its complainthandling process do not take longer than they should. In the cases auditors reviewed, the ROC did not always meet the informal goals or time estimates ROC staff provided. For example, the ROC did not consistently meet its informal goal of issuing citations within 60 days once the complaint is received by its legal department. In the ten cases reviewed, the ROC met the goal in three cases but took between 68 and 129 days to issue citations in the other seven cases. According to ROC staff, the ROC has addressed the backlog that caused the delays, and as of November 2012, the legal department was issuing citations in an average of 32 days. Similarly, the ROC took longer than the 30 to 35 days staff estimated it should take to suspend or revoke a license in cases where a contractor is deemed to have admitted the acts charged by the complaint because he/she did not respond to the citation within 10 days. In the six such cases auditors reviewed, the ROC took from 51 to 138 days to issue the orders. • State of Arizona page 28 Continue efforts towards more timely data entry of disciplinary actions— The ROC has taken steps toward ensuring that its disciplinary actions are entered into its data system and posted on its Web site more quickly. According to the ROC’s disciplinary orders sent to contractors, discipline against a licensee becomes effective 40 days after the date of the order. The ROC allows this additional time so the parties can file for appeal or rehearing as allowed by A.R.S. §§41-1092 and 12-904. However, in the eight cases auditors reviewed that resulted in discipline, discipline was not entered into the data system until 32 to 162 days after the stated effective dates. In two of these cases, the ROC’s orders were provisional, which required time after the effective date for the ROC to determine whether the contractor and/or complainant had submitted proof that the problem had been resolved prior to the effective date. The other six cases involved orders where the contractor did not respond to the ROC’s citation, and in those cases the ROC can implement its orders immediately on the effective date. Because the ROC does not consider the discipline to actually become effective until it is entered into the data system, data entry delays result in delaying the disciplinary action. ROC officials reported that the delays in data entry were mainly caused by temporary staff shortages, and as of March 2012 ROC had assigned a temporary staff person to enter disciplinary information into the ROC’s data system. Auditors selected two licenses that were revoked in August 2012, and in both cases, the disciplinary action was entered into the data system less than 3 weeks after the effective dates, which was an improvement compared to the complaints closed in 2011 that auditors reviewed. In addition to continuing these efforts, the ROC should develop and implement policies and procedures to help ensure that licensee discipline is entered into its data system in a timely manner. Recommendations: 2.1 The ROC should expedite complaint resolution by encouraging contractors to address concerns more quickly. Specifically, the ROC should: a. Request a statutory change that would allow it to charge fees to cover the costs of processing the complaint if poor workmanship is not repaired prior to issuing a citation; b. Develop and implement a mechanism to identify and track costs associated with processing specific complaints if statute is changed to give the ROC permission to charge these costs to contractors; and c. Charge licensed contractors who are found to have committed a violation the costs for processing valid complaints if statute is changed to give the ROC permission to do so. 2.2 The ROC should use complaint-management reports from its data system, develop and implement new reports, or develop and implement other mechanisms, as appropriate, to track and monitor open complaints. 2.3 The ROC should develop and implement time frames for completing the key steps in its complaint-handling process. The time frames that the ROC should develop and implement include, but should not be limited to: a. Time frames for issuing citations; and b. Time frames for issuing suspension or revocation orders in cases where contractors do not respond to citations. 2.4 The ROC should modify its complaint-handling process to help ensure complaints are resolved within the time frame it establishes. 2.5 The ROC should develop and implement policies and procedures to ensure that licensee discipline is entered into its data system in a timely manner. Office of the Auditor General page 29 State of Arizona page 30 FINDING 3 The Registrar of Contractors (ROC) relies on its data system to help perform many of its core functions, but system problems hamper its ability to perform some of its responsibilities and statutorily mandated functions. In March 2010, the ROC replaced an aging information management system with a new one. However, the parties responsible for system implementation did not take a number of key actions, resulting in numerous problems, including inaccurate data and some inefficient practices. The ROC has taken some actions to address the problems but has been unable to take others because of concerns the actions could lead to new problems. Some of the system’s problems prevent the ROC from performing some statutorily mandated functions. For example, disciplinary actions taken against contractors who operate under another license name after their license is suspended or revoked have decreased because the system cannot identify such contractors. The ROC plans to replace its system with one that better meets its needs. To avoid future problems, the ROC needs to continue its efforts to fix the inaccurate data and take key steps during implementation of the new system to ensure it efficiently and effectively supports ROC’s functions. Problems with data system hamper ROC’s ability to perform core functions ROC installed new data system in 2010 In March 2010, the ROC replaced the computer system it had been using to help it to perform many of its core functions, such as licensing contractors, investigating complaints about licensed and unlicensed contracting, processing Residential Contractors’ Recovery Fund (Recovery Fund) claims, and providing information to the public about contractors and complaints, with a new system called the Registrar of Contractors Information Management System (ROCIMS) (see textbox). ROCIMS components The system includes a database, some process tracking, and management reports regarding: • • • • Licensing applications and renewals Complaints against licensed contractors Investigations of unlicensed contractors Recovery Fund claims In addition, the system includes an online application form for license renewal. Source: Auditor General staff analysis of ROCIMS and the ROC’s Web site. Implementation of ROCIMS was part of a state-wide effort to implement an electronic licensing system that could be used by any licensing agency in Arizona. Since at least 2001, the ROC had sought to develop a browser-based system for its licensing and regulation activities. The development of ROCIMS, which was begun under prior agency management in January 2007, resulted from the ROC’s effort to acquire and implement a system that would address its critical business functions such as licensing, inspections, and fiscal management. Because other state agencies were also making requests for similar systems, the State’s Government Information Technology Agency (GITA) developed a strategic plan in an attempt to identify an electronic licensing solution that could be used by multiple state entities.1 GITA’s plan 1 As of 2011, GITA merged with two other large technology groups and is now known as the Arizona Strategic Enterprise Technology Office (ASET), which is part of the Arizona Department of Administration (ADOA). Office of the Auditor General page 31 identified a Web-based, commercial, off-the-shelf licensing information management system from CAVU Corporation (CAVU/IDS).1 In January 2007, GITA and the State’s Information Technology Authorization Committee approved the ROC’s request for its ROCIMS project. The ROC became the first agency in Arizona state government to use the CAVU system. As of April 2013, one other state agency—the Arizona Supreme Court’s Administrative Office of the Courts—is using the system, but on a more limited basis than the ROC. The Department of Liquor Licenses and Control has attempted to implement the same system, but reported that it has still not done so despite being in its third year of testing the system and the related data conversion. The Arizona Department of Financial Institutions stated that it made payments for and preparations to implement the system, but as of April 2013, implementation had been placed on indefinite hold. ROCIMS was not properly implemented, which has affected the ROC’s efficiency A number of key actions were not taken prior to, during, and after implementation of the new information management system, which has resulted in several problems with the system and has created inefficiencies in some ROC processes. Responsibilities for the various aspects of system implementation were shared among several parties, which complicated the new system’s implementation. Additionally, the successful implementation of ROCIMS required various actions that were not properly followed. Specifically, the ROC’s business practices were not evaluated and modified to match the new system’s capabilities, an adequate data migration plan was not developed and executed, the system was not adequately tested prior to implementation, and training of ROC staff on the new system was inadequate. In addition to data inaccuracies resulting from the poor implementation, some security features were not being used appropriately. The ROC has taken some steps toward resolving the problems with ROCIMS, but has been unable to take some additional steps because of concerns about other problems that might result. An April 2011 postimplementation assessment by the software vendor identified a number of concerns. Many of the problems with ROCIMS’ implementation that are highlighted in this report were also identified in previous assessments of the system that, according to ROC officials, were conducted at the ROC’s request. For example, an April 2011 post-implementation assessment by CAVU/IDS identified a number of concerns, including problems with communication during and after implementation and problems with data migration that resulted in incorrect, duplicated, and commingled data. A consultant hired by the ROC identified similar problems in a September 2011 report. The consultant found that errors in ROCIMS’s implementation resulted from the requirements not being sufficiently defined as well as insufficient data analysis. 1 In April 2010, Iron Data, LLC acquired CAVU Corporation and now offers the CAVU suite as part of its product offerings. Iron Data reported that over 30 agencies in 17 states use CAVU to manage back office operations and online regulatory portals. State of Arizona page 32 The report also criticized the project management and software development life cycle management provided during implementation. Implementation responsibilities shared among multiple parties, complicating implementation—GITA, its Web portal contractor, and the ROC each shared responsibility for ROCIMS implementation under the contractual arrangement set forth in the project’s Statement of Work signed in July 2008 (see textbox). This arrangement established NIC Inc. (NIC) as an intermediary between the ROC and the software vendor, CAVU/IDS. Specifically, NIC, as part of its responsibility for the state Web portal and because of its technical resources, contracted separately with CAVU/IDS to provide the software for ROCIMS. Under the terms of that contract, NIC—not the ROC—was to communicate with CAVU/IDS for professional services and/or technical support. This was problematic for two reasons. First, it meant that ROC staff were not communicating directly with CAVU/IDS staff, limiting their ability to explain their requests, ask follow-up questions, and obtain clarification of technical support information. Second, because NIC had to pay CAVU/IDS for its assistance, NIC staff may have been reluctant to make requests for changes and support. Specifically, according to CAVU/IDS’ April 2011 post-implementation assessment report, the cost associated with this arrangement led to a lack of involvement in the project from CAVU/IDS. ROCIMS implementation roles by party GITA1 • • • Develop strategic electronic licensing plan and coordinate selection of the CAVU/IDS solution Coordinate with NIC and the ROC to develop the ROCIMS Statement of Work Engage NIC to subcontract with CAVU/IDS for the electronic licensing solution and support NIC • • • • • Implement the subcontract with CAVU/IDS Provide staffing resources, including project management, software configuration, and data migration Manage project schedules, action items, task assignments, and deliverables schedules Provide a training program to meet the ROC’s internal training requirements Develop a testing plan and procedures ROC • • • Designate a project manager to work with GITA and the NIC project manager Provide business and program expertise to allow NIC to develop an accurate testing plan Complete all required testing prior to acceptance and deployment into use 1 As of 2011, GITA merged with two other large technology groups and is now known as the Arizona Strategic Enterprise Technology Office (ASET), which is part of the ADOA. Source: Auditor General staff analysis of July 2008 ROCIMS Project Statement of Work and information provided by ADOA officials. Office of the Auditor General page 33 Key implementation steps inadequately performed—Effective implementation of a new information management system requires a number of key steps that were not properly followed during the ROCIMS implementation. Specifically: According to a 2004 article, data conversion is one of the most complex and risky activities in implementing a new system. • Business practices not evaluated prior to selection of system or modified appropriately afterwards—Modification of some business practices is important when implementing a commercial, off-the-shelf solution because such systems generally only allow limited customization and may not be able to be easily modified to conform to a particular entity’s existing business practices. However, although the Project Investment Justification (see textbox) submitted to GITA indicated that the ROC’s business practices were to be reengineered, according to ROC officials, the former ROC staff and contractor personnel responsible for selecting and preparing to implement the new system did not adequately evaluate the ROC’s business practices prior to selecting the system or modify them as needed before ROCIMS was implemented. For example, the Project Investment Justification— new system’s data entry screens Document with standard information did not match the order of about a proposed IT project for review information on the ROC’s license by GITA or ASET. application forms, but the forms Source: ADOA-ASET description of a Project were not redesigned to align with Investment Justification document. the new system. • Data migration plan not developed and followed to ensure successful transfer of data to new system—According to a 2004 article on implementing commercial, off-the-shelf software, data conversion is one of the most complex and risky activities in implementing a new system.1 However, no documented data migration plan was developed or followed to ensure the successful transfer of data to the new system. The only written guidance used during the ROCIMS data migration process was a standard conversion guide created by CAVU/IDS that was not specific to the ROC. Although the guide identified key questions to answer as part of developing a data conversion strategy, noted the importance of avoiding duplicate records, and stated that the conversion plan should accommodate the time needed to perform final testing, it did not contain specific steps for guiding the data migration process. The ROC stated there were no additional data transfer plans created and that it relied primarily on NIC to guide the data migration. Yet, because the ROC had the detailed knowledge of its previous system and the data it contained, the ROC also had a responsibility to assist with the data migration. According to NIC, it met with the ROC and decided what data would be migrated for successful use of the new system, but these decisions were not formally documented, and the data that the ROC wanted to migrate to the new system was not adequately defined until after ROCIMS was implemented and in use. 1 Thomas, G.A., & Jajodia, S. (2004). Commercial-off-the-shelf enterprise resource planning software implementations in the public sector: Practical approaches for improving project success. Journal of Financial Management, 53(2), 12-19. State of Arizona page 34 • Testing of the new system inadequately performed—Best practices for acquiring and implementing information technology (IT) systems include developing a formal test plan and conducting proper testing to ensure that the system operates in line with expectations. The statement of work for the project defined responsibilities for testing, including roles for both NIC and the ROC. However, ROCIMS was implemented without a formal test plan, guidelines, or methodologies, and it was not tested to ensure that it was working as intended. For example, according to ROC IT staff, although limited testing was performed to determine whether the system accepted new information, controlled test cases were not created to ensure that existing data worked in the new system. • Training on the new system was insufficient—Best practices for IT management suggest that training should be planned for both system users and the IT staff expected to maintain the system. The system’s Statement of Work specified that NIC would provide a training program to meet ROC’s internal training requirements, which were to be finalized by the ROC and NIC during implementation of the new system. However, only limited training was provided to ROC staff on how to use ROCIMS when the system was implemented, and during the audit, ROC IT staff still lacked adequate knowledge of the system and its functionality. ROCIMS has data errors and other problems—Similar to the previously discussed system assessments, the audit found commingled, duplicated, and inaccurate data; operational inefficiencies; and limits in what ROCIMS can do. In addition, some key security features were not being fully used. Specifically: • Duplicated, missing, and inaccurate records—ROC officials reported that of the more than 544,000 contact records in ROCIMS, more than 112,000 records were duplicates as of August 2012. According to ROC officials, after switching to the new system in March 2010, staff began noticing that the system had duplicate records for some licensed contractors, while other licensed contractors were not listed in the new system at all. Also, according to an April 2011 CAVU/ IDS assessment of the system, in some instances, the system contained only the “doing business as” (DBA) name of the entity instead of the legal name of the licensed individual or business. Lack of the legal, licensed name makes it difficult for the ROC to properly identify licensed contractors and associated licenses, and may affect the ROC’s ability to effectively notify the responsible licensee of certain enforcement actions the ROC is taking. In ROCIMS, more than 112,000 of the ROC’s more than 544,000 contact records were duplicates as of August 2012. ROC officials also reported that an additional 18,000 or more license records either lacked information or contained incorrect information regarding the individuals associated with the licenses. In addition, in May 2012, the ROC identified more than 22,000 complaint records that contained inconsistent information regarding complaint closure. Specifically, nearly 12,000 records had a closed date, but did not have a status of closed, while another more than 10,000 records had a status of closed but had no closed date. Office of the Auditor General page 35 • In response to a ROC employee inquiry, the system returned ten results with different iterations of one individual licensee’s name. Operational inefficiencies—According to the ROC, employees do not trust ROCIMS data, so they research and confirm information for each record that they review, update, and investigate. In addition, when staff find duplicate records for a contractor, they must manually determine which record is the most complete before performing work on the license. This can be time consuming and confusing. For example, auditors observed a ROC employee looking up an individual licensee’s name. The system returned ten results with different variations of the individual’s name. The employee then accessed each record and reviewed information before selecting the correct record. The employee reported that it took approximately 3 to 5 minutes per record to conduct this review. Additionally, the ROC has had to develop various workarounds to resolve problems that its staff encounter in working with ROCIMS. For example, the system’s search function does not provide accurate information because some of the data is incorrect. As a result, ROC staff developed a workaround that consists of an additional search function that allows ROC staff to search for data in ROCIMS as well as in the old data that was contained in the prior system. According to the April 2011 CAVU/IDS assessment, workarounds have caused frustrations because, by not solving the problem directly, they complicate other processes and cause problems in other areas. State of Arizona page 36 • Limits in system functions—ROCIMS does not have a field for capturing a licensee’s DBA information, which would allow the ROC to track the history of business names under which the contractor is “doing business as.” Instead, the ROC uses an address-type field to record DBA titles and information. Since this information must be manually completed, DBA historical information has not been entered for all businesses for which it would be applicable, and even for those for which it has, searching and identifying all possible names that have been used is difficult. This can lead to two problems. First, as previously mentioned, not having a licensee’s correct information can affect the accuracy of legal documents. In addition, the ROC may be unable to identify or provide the public with information on a contractor who has used previous business names. For example, consumers who use the ROC’s public information to try to make informed choices about which contractor to hire may not be able to identify those who have a history of poor workmanship or those who have changed their business name following mergers, bad press, or in order to try to avoid fines or warranty work. • Some security functions underused—Prior to the audit, the ROC was not using some of the new system’s security functions designed to secure the system from unauthorized access. Because ROCIMS is a Web-based system and does not require a user to be on the ROC’s internal network to use it, the system can be accessed from anywhere. Thus, it is even more important that the system—and the information it contains—be adequately secured. However, ROCIMS was not set up with some common password requirements such as requiring users to change their passwords regularly. During the audit, the ROC set up this feature. In addition, some user accounts were set up so multiple people could use them, instead of having all accounts established individually so users could access only the components and information they needed to do their jobs. ROC IT staff reported that they had been unable to obtain information from the ROC’s contractors about what the user access permissions would allow, but they obtained the information during the audit and took steps toward correcting these weaknesses. ROC has taken some actions to address ROCIMS problems but has been unable to take other needed actions—In addition to the steps taken to improve its use of security management features, the ROC has made several efforts to address the problems with ROCIMS. In addition to initiating the previously discussed assessments of the system, the ROC has: • Hired new IT personnel—In early 2012, the ROC hired a new chief information officer to manage its IT function. The ROC also hired a business analyst to perform various tasks such as evaluating and documenting the ROC’s business processes and providing support for IT projects to ensure that business requirements are clearly communicated. • Developed a data cleanup plan—During the audit, the ROC drafted a data cleanup plan to address incorrect data in the system. ROC officials indicate that their approach includes training staff on standard processes for inputting data, manual data clean-up efforts, and scripted data clean-up efforts (see textbox). As of April 2013, the ROC reported that it had received proposals from eight vendors for a contract to write scripts to address instances where there are a large number of records with common issues, and to write test plans and design and build a data warehouse for Script—A series of storing the cleaned-up data. Bringing in a instructions that tells a contractor to work with its staff on addressing computer program how to the ROCIMS problems was one option perform a specific procedure. recommended in the external consultant’s September 2011 report. The ROC has hired a new chief information officer and a business analyst. Other actions needed to address ROCIMS problems have not been taken because of the ROC’s concerns that they could lead to more problems. Specifically, although two updates to the system have been available from the software company since June 2011, the ROC has not installed these updates. Hardware and software vendors periodically issue updates, or patches, to their products to correct security vulnerabilities and to improve usability and performance. According to CAVU/IDS, implementing the two updates could fix many of the reported problems that the ROC is experiencing with ROCIMS. Additionally, CAVU/IDS reported that an upcoming update should provide a screen wizard to help with validation checks on data that is entered. However, according to the ROC, it has not implemented these updates because the information provided with the updates does not contain enough detail to design appropriate tests. In order to determine whether the updates would in fact Office of the Auditor General page 37 benefit the ROC, it would need to implement them in a test environment first and determine whether they led to any problems, and then, if appropriate, implement them in ROCIMS. In addition, ROC officials reported that past updates had led to problems. For example, they stated that when ROCIMS was implemented, it did not include key relationships between individual people and their associated licenses. An update corrected this problem for the data migrated from the ROC’s old system, but it overwrote the key relationships ROC staff had manually entered for new licenses issued after ROCIMS’s implementation. ROC unable to use some system functionality and cannot effectively perform some statutorily required functions In addition to the data accuracy and system problems, the ROC is unable to use one of the system’s primary features and is unable to effectively perform some statutorily required functions. First, the ROC is unable to use the system’s online application feature, which is one of the system’s primary purposes and allows applicants to apply for new licenses online. As a result, the ROC has had to continue to process new license applications manually. Additionally, the ROC cannot effectively perform two statutorily required functions related to preventing disciplined contractors from continuing to practice under another license name and suspending the licenses of the responsible parties for which the Recovery Fund has paid a claim. Finally, because of system limitations and data inaccuracies, the ROC may not be able to rely on ROCIMS to provide accurate licensee information to the public. ROC unable to use ROCIMS’s online new license application feature or rely on renewal-payment processing—One of the CAVU system’s The ROC is concerned that enabling the new license online application feature before data system problems are resolved could exacerbate system problems. State of Arizona page 38 main benefits is an online license application feature, which allows license applicants to apply for and renew licenses online. However, the new license application system feature has not been set up and activated in ROCIMS, although the renewal feature is activated. According to the ROC, it is concerned that enabling the new license application feature before data system errors are resolved could exacerbate the problems it experiences with the system. As a result, the ROC continues to require contractors and licensees to submit paper applications for new licenses, and ROC staff must process the submitted applications manually. Problems with ROCIMS have also created other inefficiencies: • When ROC staff manually enter information on new license applications into ROCIMS, the same information must be entered three different times into separate parts of the system. • ROCIMS has not been set up with edit checks to help ensure information entered into the system is complete and accurate. As a result, ROC staff manually complete a checklist to ensure that all required information for an application has been received before the application is entered into the data system and the 60-day licensing time frame prescribed by the ROC’s administrative rules and required by statute begins. • ROC staff manually check each license renewal payment to ensure that ROCIMS processes it correctly. In February 2012, the ROC discovered that some contractors who paid their renewal fees online were charged twice. Although the vendor has issued an update to its software to address the problem, the ROC has not implemented the update because of concerns about the impact the update might have on other changes the ROC has made to the system to address other problems. As a result, the ROC reported that the time it took to process a new license application increased after implementation of ROCIMS. According to the ROC, prior to ROCIMS’s implementation, it could generally complete the new license process within 60 days, as required by its administrative rules. By comparison, according to the ROC’s data analysis of licensing time frames, the ROC processed 18.1 percent of new license applications in more than 60 days in fiscal year 2011, after the ROC began using ROCIMS. For fiscal year 2012, the ROC had reduced the number of new license applications processed in more than 60 days to 7.9 percent. However, a ROC official expressed concern that if the number of new applications increased, the number processed in more than 60 days would also increase. In fiscal year 2012, the ROC reported that it processed 2,922 new license applications, approximately 21 percent fewer than the 3,704 new license applications it reported processing in fiscal year 2011. Not only do these delays affect the length of time that applicants must wait for a licensing decision, if the ROC takes longer than 60 days to process a new license application, the ROC must refund the license application fees and pay penalties to the State General Fund. Specifically, A.R.S. §41-1077 requires agencies that do not meet the license processing time frames established in their administrative rules to refund applicant fees and pay penalties to the State General Fund. In fiscal year 2010, the ROC paid only $288 in penalties to the State General Fund. However, because of the problems that the ROC has experienced in timely processing of new license applications, it paid $6,450 in penalties in fiscal year 2011 and reported paying $973 in fiscal year 2012. Although statute requires that the ROC also refund the application fees if it takes longer to process a license application than the licensing time frame prescribed by the ROC’s rule, which is 60 days, according to the ROC, it did not have a process for doing so (see Sunset Factor 2, page 56, for more information). During the audit, the ROC began working toward developing such a process. As of April 2013, the process was not fully developed, but ROC officials reported that the ROC had issued refunds in a small number of cases. Because of problems in timely processing of new license applications, the ROC paid penalties to the State General Fund. ROC unable to effectively perform two statutorily required functions— Because it cannot rely on the data in ROCIMS or the system is unable to effectively present the needed data, the ROC is unable to perform two statutorily required functions. Specifically: Office of the Auditor General page 39 It is difficult to accurately identify all individuals in ROCIMS whose licenses should be suspended until the amount paid from the Recovery Fund is repaid. • The ROC is not taking action required by statute to prevent disciplined contractors from continuing to practice under another license name. A.R.S. §32-1154(B) states that a contractor’s license can be revoked or suspended if the license’s list of individuals includes a person who is also named on a license already under suspension or revocation as provided in A.R.S. §321154(A)(21). To comply with this statute, the ROC must be able to identify all individuals named on a revoked or suspended license and then administer the same discipline to all related licenses. However, according to the ROC, it cannot identify all individuals whose other licenses should be disciplined when a license is suspended or revoked because records regarding key individuals associated with licenses are incomplete or incorrect in ROCIMS. ROC management reported that it has taken disciplinary action against only four contractors for this statutory violation between March 2010 and August 2012, compared to the estimated 100 to 150 disciplinary actions per year for this statutory violation before ROCIMS was implemented. Failure to comply with this statute could allow contractors with revoked or suspended licenses to continue performing contracting work under another license. • The ROC is not suspending the related licenses of the responsible parties for which the Recovery Fund paid a claim. A.R.S. §32-1139(B) states that if any amount is paid from the Recovery Fund, then the license of the contractor shall be automatically suspended until the amount paid from the Recovery Fund is repaid in full. According to the ROC, similar to the difficulty of identifying all individuals who should be disciplined when a license is suspended or revoked, it is difficult to accurately identify all individuals in ROCIMS whose licenses should be suspended until the Recovery Fund is repaid. As a result, the ROC reported that it is unable to identify and suspend all the licenses of the responsible parties who are related to licenses for which the Recovery Fund paid a claim; therefore, a contractor for whom the Recovery Fund has paid a claim may still be performing contracting work under another license. ROC may be unable to provide accurate information to the public— Finally, the problems with ROCIMS data reliability also result in the public’s potentially receiving inaccurate information. First, because of its inability to ensure it has taken appropriate disciplinary action against related licenses, the ROC may be unable to provide accurate information to the public about a licensed contractor’s status and complaint history because it cannot ensure it has identified all other names under which the contractor may be doing business. This could potentially result in consumers’ hiring suspended or revoked contractors even after checking the ROC’s Web site to determine if the contractor has an active license. Second, if a contractor’s license status in ROCIMS is incorrect because of the data inaccuracies that resulted when data was transferred from the old system to the new system, a homeowner may hire a contractor whose license was suspended or revoked but is not shown as such on the ROC’s Web site. State of Arizona page 40 ROC plans to replace ROCIMS but still needs to address ROCIMS-related implementation problems Although the ROC has reported its intention to replace ROCIMS with a new information management system, it should take a number of actions to address problems related to ROCIMS, which would also aid in the development and implementation of a new information management system. Specifically, the ROC has decided to develop a new information management system to better meet its needs. Regardless of whether it implements a new system or retains ROCIMS, to ensure its system efficiently and effectively supports the ROC’s functions, the ROC should continue with its efforts to correct its data, align its business practices with how its system operates, ensure its staff are trained on the system, and create processes for managing and updating its information management system. ROC intends to replace system—As of March 2013, ROC officials reported that they had decided to discontinue their agency’s efforts to use the ROCIMS system and plan to develop a new system to meet the ROC’s needs. It plans a phased approach, with the first phase—overall system architecture and the new and renewal licensing application process—to be implemented by June 30, 2014. In developing each phase of the new system, the ROC plans to establish project teams, hold regular working sessions, document the existing processes, identify opportunities to streamline processes, and document the workflows and business rules of the new system to ensure accurate processing. The ROC plans to implement the first phase of a new system by June 30, 2014. Replacing ROCIMS was one of the options presented in the September 2011 external consultant’s report. The external consultant indicated that although ROCIMS had the features needed to perform the tasks required of the ROC’s mission, errors in how ROCIMS was implemented made it ineffective and inefficient. The consultant concluded that the system could be sufficiently remediated to make it acceptable, but recommended three different options for the ROC to consider: • Continue efforts to work with the software vendor to remediate the system, and develop or purchase other software tools to perform functions that are not included in ROCIMS; • Contract with a third party to manage the remediation, including project management, software development, and program governance; or • Replace ROCIMS with either a customizable off-the-shelf system or a fully customized system designed specifically for the ROC. Finally, the consultant noted that regardless of whether the ROC replaces ROCIMS, it will need to ensure that the data housed within ROCIMS is accurate and that it follows sound practices for implementing ROCIMS changes or an entirely new information management system. Office of the Auditor General page 41 ROC should continue with its efforts to fix the inaccurate data— Correcting problems with the data in ROCIMS is a vital step in helping to ensure that the ROC is able to rely on its information management system to effectively and efficiently support its critical functions, whether using ROCIMS or a replacement system. In April 2013, ROC officials reported that the ROC was in the process of cleaning up and reconciling the data through training staff; manual data cleanup efforts; and hiring a vendor to write scripts for addressing issues where many records have common issues, write test plans, and design and build a data warehouse for storing the cleaned-up data. Additionally, as part of its data clean-up effort, the ROC should develop and implement a test plan to ensure that the data has been corrected and policies and procedures to ensure that the data remains complete and accurate. ROC should evaluate and redesign its business practices as needed—The ROC should further analyze and document its business practices and, Redesigning application forms could have reduced the number of times staff visit a data entry screen to enter information. where appropriate, redesign them to better meet its operational requirements and information management system’s capabilities, whether for ROCIMS or a replacement system. Although business process redesign should have been considered and completed before or as part of ROCIMS’s implementation, the ROC’s ongoing issues with the system illustrate the need for these activities. For example, in the licensing area, the ROC could have benefited by evaluating and redesigning its business practices by changing the order of information in its application forms to be better aligned with the order of ROCIMS’s data entry screens. This could have helped reduce the number of times staff visit a data entry screen to enter information. If the ROC replaces ROCIMS with a new system, it should similarly ensure that its business practices are aligned with the system’s operations to maximize efficiency. In addition, the ROC should ensure that any redesigned business practices are documented in policies and procedures and communicated and followed throughout the ROC, including providing training where needed. ROC should ensure staff are trained on ROCIMS or the replacement system—Best practices for IT management suggest that a training plan should be developed for both system users and the IT staff who are expected to maintain a data system. The plan should include information such as who will be trained, what they will be taught, and when training will occur. It should also provide for training of new staff as they begin using the system, and ongoing training to address changes to the system as it occurs. Therefore, the ROC should develop a training plan that ensures proper training is provided to both system users and IT staff and that new employees are properly trained on the system as they begin to use it. Training for both system users and IT staff should be relevant to their use of and responsibilities for the system. ROC should create processes for managing and updating its system—Regardless of whether it retains ROCIMS or implements a new information management system, an overall plan is needed to keep the problems the ROC has experienced with ROCIMS from recurring. Specifically, the ROC should devel- State of Arizona page 42 op and periodically update a project management plan that would address ROCIMS problems and/or prevent these problems from occurring in a new system. For example, the project management plan should identify the specific actions needed to address the data accuracy problems and ensure that fixes to the data are tested and working properly. Additionally, IT standards and best practices indicate that organizations should have a systematic, accountable, and documented process for testing and applying system updates as appropriate. As mentioned previously, the ROCIMS system vendor has issued recent updates intended to fix some of the problems the ROC has experienced. The ROC should evaluate and test these updates to determine whether it can safely implement them. If the ROC replaces ROCIMS with a new system, it should develop and follow appropriate processes to ensure that the new system is appropriately maintained and updated as necessary. The ROC should also incorporate and use appropriate security controls for ROCIMS or a replacement system. Specifically, it should develop policies and procedures regarding security controls and train IT staff on the proper operation of those controls so that the ROC has a way of managing and enforcing system and user security policies and procedures. During the audit, the ROC adopted a formal system development lifecycle (SDLC) methodology to guide its efforts for future IT system projects. An effective SDLC methodology is important to help better ensure the success of project implementation. SDLC is a conceptual model used in project management that describes the stages involved in an information system development project, from an initial feasibility study through maintenance and ultimate retirement of the completed application or system. In general, an SDLC methodology provides for a number of steps encompassing the planning, analysis, selection, design, testing, implementation, and maintenance of a system. It helps ensure that the right people are involved in the design and selection of the system, and that the system meets the business needs of the organization implementing it. Although the ROC did not have a methodology at the time of ROCIMS implementation, it developed and finalized an SDLC policy and procedure manual during the course of the audit. Auditors reviewed these documents and found them to contain the necessary elements to guide management, users, and efforts during the development and implementation of IT system projects and to be in line with IT standards and best practices. For future IT projects, the ROC should follow its system development lifecycle methodology policy and procedures. During the audit, the ROC adopted a formal system development lifecycle methodology to guide its efforts for future IT system projects. Office of the Auditor General page 43 Recommendations: 3.1 Regardless of whether the ROC decides to correct or replace ROCIMS, it should: a. Continue with its efforts to fix the incorrect data in ROCIMS; b. Provide training on data correction efforts and allocate time for its staff to identify and update all records that are incorrect; c. Test to ensure that the data has been corrected; and d. Develop and implement practices to ensure that the data remains accurate and complete. 3.2 To ensure that its business practices are aligned efficiently with ROCIMS or a replacement system, the ROC should: a. Analyze and document its applicable business practices; b. Where appropriate, redesign its business practices to most efficiently meet its operational requirements and align applicable forms and business procedures with ROCIMS or a replacement system; c. Develop and implement policies and procedures to document any revised business practices; and d. Communicate the changes to appropriate staff, including providing training where needed. 3.3 To better ensure that its staff understand and are able to use and maintain ROCIMS or a replacement system, the ROC should provide its staff with training relevant to their use of and responsibilities for the system by: State of Arizona page 44 a. Developing a training plan for system users and IT staff that includes who will be trained, what they will be taught, and when training will occur; b. Training staff according to the plan; c. Training new staff as they begin using the system; and d. Providing training to address changes to the system as it occurs. 3.4 Regardless of whether the ROC decides to correct or replace ROCIMS, to ensure that its system is appropriately managed and maintained, the ROC should: a. Improve project planning and oversight by developing, implementing, and periodically updating a project management plan; b. Develop and implement a systematic, accountable, and documented process for testing and applying updates; and c. Install updates after they have been properly evaluated and tested. 3.5 To better ensure the security of information within ROCIMS or a replacement system, the ROC should plan for, incorporate, and use appropriate security controls. 3.6 If the ROC replaces ROCIMS with a new system, it should follow the formal system development lifecycle methodology that it adopted during the course of the audit. Office of the Auditor General page 45 State of Arizona page 46 Other Pertinent Information The Legislature established the Residential Contractors’ Recovery Fund (Recovery Fund) to assist homeowners who suffer financial losses because of a licensed residential contractor’s actions. Before a homeowner can access the Recovery Fund, he/she must first either go through the Registrar of Contractors’ (ROC) complaint process or obtain a civil court judgment against the contractor. Although the ROC has implemented new practices to increase the speed of processing Recovery Fund claims, homeowners are not receiving payments until approximately 12 to 13 months after the claim has been approved. As of July 2012, the ROC had nearly $3.9 million in approved Recovery Fund claims that it could not pay because the Recovery Fund had not recovered from a total of $8.5 million in required transfers to the State General Fund in fiscal years 2009 through 2012. Residential Contractors’ Recovery Fund protects homeowners harmed by licensed contractors Recovery Fund created to protect homeowners The Recovery Fund was established by the Arizona Legislature in 1981 to assist residential homeowners who have suffered a financial loss caused by a licensed residential contractor’s actions. According to the National Association of State Contractors Licensing Agencies, as of 2010, ten states had construction recovery funds.1 Arizona’s Recovery Fund is maintained mostly through contractor fees paid when a new residential contractor’s license is issued or an existing license is renewed. For example, during fiscal year 2012, Recovery Fund net revenues totaled nearly $4.5 million, primarily from the fees for new and renewed licenses (see Table 2, page 6). In addition to license application fees, new residential and dual contractors who do not post a surety bond or cash deposit must also pay a Recovery Fund assessment, which is $300 for the first year and $150 for the second year, or one $450 payment for the 2 years (see Table 1, page 2), which is deposited into the Recovery Fund. A contractor must renew his/her license every 2 years and pay a $300 assessment to the Recovery Fund at each renewal. In lieu of paying the Recovery Fund assessment, contractors can choose to post a $200,000 bond or cash deposit, which covers the maximum amount that the Recovery Fund can pay out on a license. According to the ROC, at least four contractors have chosen to post a bond rather than pay the Recovery Fund fee. In addition, the Recovery Fund also receives money by earning interest income on investments of Recovery Fund monies and by recovering money from contractors and their bonding companies after a claim is paid for damages done by the contractor (see Table 2, page 6). The main use of Recovery Fund monies is to make payments to homeowners, who can recover the amount of actual damages caused by a licensed contractor up to $30,000. According to the ROC, in fiscal year 2012, 360 homeowners received payments from the Recovery Fund for approved claims (see Table 4, page 48). Pursuant to Arizona Revised Statutes (A.R.S.) §321139(A), the total amount the Recovery Fund can pay is capped at $200,000 per license for the lifetime of the license. Once the $200,000 cap on a contractor’s license has been reached, any additional claims cannot be paid from the Recovery Fund. In cases where multiple homeowners file complaints against a licensee and these claims total more than $200,000, the claims are 1 National Association of State Contractors Licensing Agencies. (2010). Contractor recovery funds. Phoenix, AZ: Author. Office of the Auditor General page 47 Table 4: Recovery Fund key facts during fiscal year 2012 Claim liimits Per consumer Total per license Claims received and pa aid Claims received Claims paid out Total claims amount paid Claims approved but unpaid as of July 1, 2012 Source: $30,000 $200,000 347 360 $3,976,590 324 Auditor General staff analysis of fiscal year 2012 Arizona Financial Information System (AFIS) data, A.R.S. §§32-1132(A) and 32-1139, and information provided by ROC staff. paid to homeowners based on the pro rata share of each homeowner’s claim to the total amount. As of September 2012, the ROC was monitoring 30 contractors who either had reached or were approaching the $200,000 limit. The ROC also uses monies from the Recovery Fund to pay for administrative costs to operate the Recovery Fund. In fiscal year 2012, the ROC spent $474,367 on Recovery Fund operating costs, primarily for personnel costs (see Table 2, page 6). A.R.S. §32-1134(A) allows the ROC to use 10 percent of the Recovery Fund’s fund balance for employee and contracted services, equipment, and operational costs of managing the Recovery Fund. However, since 2009, Budget Reconciliation Bill provisions have allowed the ROC to use up to 14 percent of the prior fiscal year revenues from the Recovery Fund to pay for these administrative costs. In fiscal years 2009 through 2012, the administrative expenses were within the allowable 14 percent limit each year (see Sunset Factor 9, pages 63 through 64, for additional information regarding the limit on Recovery Fund administrative expenses). ROC reviews Recovery Fund claims The process of obtaining money from the Recovery Fund requires homeowners to follow one of two paths: (1) the ROC’s complaint process that results in either a revocation or suspension of the contractor’s license or (2) a civil process that results in a court judgment against the contractor. These processes must start no later than 2 years from the date of the act that caused the harm or the date of occupancy. More homeowners use the complaint process to access the Recovery Fund than the civil process. A homeowner who follows the ROC’s complaint path must submit a claim to the Recovery Fund that includes information showing the amount paid to the licensed contractor and proof of either the cost to repair or complete the project. The ROC State of Arizona page 48 determines whether the homeowner is eligible for payment and determines the amount to be paid. The ROC gives homeowners the opportunity to request a hearing to review the ROC’s eligibility decision or the payment amount. Contractors can also request a hearing to contest the payment amount or the homeowner’s eligibility. For example, if the ROC decides to award the homeowner $5,000 to repair a pool, but the contractor believes the repair should cost only $3,000, the contractor can request a hearing to determine what amount will be paid. Contractors have the right to request a hearing to determine the amount to be paid because a contractor is required to repay the Recovery Fund any money paid as a result of his/her actions. After the ROC approves the claim, the money may be disbursed and the contractor’s license will remain or be suspended until repayment is made. A homeowner who follows the civil process path receives the amount ordered by the court, up to $30,000, which by statute cannot include attorney fees except in a contested case that is appealed to the superior court. In the civil process, a homeowner who files a lawsuit against the contractor must notify the ROC both when there is pending litigation and following the court order directing payment from the Recovery Fund. After the homeowner receives the order directing payment from the Recovery Fund, the money may be disbursed. Process for approving Recovery Fund claims has improved The ROC has taken steps to reduce the length of time that it takes to review Recovery Fund claims. According to the ROC, in September 2010, it had a backlog of more than 1,100 Recovery Fund claims that had not been reviewed, and claims were filed approximately 11 months before the ROC Recovery Fund eligibility criteria began reviewing them. However, according to the ROC, as of July 2012, it had only 169 claims awaiting review, and it Contractor must have been licensed and in further reported that its total turnaround time was approximately good standing at the time of the contract. 2 months from its receiving the claim to approving or denying Except where the claim results from a court the claim. According to the ROC, it is able to review claims order, the contractor’s license must have faster because it introduced a triage process in the summer been suspended or revoked as a result of the of 2010 to quickly filter out ineligible claims. Homeowners are claimant’s complaint. eligible for the Recovery Fund only if the homeowner has met Original complaint must have been filed statutory criteria (see textbox). According to the ROC, under within 2 years of the act that led to the the old system, the ROC would determine the appropriate complaint. monetary amount for all claims, even if it was determined that the homeowner was ineligible for the Recovery Fund. Now, Property must be a class three residential the ROC reviews eligibility first and determines monetary property that the claimant occupied or amounts only for eligible claims. intended to occupy at the time of the • • • • contract. In addition, the ROC has modified some of its award orders to make them effective immediately instead of after 40 days. Source: Auditor General staff analysis of A.R.S. §32-1131 et seq. Office of the Auditor General page 49 Specifically, when a contractor does not respond within 15 days to the ROC’s notice that a Recovery Fund claim will be paid, the ROC’s decision and award order formerly stated that the award would take effect 40 days after the date of the order unless the contractor filed a judicial review action and obtained a stay order within that time frame. The ROC reviewed the statutes regarding appeals of agency decisions and determined that these orders are not subject to judicial review except for the purpose of questioning the ROC’s jurisdiction over the matter pursuant to A.R.S. §12-902(B). As of September 2012, the ROC’s decision and award orders in these cases are effective immediately. Case example In October 2008, a homeowner filed a complaint against a contractor who had abandoned a pool project by leaving the pool’s interior incomplete and failing to install the pool equipment. After the complaint went through ROC’s administrative complaint process, the contractor’s license was revoked, and in April 2010, the homeowner filed a claim for Recovery Fund monies. However, because of the backlog of Recovery Fund claims, ROC staff did not review the homeowner’s claim until March 2011, almost 1 year later. Based on its review, ROC approved the claim for more than $8,000 from the Recovery Fund in May 2011. However, because of the shortfall in the Recovery Fund’s fund balance, the homeowner did not receive payment until April 2012, approximately 3.5 years after filing the initial complaint with ROC and nearly 2 years after filing a claim for Recovery Fund monies. This was also about 3 years after the empty pool was identified as a safety hazard and the homeowner paid another contractor to finish the pool. Source: Auditor General staff review of case files including some cases that submitted a Recovery Fund claim. State of Arizona page 50 Recovery Fund payments delayed Although some process improvements have been made, a shortfall in the Recovery Fund’s fund balance has meant that payments are still not being made until approximately 12 to 13 months after claims are approved. According to the ROC’s Web site, a homeowner whose claim was approved in July 2012 can expect payment in July 2013 (see Table 5). Table 5: Recovery Fund estimated payout schedule As of January 2013 Claim approval date Estimated payment time frame January-February 2012 March 2013 March 2012 April 2013 April-May 2012 May 2013 June 2012 June 2013 July 2012 July 2013 Source: Auditor General staff compilation of information posted on the ROC’s Web site and analysis of Recovery Fund pending claims to verify reliability of the ROC’s estimates. This delay is in addition to the time a homeowner has already spent in the civil process or the ROC’s complaint process trying to resolve the issue, which can take a long time (see Finding 2, pages 23 through 29). Homeowners with property damage that needs to be repaired immediately may have to pay for the repair without any reimbursement for several years. This can result in financial hardship for the homeowner (see Case example). The shortfall in the Recovery Fund’s fund balance stems from several years of legislatively required transfers to the State General Fund to provide adequate support and maintenance for state agencies. Specifically, approximately $6.6 million in legislatively required transfers from the Recovery Fund to the State General Fund occurred in fiscal year 2009. In addition, more than $1.8 million was transferred over fiscal years 2010, 2011, and 2012 (see Table 2, page 6). Although the ROC has continued to approve and pay some claims, as shown in Figure 2, as of July 31, 2012, the ROC had approved nearly $3.9 million in claims that it had insufficient monies in the Recovery Fund to pay. According to the ROC, it has continued to pay Recovery Fund claims by delaying payments until sufficient fees have been received to pay some of the approved claims, and these payments occur at least once per month. Claimants are paid in the order in which their claims were approved. Figure 2: Recovery Fund cash balance and unpaid claims As of July 31, 2012 (Unaudited) $4,500,000 $4,000,000 $3,500,000 $3,000,000 $2,500,000 $2,000,000 $1,500,000 $1,000,000 $500,000 $0 $3,895,676 $220,419 Fund balance Source: Unpaid claims Auditor General staff analysis of AFIS Status of General Ledger—Trial Balance and a ROC report detailing unpaid claims. This decrease in available monies has also affected the interest earned by the Recovery Fund. As of fiscal year 2012, interest revenues have decreased substantially. For example, in fiscal year 2008, the Recovery Fund received nearly $600,000 in interest. However, after the fund transfers in fiscal year 2009, the Recovery Fund earned $30,618 in interest in fiscal year 2010 and earned only $279 in interest in fiscal year 2012 (see Table 2, page 6). ROC expects Recovery Fund deficit to improve ROC officials expect the wait time between when a Recovery Fund claim is approved and when it is paid to narrow. According to the ROC, the wait time for payout is slowly beginning to decrease, and the number of claims waiting to be paid will continue to decline if circumstances remain stable. ROC officials stated that a contributing factor could be that complaints received during the recession were more frequently about abandonment and failure to pay a subcontractor, and since complaints about failure to pay a subcontractor are ineligible for the Recovery Fund, the number of Recovery Fund claims declined. Although the wait times for the ROC to pay claims increased from an approximate 1-month wait in April 2011 to a 13- to 14-month wait in January and February 2012, the wait times have stabilized and may be slightly decreasing. However, if wait times do not improve, other options could be considered. For example, the ROC could increase assessments in order to narrow the gap. As mentioned earlier, the assessments for participating in the Recovery Fund are $300 for a 2-year license renewal. According to A.R.S. §32-1132(B), the ROC could increase these assessments Office of the Auditor General page 51 to $600 for 2 years. The ROC has not done so, in part because of the 2011 repeal of A.R.S. §32-1134.01, which had authorized it to issue an emergency assessment on participating contractors when the Recovery Fund fund balance was below $2 million. ROC officials believe that this repeal indicated the Legislature’s preference not to increase assessments. Alternatively, the ROC could consider the approach taken by other agencies. For example, according to its director, the Arizona Department of Real Estate’s recovery fund has sufficient monies to pay approved claims even after transfers to the State General Fund because it increased the frequency of renewals from every 4 years to every 2 years and has used other monies to pay recovery fund administrative expenses.1 At the present time, the ROC does not have statutory authority to use monies from its other fund to pay for Recovery Fund claims or administrative expenses. 1 Auditors also looked at two other states’ recovery funds to try to identify options for narrowing the gap between revenues and expenditures, but the other states do not provide models for Arizona to follow. According to the Nevada State Contractors’ Board, Nevada’s recovery fund did not experience transfers to other funds and has continued to function as usual. According to the Florida Construction Industry Licensing Board, its recovery fund was unable to pay claims for approximately 2 years, primarily because the fund received a share of building permit revenue after other expenses were paid instead of receiving a dedicated stream of contractor renewal fees. A 2010 statutory change to the recovery fund’s funding mechanism created a dedicated revenue stream resulting in additional revenues to pay claims. Florida officials are unable to project when funds will become available to pay all pending claims. However, pending claims total more than $13 million so it may take some time for revenues to catch up with current claims. State of Arizona page 52 SUNSET FACTORS In accordance with Arizona Revised Statutes (A.R.S.) §41-2954, the Legislature should consider the following 12 factors in determining whether the Registrar of Contractors (ROC) should be continued or terminated. This analysis includes some recommendations for the ROC that were not raised in earlier sections of the report. These include recommendations to refund fees to license applicants whose licenses were issued or denied outside of the ROC’s 60-day review time frame and for the ROC to provide complete and consistent information to the public (see Sunset Factor 2, pages 54 through 59), and two recommendations regarding potential statutory changes (see Sunset Factor 9, pages 62 through 64). Sunset factor analysis 1. The objective and purpose in establishing the ROC and the extent to which the objective and purpose are met by private enterprises in other states. The Legislature established the ROC in 1931. Its mission is to promote quality construction by Arizona contractors through a licensing and regulatory system designed to protect the health, safety, and welfare of the public. Specifically, the ROC is responsible for: • Issuing licenses to qualified commercial and residential contractors; • Processing complaints and inspecting allegations of poor workmanship against licensed contractors (see Finding 1, pages 9 through 21); • Investigating complaints against unlicensed contractors and referring these cases to county or city prosecutors for prosecution; and • Administering the Residential Contractors’ Recovery Fund (Recovery Fund), which provides recourse to homeowners who are financially harmed by the actions of a licensed residential contractor (see Other Pertinent Information, pages 47 through 52). All states have some state-level regulation of contracting. According to a 50-state survey published by the National Association of State Contractors Licensing Agencies (NASCLA) in 2012, states vary in the number of agencies involved in this regulation, the types of contracting they regulate, and the role of local government in regulating the contracting industry.1 Specifically: • Number of state agencies—The NASCLA survey found that 32 states divide regulatory responsibilities for the contracting industry among multiple state agencies. For example, Alabama has separate state licensing agencies for general contractors, home builders, plumbers and gas fitters, and electrical contractors, and its insurance department is responsible for licensing sprinkler contractors, fire pump testers, and blasters. In addition to Arizona, 17 states use only one central regulatory body for regulating contracting at the state level, 9 of which–like Arizona–regulate nearly all types of construction trades. 1 National Association of State Contractors Licensing Agencies. (2012). Contractor’s state licensing information directory. Phoenix, AZ: Author. Office of the Auditor General page 53 • Types of contracting regulated—The NASCLA survey found wide variation in the types of contracting regulated at the state level. In addition to Arizona, 27 states, including 7 states that require some contractors to be registered and other contractors to be licensed, regulate all types of residential and commercial contracting or virtually all contracting above a specified dollar threshold. Three states—Massachusetts, Minnesota, and Pennsylvania— regulate residential contracting but do not regulate all commercial contracting. Seventeen states regulate only certain trades. For example, Colorado regulates only plumbing and electrical work. Two states regulate only asbestos removal contractors. Finally, nine of the states that regulate nearly all types of construction trades do so—like Arizona—through a central regulatory body. • Role of local government—According to the NASCLA survey, local governments provide additional regulation in some states. For example, most general contractors in Colorado are regulated at the city or county level, while trades that are not regulated at the state level in Georgia may require licensure in some municipalities and counties. In addition, Florida, which has state-level certification that allows contractors to work anywhere in the state, also has county-level certification that allows contractors to work only in the county in which they are certified. Cities and counties in Arizona and other states also establish local building codes and issue construction permits. The NASCLA survey did not identify any states that rely on private enterprise to regulate contracting. 2. The extent to which the ROC has met its statutory objective and purpose and the efficiency with which it has operated. ROC has met some of its statutory objectives and purposes and has taken steps to improve the efficiency of its operation, but more can be done in several areas. Some examples in which the ROC has met its objectives and purpose and improved its operations include: • Licensing qualified applicants—The ROC works to ensure that licensed contractors are qualified. As discussed in the Introduction and Background (see pages 1 through 8), the ROC reviews license applicants’ work experience and their scores on business and trade examinations to ensure they are qualified. In the five applications auditors reviewed that were issued in fiscal year 2012, the ROC appropriately determined that the applicants were qualified prior to issuing a license. For example, all five applicants had completed the testing requirements for the licenses they were applying for. In addition, the ROC has taken steps to better ensure that it prevents unsafe contractors from becoming licensed. Although A.R.S. §32-1122(H) gives State of Arizona page 54 the ROC authority to perform background checks, according to the ROC, from 2002 through 2012, it required fingerprints and background checks only if an applicant self-reported a criminal history on his/her application. However, in August 2012, the ROC entered into an agreement to use a subscription service to obtain the criminal history of all new applicants. As of December 2012, the ROC reported that it had tested the software and was developing an implementation plan for using the software in 2013. • Investigating unlicensed contracting—The ROC works to protect the public from individuals who perform contracting work without a license. According to A.R.S. §32-1151, contracting without a license is illegal in Arizona. The ROC reported that it conducts undercover sting operations to locate unlicensed contractors. For example, in June 2012, the ROC announced that it had joined contracting boards in Oregon, California, and Nevada in a sting of unlicensed contractors advertising on Craigslist. According to the ROC, the Arizona sting led to 15 unlicensed contractor investigations. In addition, in auditors’ review of five unlicensed contractor investigation cases received between calendar years 2008 and 2012, the ROC investigated all five cases in a timely manner. Specifically, the ROC conducted the investigation and referred the cases to prosecutors within 11 to 99 days. Further, the ROC posts information about unlawful contracting on its Web site, including information about individuals convicted of contracting without a license. • Improving Recovery Fund claims processing—The ROC has made improvements to the way it processes Recovery Fund claims. To obtain reimbursement from the Recovery Fund for repairs necessitated by the actions of a licensed contractor, if work has not begun to repair the project, the homeowners must submit a claim that includes three estimates of the repair costs. If the repair has already started or has been completed, the homeowner must provide a copy of the new contract, invoices, receipts, and proof of payment on the new contract. ROC staff review the estimates to determine the reimbursement amount. To improve efficiency, the ROC has introduced a triage process to quickly filter out ineligible claims, eliminating the need for staff to review repair estimates for ineligible claims. In addition, as of November 2012, the ROC had obtained claims-adjusting software and was in the process of testing the software. The claims-adjusting software will enable ROC staff to determine reimbursement amounts independently instead of reviewing estimates obtained by the homeowner. If tests show that the software provides accurate results, the ROC plans to begin using the software in 2013. Using this software may decrease the risk of fraud in claim amounts. Despite improvements in the claims-approval process as of summer 2012, Recovery Fund payments are delayed approximately 12 to 13 months because of a deficit in the Recovery Fund (see Other Pertinent Information, pages 47 through 52). Office of the Auditor General page 55 • Management of fleet vehicle use—The ROC has improved its management of the fleet vehicles it leases from the Department of Administration (DOA). A 2003 performance audit conducted by the Office of the Auditor General identified inappropriate and inefficient use of the ROC’s vehicle fleet. During the time of the audit, the ROC leased 65 vehicles and assigned nearly all of them to its employees for take-home use. In addition, 22 of its leased vehicles were driven fewer than the DOA’s efficient use standard of 10,000 miles per year. The audit report recommended that the ROC stop providing take-home vehicles to many of its employees and eliminate the inefficiently used vehicles. Since 2009, ROC management has returned approximately one-half of its leased vehicles to the DOA, and as of April 2013, the ROC was leasing only 33 fleet vehicles. In addition, the ROC has stopped assigning vehicles to individual employees and uses a check-out system instead. However, the audit found that the ROC can better meet its statutory objectives in the areas of licensing, complaint handling, data management, and public information. Specifically: State of Arizona page 56 • Refunding fees to license applicants for untimely processing of applications—ROC should develop and implement policies and procedures to refund fees to applicants whose licenses were issued or denied outside of the ROC’s 60-day time frame. According to the ROC’s analysis, in fiscal year 2011, it issued 671 licenses outside of the 60-day time frame, as required by statute. As prescribed by A.R.S. §41-1077, when the ROC issues or denies a license application outside of the time frame defined in its rules, the ROC is required to pay penalties to the State General Fund and refund the application fees to the applicant. The ROC has paid penalties to the State General Fund. Specifically, in fiscal year 2011 the ROC paid $6,449 to the State General Fund. However, according to the ROC, it has not refunded fees to applicants whose licenses were issued or denied outside of the time frame as required by statute. Auditors determined that if the ROC refunded fees to these applicants for fiscal year 2011, the ROC would need to refund $371,385 in fees. Therefore, the ROC should develop policies and procedures to refund fees to applicants who are licensed or denied a license outside of the 60-day time frame. During the audit, the ROC began working toward developing such a process. As of April 2013, the process was not fully developed, but ROC officials reported that the ROC had issued refunds in a small number of cases. In addition, it should work with its Attorney General representative to determine whether it needs to refund fees to applicants from previous years whose licenses were issued or denied outside of the time frame. • Complaint handling—ROC does not ensure that complaints against licensed contractors are adequately resolved (see Finding 1, pages 9 through 21). The ROC receives complaints from the public regarding licensed contractors who allegedly abandon work, fail to pay subcontractors, and perform poor workmanship. However, the ROC closes some complaints without verifying that workmanship problems were adequately addressed and comply with workmanship standards. As a result, the public is not protected, and complainants may experience frustration and difficulty when attempting to address problems that are not adequately resolved. Further, the ROC lacks policies and procedures to ensure that it consistently investigates and takes action on complaints that were received within 2 years of the contractor’s work as prescribed by statute. Thus, the ROC should strengthen its complaint resolution processes to ensure that problems are adequately addressed and that licensed contractors are appropriately disciplined. Also, the ROC should develop guidance for its staff to ensure that it consistently applies the 2-year statutory time limit for filing a complaint so that it can investigate and, if necessary, take action on these complaints. In addition, the process for handling complaints that result in citations and discipline is lengthy and allows complaints that result in discipline to remain unresolved for long periods of time (see Finding 2, pages 23 through 29). A lengthy complaint-handling process can negatively affect the public by providing inadequate information to consumers who may be seeking to hire contractors and prolonging the period until homeowners receive needed repairs. ROC can take several steps to resolve complaints more quickly. Specifically, the ROC should explore and implement ways to incentivize contractors to resolve problems before a citation is issued, monitor complaints as they move through the process, add time frames for key steps to the complaint process, and develop and implement policies and procedures to ensure that disciplinary actions are entered into its data system in a timely manner. • Data management—Problems with the ROC’s new data system have affected the ROC’s ability to perform some of its core functions (see Finding 3, pages 31 through 45). In 2010, the ROC installed a new online licensing and data management system called Registrar of Contractors Information Management System (ROCIMS). Responsibility for the implementation of ROCIMS was shared between the State’s Government Information Technology Agency (GITA), the State’s Web portal contractor NIC Inc. (NIC) and the ROC.1 However, because the parties responsible for system implementation did not take a number of key actions, ROCIMS was not implemented successfully and has hampered the ROC’s ability to perform some of its core functions. For example, business practices were not evaluated and modified to match the system’s capabilities, which has resulted in the ROC’s staff using time-consuming workarounds to complete tasks. In addition, a data migration plan was not developed and followed, which has resulted in thousands of duplicate, inaccurate, or missing records in ROCIMS. Further, ROC has not been able to use ROCIMS’s online application feature for new licenses and has continued to require applicants to submit paper applications 1 As of 2011, GITA merged with two other large technology groups and is now known as the Arizona Strategic Enterprise Technology Office (ASET), which is part of the Arizona Department of Administration. Office of the Auditor General page 57 that staff must enter manually. Some of the problems the ROC has experienced with the system have prevented the ROC from performing statutorily mandated tasks, such as identifying contractors who operate under other license names after they have had one of their licenses suspended or revoked, and disciplining these additional licenses. Although the ROC has reported that it intends to replace ROCIMS with a new system, it should take steps to address existing problems related to ROCIMS and to help ensure successful implementation of the new system. Specifically, the ROC should continue its efforts to correct the system’s data problems, analyze its business practices and redesign them as appropriate, ensure its staff are trained on the system, and create processes for managing and updating the system. • Public information—One important part of a regulatory agency’s responsibilities is providing information about licensees that allows consumers to make informed decisions about using the services of the licensees it regulates. The ROC fulfills its responsibility to provide public information in various ways. For example, its Web site contains information designed to help consumers, such as advice on how to select a contractor, information on how to file a complaint against a contractor, and a list of individuals convicted of contracting without a license. It also publishes 2 years’ worth of complaint histories for all licensed contractors and allows individuals to verify whether a contractor is licensed. This information is important for consumers who are deciding whether to hire a contractor and also useful for contractors who are deciding whether to hire a subcontractor. Further, the public may obtain additional complaint history information by calling the ROC’s phone center. However, the ROC can provide more timely public information. The ROC’s delay in entering contractor discipline into its data system can result in consumers’ not having access to up-to-date information on licensees (see Finding 2, pages 23 through 29). For example, once the ROC determines that a license will be disciplined, the ROC finalizes the discipline by entering the information into its data system so that information is available on its Web site. According to ROC officials, the ROC does not consider discipline to be effective until the data entry date. However, the ROC has delayed entering the discipline into its data system in some cases, which has prevented the public from receiving accurate information about licensed contractors’ complaints and discipline. When the ROC delays entering discipline into its data system, consumers may hire a problem contractor who, if discipline had been promptly entered into the ROC’s data system, would not be legally able to work. In addition, for complaints where the contractor was not disciplined, the ROC’s Web site does not provide consumers with information on how the complaint was closed. Specifically, it lists the total number of complaints that State of Arizona page 58 were corrected, settled, or withdrawn without identifying the number closed in each of those three ways. The Web site also does not indicate whether the complaints were related to poor workmanship, abandonment, or failure to pay subcontractors. Additionally, the ROC’s practice of not disciplining contractors if the contractors fix the work means that the public may be unable to determine that a contractor has had a history of workmanship that did not meet industry standards. The ROC should develop and implement a method for providing additional complaint information to the public including information about the type of complaint and how the ROC resolved the complaint (see Finding 1, pages, 9 through 21, for additional information). Further, the ROC should develop and implement policies and procedures to guide the ROC’s call center customer service representatives in providing information to the public. The ROC’s call center customer service representatives do not always provide complete and consistent information to the public. Specifically, when contractors have multiple licenses, customer service representatives sometimes provide information on only one license. For example, auditors called the ROC to obtain public information on four licensees, but in three cases, the ROC did not provide information on associated licenses held by the licensee, even for associated licenses that had complaints. According to ROC management, customer service representatives are trained to transfer more complex questions from the call center to another department, such as its legal department, for more information. According to the ROC, ROCIMS does not show call center staff a clear connection between a licensee and his or her multiple licenses. In addition, the ROC does not have written policies and procedures to guide customer service representatives. Therefore, the ROC should develop and implement policies and procedures that ensure call center customer service representatives provide consistent and complete information. The ROC should also develop and provide guidance to call center customer service representatives about how to identify information in ROCIMS in order to provide complete information. 3. The extent to which the ROC serves the entire State rather than specific interests. The ROC provides services across the State to licensees and the public, but it could make one improvement to make it easier for license applicants, regardless of where they live in the State, to apply for a license. The ROC serves the entire State by providing state-wide inspection and investigation services for complaints against contractors, by maintaining offices in eight cities state-wide, and by providing online resources for consumers and contractors. The ROC has office locations in Phoenix, Flagstaff, Kingman, Lake Havasu City, Prescott, Show Low, Tucson, and Yuma. As of October 2012, the agency employed 28 inspectors and 11 investigators state-wide. In addition, the ROC’s Web site allows contractors and consumers across the State to verify whether a contractor is licensed and how many complaints the licensee has had in the past 2 years. Office of the Auditor General page 59 However, because of problems with ROCIMS, the ROC is unable to use the system’s online license application feature (see Finding 3, pages 31 through 45). As a result, license applicants must mail their applications to the ROC or visit a ROC office in person to apply for a license. Once the ROC has addressed the problems in ROCIMS, it will be able to improve its service to contractors throughout the State by implementing the online license application feature. 4. The extent to which rules adopted by the ROC are consistent with the legislative mandate. General Counsel for the Office of the Auditor General has analyzed ROC’s rulemaking statutes, as well as the ROC’s pending 5-year rule review, and believes that the ROC has fully established rules required by statute. 5. The extent to which the ROC has encouraged input from the public before adopting its rules and the extent to which it has informed the public as to its actions and their expected impact on the public. ROC has not modified its rules since 2005, but according to the ROC, it encourages input from stakeholders before adopting rules. The ROC has two advisory councils made up of stakeholders—the Industry Advisory Council and the Labor Advisory Council—that can provide input to the ROC on proposed rule changes. In March 2011, the ROC sought input from stakeholders regarding possible changes to workmanship standards regarding the growing solar power industry. ROC also reported that it will accept public comment as required by the rulemaking process. Pursuant to A.R.S. §41-1023(B), once the Secretary of State has published proposed rule changes, the ROC must allow the public the opportunity to submit written statements on the proposed rule change. 6. The extent to which the ROC has been able to investigate and resolve complaints that are within its jurisdiction. ROC has sufficient statutory authority to investigate and resolve complaints within its jurisdiction except for some complaints affected by A.R.S. §32-1155(C). As discussed in Sunset Factor 9 (see pages 62 through 64), A.R.S. §32-1155(C) prohibits the ROC from disciplining a contractor for poor workmanship if the work was modified even if the modifications were made for the homeowner’s safety. For other complaints, the ROC has various disciplinary options available, such as requiring the contractor to pay a civil penalty, and suspending or revoking a license. However, the ROC’s process does not ensure that complaints against licensed contractors are adequately resolved (see Finding 1, pages 9 through 21). Specifically, the ROC closes some complaints without verifying that workmanship problems were adequately addressed and comply with workmanship standards. State of Arizona page 60 The ROC should modify its complaint resolution process to ensure that problems are adequately addressed. Additionally, the ROC should strengthen its processes for disciplining licensed contractors when appropriate, including using consent agreements and escalating discipline for contractors with prior substantiated complaints. In addition, the ROC’s process for handling complaints that result in citations and discipline is lengthy and allows complaints to remain unresolved for long periods of time (see Finding 2, pages 23 through 29). The ROC needs to address the factors that contribute to lengthy complaint resolution. 7. The extent to which the attorney general or any other applicable agency of the state government has the authority to prosecute actions under the enabling legislation. According to A.R.S. §41-192, the Attorney General’s Office has authority to prosecute actions on behalf of the ROC and, as such, has authority to represent the ROC in hearings where the ROC has intervened or is named as the complainant. The Attorney General’s Office also performs other services for the ROC. For example, after the ROC pays Recovery Fund claims, the ROC then sends the claim to the Attorney General for collection pursuant to A.R.S. §321138. In addition, according to A.R.S. §32-1164, contracting without a license is a class 1 misdemeanor. According to the ROC, it turns these cases over to county or city prosecutors for prosecution. However, if these cases involve acts of consumer fraud, then the ROC will turn these cases over to the Attorney General for prosecution. According to the ROC, every year it turns over about 670 cases of unlicensed contracting to county and city prosecutors. 8. The extent to which the ROC has addressed deficiencies in its enabling statutes that prevent it from fulfilling its statutory mandate. In 2008 through 2012, several changes were made to the ROC’s enabling statutes, including two changes proposed by the ROC to improve its ability to fulfill its statutory mandate. According to the ROC, additional changes are not required to comply with its statutory mandate although changes could be made to clarify and strengthen statute and improve the efficiency of its operations (see Sunset Factor 9, pages 62 through 64). Specifically, the ROC proposed the following changes: • Laws 2011, Ch. 250, added A.R.S. §32-1155.01, which introduced complaint handling through arbitration. A.R.S. §32-1155.01 established authority for an arbitration process that allows the ROC to require parties in complaints valued at less than $5,000 to go through arbitration to resolve the issue. As of November 2012, the ROC was working on developing an arbitration process in accordance with this statutory change, but had not yet established a process. In addition, this legislation also made changes to A.R.S. §32-1139, which affects Recovery Fund amounts available to homeowners. Prior to these Office of the Auditor General page 61 changes, statute required that if claims against any one residential contractor’s license exceed $200,000, the claims shall be paid in the order of the date of entry of the order of the registrar or court. The revision requires that if claims against any one residential contractor’s license exceed $200,000, the claims shall be paid based on a pro-rata share of the common liability. This legislation also modified A.R.S. §32-1154(F) to eliminate the provision that the contractor must have refused or been unable to comply with an order to remedy the violation in order for the homeowner to be eligible for the Recovery Fund. According to the ROC, this change provided more flexibility in disciplinary actions that it can take without excluding homeowners from the Recovery Fund. • Laws 2012, Ch. 297, §27, and Budget Reconciliation Bills in 2009, 2010, and 2011 made temporary revisions to A.R.S. §32-1134, affecting the ROC’s costs for administering the Recovery Fund. These revisions have allowed the ROC to use up to 14 percent of the total Recovery Fund revenue from the previous fiscal year to pay administrative expenses. (See Sunset Factor 9, pages 63 through 64, for further information regarding this provision.) Other statutory changes affecting the ROC included: 9. • Laws 2008, Ch. 261, revised A.R.S. §32-1154(B), eliminating the requirement for the ROC to investigate certain complaints. Prior to the revision, A.R.S. §321154(B) required the registrar to investigate the acts of any contractor in this State when the ROC received the written complaint of any person. The revision changed statute to require the ROC to investigate complaints received from any owner or contractor who is a party to a construction contract or a person who suffers a material loss or injury as a result of a contractor’s failure to perform work in a professional and workmanlike manner. • Laws 2009, Ch. 99, revised A.R.S. §32-1124, strengthening the requirements regarding contractors’ display of their license number. Prior to the revision, statute required that the licensee’s license number shall be placed on all advertising, letterheads, and other documents used by the licensee in the conduct of business. The revision added that a licensee’s license number shall also be placed on all broadcast, published, Internet, or billboard advertising. The extent to which changes are necessary in the laws of the ROC to adequately comply with the factors listed in this subsection. The Legislature should consider statutory changes in two areas. Specifically: First, according to the ROC, the passage of A.R.S. §32-1155(C) has hampered its ability to protect the public. A.R.S. §32-1155(C) was passed by the Legislature in State of Arizona page 62 2008 and states that the ROC shall not Example where the ROC could not discipline license issue a citation to a licensed contractor for failure to perform work in a profesA homeowner paid a contractor to repair her air conditioning unit sional and workmanlike manner if (1) in early July 2011. A few weeks later, on a Friday at the end of the contractor is not provided an July, the air conditioning unit stopped working. The homeowner opportunity to inspect work within 15 called the contractor who did the repairs in early July, but the days of receiving a written notice from contractor said he would not be able to come out until Monday. the registrar or (2) the contractor’s work The homeowner stated that the temperature inside was 98 degrees and there was a sick individual living in the house. has been subject to neglect, modificaBecause the original contractor could not respond until the tion, or abnormal use. This statute following week, the homeowner hired a different contractor who prevents the ROC from investigating repaired the air conditioning unit the same day for more than complaints and potentially disciplining $2,000. In September, the homeowner complained to ROC about a licensed contractor if the homeowner the poor repair in early July by the original contractor. However, modified the work, even if modificabecause the homeowner had not given the original contractor 15 tions were made because safety was days to inspect the work and the work had been modified, the an issue for the homeowner (see textROC was not able to investigate or cite the original contractor but box for an example). As a result, the instead had to dismiss the homeowner’s complaint, and the ROC cannot investigate complaints homeowner was not able to apply for Recovery Fund monies. against contractors where, because of Source: Auditor General staff analysis of the ROC’s complaint files. safety issues, homeowners have had to have work modified without allowing the original contractor 15 days to address the issue. Further, homeowners who are forced to make emergency repairs are unable to apply to the Recovery Fund for reimbursement because reimbursement can occur only after the contractor’s license has been disciplined (see Other Pertinent Information, pages 47 through 52). The ROC should seek a statutory change that would allow exceptions, such as for situations in which homeowner safety is at risk or the contractor has already gone out of business, that will enable it to better protect the public. Second, the Legislature should consider modifying the statute that limits the Recovery Fund’s administrative expenses. According to A.R.S. §32-1134, the ROC may use up to 10 percent of the Recovery Fund’s balance in any fiscal year to hire individuals to manage the Recovery Fund. However, as indicated in Sunset Factor 8 (see pages 61 through 62), Budget Reconciliation Bill provisions have allowed the ROC to use up to 14 percent of the prior fiscal year revenues from the Recovery Fund to pay for these administrative expenses. In fiscal year 2012, the ROC used this money primarily to employ five full-time Recovery Fund positions and one full-time legal department position.1 Among other responsibilities, these employees performed tasks such as determining a homeowner’s eligibility to receive money from the Recovery Fund and determining the amount to be awarded for each claim. Auditors determined that the 10 percent limit of each year’s ending fund balance would not have been sufficient to pay the Recovery Fund’s administrative expenditures for fiscal years 2008 through 2012. For example, if the ROC were allowed to use only 10 percent of the Recovery Fund’s 1 As of October 2012, the ROC had reduced its Recovery Fund staffing to four full-time positions. Office of the Auditor General page 63 fund balance to pay for administrative costs, in fiscal year 2012, the ROC would have had only $14,364 to pay these administrative expenses, which appears to be insufficient. However, the 14 percent limit based on the prior fiscal year’s Recovery Fund revenues was sufficient to cover the administrative costs. These costs totaled $474,367 in fiscal year 2012. Specifically, the ROC used 13.52 percent, 9.78 percent, 11.02 percent, and 9.95 percent of the prior fiscal year’s Recovery Fund revenues to pay for the Recovery Fund’s administrative costs in fiscal years 2009, 2010, 2011, and 2012, respectively. 10. The extent to which the termination of the ROC would significantly affect the public health, safety, or welfare. Terminating the ROC would likely pose some harm to the public’s health, safety, and welfare if its regulatory responsibilities were not transferred to another entity. Specifically, the ROC helps ensure that contractors are properly trained to work in their specific trades by requiring license applicants to supply documentation of their work history and to pass trade and business exams. Some trades, such as electrical work, can pose significant health and safety hazards to the public if not performed properly. The ROC’s Web site also provides information to the public about contractors’ license status and complaint history, helping members of the public to make an informed decision about whether or not to hire a contractor. In addition, the ROC provides a mechanism for resolving disputes with contractors without the complainant’s filing a civil action against the contractor in civil court. Finally, the Recovery Fund that the ROC administers protects homeowners who suffer a financial loss as a result of a contractor’s actions. 11. The extent to which the level of regulation exercised by the ROC compares to other states and is appropriate and whether less or more stringent levels of regulation would be appropriate. The audit found that regulation of contractors varies significantly among the states, which differ in the type of regulation and the types of contractors regulated, as well as in the level of government responsible for regulation. As discussed in Sunset Factor 1 (see pages 53 through 54), Arizona is one of 28 states that regulate all types of commercial and residential contracting or virtually all contracting above a specific dollar threshold. Seven of those 28 states require certain construction trades to register while requiring other construction trades to be licensed. Of the nation’s remaining 22 states, 2 do not regulate contractors except for asbestos abatement contractors, 3 regulate residential contractors but not all commercial contractors, and 17 regulate one or more specific trades such as electrical or plumbing. State of Arizona page 64 Although Arizona has more license types than many states, its requirements to obtain a license are generally similar to those in other states auditors reviewed.1 Only one state auditors reviewed—Louisiana, with 280 license classifications— had more license classifications than Arizona, which has 238. The ROC’s requirements for obtaining a license were generally similar to the requirements in other states that license contractors. Specifically: • Background check—In Arizona, a contractor is not required to submit fingerprints for a background check in order to become licensed, but may be asked to do so if the applicant self-reports a criminal background. Auditors’ review of 10 states with contractor licensure similar to Arizona’s found that 4 states conduct background checks prior to licensure, and 6 states do not. • Work experience—In Arizona, contractors must have between 6 months to 4 years of practical or management trade experience in order to become licensed, depending upon the license. Auditors’ review of 10 states with contractor licensure similar to Arizona found that 8 states require contractors to have work experience. Of these 8 states, 6 require between 2 to 4 years of experience, and 2 do not specify the number of years needed. • Pass an exam—In Arizona, the “qualifying party” must pass a written exam in order for a contractor to become licensed. Of the 50 states surveyed by NASCLA, 45 other states require at least some of the contractors regulated in the state to pass an examination. Four states do not require contractors to pass an examination. • Bond requirements—In Arizona, a contractor must post a bond in order to become a licensed contractor, and this bond amount varies depending on the type of license the contractor is issued and the level of anticipated work. Auditors’ review of 10 states that license contractors similar to Arizona found that 7 states require applicants to post a bond, and the bond amount can range from $1,000 to $500,000. 12. The extent to which the ROC has used private contractors in the performance of its duties as compared to other states and how more effective use of private contractors could be accomplished. ROC has used private contractors to help it perform its duties. For example: • License examinations—In Arizona, in order to become a licensed contractor, an applicant must first pass an examination. The ROC uses a private 1 In addition to reviewing the 50-state information in the NASCLA report, auditors reviewed statutes and Web sites for 14 states: California, Colorado, Florida, Idaho, Louisiana, Nevada, New Mexico, North Carolina, Oregon, South Carolina, Tennessee, Texas, Utah, and Washington. Nine of the states were selected because the ROC identified them as being similar to Arizona in having experienced housing booms and/or in the way they regulate contracting, and the remaining 5 states were selected to provide an understanding of different ways of regulating contracting. Of the 14 states, 10 regulate contractors in a similar way to Arizona’s, requiring virtually all contractors to be licensed. Office of the Auditor General page 65 contractor to administer these examinations to applicants. Several other states also use an outside vendor to administer examinations to license applicants. In addition, at least one state, New Mexico, uses this private contractor to collect all licensing information, provide the exam, and issue the license. In Arizona, once an applicant passes the examination and meets the other requirements for licensure, the ROC will issue a license. • Information technology—To obtain and manage ROCIMS, the ROC contracted with the Government Information Technology Agency (GITA) and the State’s dedicated contractor, NIC.1 NIC is the entity that coordinates Arizona’s Web portal, and NIC has a contract with Iron Data LLC, the company that owns the commercial software (see Finding 3, pages 31 through 45, for additional information regarding the ROC’s data system). In addition to these contractors, who were used to obtain ROCIMS, the ROC reported that it has contracted with other private contractors to perform some duties related to the system, such as hiring a technical support specialist to design, install, modify, or maintain system software. • Temporary services—On occasion, the ROC uses contracts for temporary service employees to perform some duties of a limited duration. For example, the ROC has hired a temporary legal secretary. The audit identified a need for the ROC to correct data accuracy problems in its information technology system. One option to do so is to use private contractors to develop an automated way to correct the data (see Finding 3, pages 31 through 45, for additional information). 1 As of 2011, GITA merged with two other large technology groups and is now known as the Arizona Strategic Enterprise Technology Office, which is part of the Arizona Department of Administration. State of Arizona page 66 APPENDIX A This appendix provides information on the methods auditors used to meet the audit objectives. This performance audit was conducted in accordance with generally accepted auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Methodology Auditors used various methods to study the issues in this report. These methods included reviewing the ROC’s statutes, rules, policies, and procedures; interviewing ROC officials and staff, various stakeholders, complainants, and officials from other state contractor regulatory agencies; and reviewing information from the ROC’s Web site.1 Auditors also compiled and analyzed financial information, including unaudited information from the Arizona Financial Information System (AFIS) Accounting Event Transaction File for fiscal years 2008 through 2012; the AFIS Management Information System, Status of General Ledger—Trial Balance screen, for fiscal years 2010 through 2012 and for July 31, 2012; and the ROC’s fiscal year 2013 budget submission. In addition, auditors used the following specific methods to meet its audit objectives: • The Auditor General and staff express appreciation to the Director and staff of the Registrar of Contractors (ROC) for their cooperation and assistance throughout the audit. To determine whether the ROC’s processes and practices help ensure that complaints are handled appropriately and in a timely manner, auditors examined ROC documents and reviewed a December 2008 report by the Ombudsman’s Office.2 To understand how these processes and practices play out in actual cases, auditors reviewed (1) ten complaints against licensed contractors that were closed in fiscal year 2011 without the ROC‘s issuing a citation to the contractor, (2) ten complaints against licensed contractors that were closed in fiscal year 2011 and involved a citation, (3) complaints related to a contractor whose case was reported in an April 2012 news article regarding the contractor’s payment to settle accusations of fraud, and (4) complaints related to one of the four contractors that were reviewed to assess how the ROC provides information to the public. Further, auditors examined the ROC’s guidelines for elevating discipline when licensed contractors have multiple pending complaints. Additionally, auditors performed a spot check of two licenses the ROC revoked in August 2012 to determine how long the ROC took to impose the revocation by posting the information on its Web site. In addition, auditors examined the Arizona Agency Handbook and reviewed practices at other Arizona regulatory agencies.3 Finally, auditors reviewed 1 Auditors interviewed officials at the Florida Construction Industries Licensing Board, the Nevada State Contractors Board, the New Mexico Regulation and Licensing Department’s Construction Industries Division, the North Carolina Licensing Board for General Contractors, the Oregon Construction Contractors Board, the Tennessee Board for Licensing Contractors, and the South Carolina Department of Labor, Licensing and Regulation. 2 Arizona Ombudsman Citizen’s Aide. (2008). Final report of investigation, Registrar of Contractors. Phoenix, AZ: Author. 3 Auditors interviewed officials or reviewed Web site information regarding other Arizona regulatory agencies including the Arizona State Board of Nursing, the Arizona Medical Board, the Arizona Board of Appraisal, the Arizona State Board of Physical Therapy, and the Arizona State Board of Pharmacy. Office of the Auditor General page a-1 the National State Auditors Association’s recommended best practices for state regulatory programs.1 • To assess whether the ROC’s Information Management System (ROCIMS) is reliable, accurate, and adequate for the ROC’s needs, auditors reviewed thirdparty evaluation reports analyzing ROCIMS, and observed ROC staff use of the ROCIMS licensing function. In addition, auditors interviewed Iron Data LLC, NIC Inc., and Arizona Strategic Enterprise Technology representatives to obtain background information on the development and deployment of ROCIMS and the problems that affect ROCIMS. Further, auditors reviewed project documents, including the project investment justification, the statement of work, the system functional requirements, a post-implementation assessment of the system, a system administration manual, and a report from the ROC showing inaccurate or duplicated data. Additionally, auditors interviewed representatives from agencies that use or will soon use this same e-license system CAVU developed, including the Nebraska Department of Banking and Finance; the City of Raleigh, North Carolina; the Arizona Supreme Court’s Administrative Office of the Courts; the Colorado Department of Regulatory Agencies’ Division of Registrations; the Connecticut Department of Consumer Protection; the Arizona Department of Liquor Licenses and Control; and the Arizona Department of Financial Institutions. Finally, auditors researched applicable IT industry standards and recognized best practices. • To obtain information about the condition of the Residential Contractors’ Recovery Fund (Recovery Fund), auditors reviewed ROC reports detailing the number of contractors who are approaching the statutory limit of $200,000 per license for Recovery Fund payments, the numbers of claims awaiting approval, and approved claims awaiting payment. Further, auditors analyzed the ROC’s September 2012 projected claim payment schedule and compared it with the Arizona Financial Information System Accounting Event Transaction File for fiscal years 2011 and 2012. Finally, auditors interviewed officials at the Arizona Department of Real Estate regarding the Real Estate Recovery Fund. • To obtain information used in the Introduction section, auditors reviewed the National Association of State Contractors Licensing Agencies’ 2012 Edition of the Contractor’s State Licensing Information Directory to obtain information about the regulation of contractors for all 50 states.2 In addition, auditors examined the Office of Administrative Hearings (OAH) fiscal year 2012 report to determine how many cases the ROC sent to OAH in fiscal year 2012. Further, auditors reviewed five complaints against unlicensed contractors that the ROC received in calendar years 2008 through 2012. Additionally, to determine the number of individuals the ROC employed, auditors reviewed a ROC report 1 National State Auditors Association. (2004). Carrying out a state regulatory program. Lexington, KY: Author. 2 National Association of State Contractors Licensing Agencies. (2012). Contractor’s state licensing information directory. Phoenix, AZ: Author. State of Arizona page a-2 based on information in the Arizona Department of Administration Human Resources Information System. • • To obtain information used in the sunset factors, auditors conducted a review of statutes and Web sites for 14 states to determine how Arizona’s licensing requirements compare to other states.1 In addition: ◦ To determine whether the ROC’s processes and practices helped ensure that new licenses are issued to qualified applicants, auditors reviewed a sample of five licensing applications received between June 2011 and January 2012. In addition, auditors reviewed the ROC’s analysis of licenses acted upon outside of the 60-day time frame for fiscal years 2010 through 2012. ◦ To assess whether the ROC shares appropriate information regarding licensees with the public, auditors placed four phone calls to the ROC’s call center in March and April 2012 requesting information about four licensees and reviewed complaint history information presented on the ROC’s Web site. ◦ To determine the impact of A.R.S. §32-1155(C) on the ROC’s ability to protect the public, auditors reviewed two cases the ROC identified in which it was forced to dismiss the complaints because the homeowners, concerned for the safety of family members, had the work repaired without giving the original contractor 15 days to fix the issue or had the work fixed by a different contractor. ◦ Finally, audit staff examined the ROC’s rule-making statutes as well as the ROC’s 5-year rule review to determine if the ROC had fully established rules required by statute. Auditors’ work on internal controls included reviewing the ROC’s policies and procedures for ensuring compliance with ROC statutes and rules. For example, auditors reviewed policies and procedures and tested the ROC’s compliance with various policies and procedures and/or statutes and rules for complaint handling, licensing, and providing information to the public. Auditors’ conclusions on these internal controls are reported in Finding 1 (see pages 9 through 21), Finding 2 (see pages 23 through 29), Finding 3 (see pages 31 through 45), Sunset Factor 2 (see pages 54 through 59), Sunset Factor 3 (see pages 59 through 60), and Sunset Factor 9 (see pages 62 through 64). 1 Auditors reviewed statutes and Web sites for 14 states: California, Colorado, Florida, Idaho, Louisiana, Nevada, New Mexico, North Carolina, Oregon, South Carolina, Tennessee, Texas, Utah, and Washington. Nine of the states were selected because the ROC identified them as being similar to Arizona in the way they regulate contracting, and the remaining 5 states were selected to provide an understanding of different ways of regulating contracting. Office of the Auditor General page a-3 AGENCY RESPONSE ARIZONA REGISTRAR OF CONTRACTORS 1700 W. Washington St. Suite 105 ● Phoenix AZ 85007-2812 ● 602-542-1525 ● AZ Toll Free 877-692-9762 ● Fax 602-542-1599 Janice Brewer, Governor William A. Mundell, Director 7/15/2013 Debra K. Davenport, Auditor General Office of the Auditor General 2910 North 44th Street, Suite 410 Phoenix, Arizona 85018 Dear Ms. Davenport: Please find the Registrar of Contractor’s response to your agency’s performance audit and sunset review. The Registrar of Contractors appreciates the diligence and hard work of the Auditor General’s staff in completing this performance audit, as well as their consideration of agency feedback throughout the audit process. Finding 1: ROC should consistently ensure that complaints are adequately resolved Response: Finding 1 states that the ROC should ensure that complaints are adequately resolved prior to closing them. The audit recognizes that since at least 1988, the ROC has used a unique complaint resolution process where unlike other regulatory agencies the complainant often acts as the moving party, rather than the agency. As discussed in the audit, the ROC allows a complainant to close their case for reasons such as the issues having been resolved, having received a monetary settlement, or having received a promise to complete repairs. Regardless of the reasons, it is important to recognize that complaints were closed after the person who filed the complaint showed a desire to discontinue the case. The agency agrees that improvements can be made in the complaint-handling process. The agency also believes this audit contains good recommendations for continual improvement. Recommendations: 1.1 The ROC should develop and implement policies and procedures to better ensure that complaints are adequately addressed prior to closing them. These policies and procedures should: a. Specify under what circumstances complaints should be closed based on written documentation from the contractor or complainant and/or verbal statements by the complainant indicating that corrective action had been taken; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Specify under what circumstances complaints should not be closed until ROC inspectors conduct follow-up inspections to verify that work has been properly completed; and Visit our website at: www.azroc.gov Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. c. Specify the steps the ROC will take if corrective action was not properly completed. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 1.2 The ROC should develop and implement policies and procedures to guide its use of consent agreements to discipline licensed contractors when appropriate. These policies and procedures should: a. Consider not only the nature of the violation and/or the repeat nature of the violation, but also mitigating and aggravating factors, such as whether the licensed contractor addressed workmanship problems in a timely manner; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Specify that if licensed contractors who have substantiated violations decide not to enter into a consent agreement, the ROC should proceed with its complaint-handling process by referring these cases to OAH. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 1.3 The ROC should develop and implement policies and procedures for escalating discipline for contractors with prior complaints that resulted in substantiated violations, to ensure that licensees with multiple substantiated complaints or a history of substantiated complaints receive appropriate discipline. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 1.4 The ROC should develop and implement guidelines for determining whether complaints received fall within the statutory 2-year time limit for ROC jurisdiction. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Finding 2: ROC should streamline complaint-resolution process Response: Finding 2 recommends the agency streamline the complaint-resolution process. The agency agrees that speeding up the complaint-resolution process is a vital goal. The agency has been making improvements in the complaint resolution process for several years. For example: Visit our website at: www.azroc.gov Measure Reference in Audit (FY 2011) Imposing Discipline 148 days (1 case) Issuing Citation 68-129 days (7 cases) Most Recent 6 Months 48 days (median of 286 cases) 44 days (median of 286 cases) While the significant progress has been made, the agency believes even the current timeframes can be reduced by implementing additional changes such as those discussed in this audit. Recommendations: 2.1 The ROC should expedite complaint resolution by encouraging contractors to address concerns more quickly. Specifically, the ROC should: a. Request a statutory change that would allow it to charge fees to cover the costs of processing the complaint if poor workmanship is not repaired prior to issuing a citation; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Develop and implement a mechanism to identify and track costs associated with processing specific complaints if statute is changed to give the ROC permission to charge these costs to contractors; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. c. Charge licensed contractors who are found to have committed a violation the costs for processing valid complaints if statute is changed to give the ROC permission to do so. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 2.2 The ROC should use complaint-management reports from its data system, develop and implement new reports, or develop and implement other mechanisms, as appropriate, to track and monitor open complaints. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 2.3 The ROC should develop and implement time frames for resolving complaints and for completing the key steps in its complaint-handling process. The time frames that the ROC should develop and implement include, but should not be limited to: a. Time frames for issuing citations; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Visit our website at: www.azroc.gov b. Time frames for issuing suspensions or revocation orders in cases where contractors do not respond to citations; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 2.4 The ROC should modify its complaint-handling process to help ensure complaints are resolved within the time frame it establishes. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 2.5 The ROC should develop and implement policies and procedures to ensure that licensee discipline is entered into its data system in a timely manner. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Finding 3: Problems with data system hamper ROC’s ability to perform core functions Response: Finding 3 discusses the agency computer system known as ROCIMS. During the past several years the agency has worked diligently to resolve various issues with ROCIMS. These efforts have resulted in (a) failed attempts, (b) new, unanticipated problems, and (c) relatively minimal efficiency gains for the effort put forth. In light of this experience, the agency has concluded that additional efforts to fix ROCIMS would be an inefficient use of agency resources. Recommendations: 3.1 Regardless of whether the ROC decides to correct or replace ROCIMS, it should: a. Continue with its efforts to fix the incorrect data in ROCIMS; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Provide training on data correction efforts and allocate time for its staff to identify and update all records that are incorrect; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. c. Test to ensure that the data has been corrected; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Visit our website at: www.azroc.gov d. Develop and implement practices to ensure that the data remains accurate and complete. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 3.2 To ensure that its business practices are aligned efficiently with ROCIMS or a replacement system, the ROC should: a. Analyze and document its applicable business practices; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Where appropriate, redesign its business practices to most efficiently meet its operational requirements and align applicable forms and business procedures with ROCIMS or a replacement system; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. c. Develop and implement policies and procedures to document any revised business practices; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. d. Communicate the changes to appropriate staff, including providing training where needed. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 3.3 To better ensure that its staff understand and are able to use and maintain ROCIMS or a replacement system, the ROC should provide its staff with training relevant to their use of and responsibilities for the system by: a. Developing a training plan for system users and IT staff that includes who will be trained and what they will be taught, and when training will occur; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Training staff according to the plan; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Visit our website at: www.azroc.gov c. Training new staff as they begin using the system; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. d. Providing training to address changes to the system as it occurs. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 3.4 Regardless of whether the ROC decides to correct or replace ROCIMS, to ensure that its system is appropriately managed and maintained, the ROC should: a. Improve project planning and oversight by developing, implementing, and periodically updating a project management plan; Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. b. Develop and implement a systematic, accountable, and documented process for testing and applying updates; and Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. c. Install updates after they have been properly evaluated and tested. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 3.5 To better ensure the security of information within ROCIMS or a replacement system, the ROC should plan for, incorporate, and use appropriate security controls. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 3.6 If the ROC replaces ROCIMS with a new system, it should follow the formal system development lifecycle methodology that it adopted during the course of the audit. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Sunset Factors: #2. Recommendations: 2a The ROC should develop and implement policies and procedures to refund fees to applicants whose licenses were issued or denied outside of the ROC’s 60-day time frame. Visit our website at: www.azroc.gov Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 2b The ROC should work with its Attorney General representative to determine whether it needs to refund fees to applicants from previous years whose licenses were issued or denied outside of the time frame. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. Where an application is not acted upon within 60 days, A.R.S. § 41-1077 requires the agency “refund to the applicant all fees charged for reviewing and acting on the application for the license.” These fees are commonly known as “application fees.” Application fees are commonly understood to be distinct from “license fees.” The one pays for processing the application and the other pays for the actual license. Unfortunately, ROC statutes do not clearly distinguish between application fees and license fees. A.R.S. § 32-1126 subsection (A) begins by stating the “license fees prescribed by this chapter shall be…” (emphasis added). In listing the specific fees, subsection (A), paragraph (1) then states “application fees for an original biennial license” (emphasis added). Only one fee amount is listed, not separate “application fees” and “license fees.” Consequently, statute does not clearly distinguish between application and license fees. Rule does not clarify the fee amount, but furthers the same confusion. Without clarity, for example, the agency is unsure whether the entire $890 charged for a general commercial contracting license is the license fee, or what portion of this amount is the application fee. If the entire amount is a license fee, no refund would be required by statute. If a portion of the amount is an application fee, only that portion would be required to be refunded. The agency agrees that guidance from its attorney general may add some clarity to this issue. 2c The ROC should develop and implement a method for providing additional complaint information to the public including information about the type of complaint and how the ROC resolved the complaint. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 2d The ROC should develop and implement policies and procedures to guide the ROC’s call center customer service representatives in providing information to the public. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. 9a The ROC should seek a statutory change that would allow exceptions, such as for situations in which homeowner safety is at risk or the contractor has already gone out of business, that will enable it to better protect the public. Visit our website at: www.azroc.gov Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. The agency sought this narrow health/safety exemption during the Fifty-first Legislature, First Regular Session (2013), but this language was removed from and not included in the final version of House Bill 2176. 9b The ROC should seek a statutory change to modify the statute that limits the Recovery Fund’s administrative expenses from 10 percent of the fund balance to 14 percent of the prior year revenues. Response: The finding of the Auditor General is agreed to and the audit recommendation will be implemented. On behalf of the Registrar of Contractors, I thank you and your staff for their professionalism during the audit process. Sincerely, William Mundell, Director Registrar of Contractors Visit our website at: www.azroc.gov Performance Audit Division reports issued within the last 24 months 11-06 11-07 11-08 11-09 11-10 11-11 11-12 11-13 11-14 12-01 Arizona Department of Veterans’ Services—Veteran Home Department of Corrections— Oversight of Security Operations Department of Corrections— Sunset Factors Arizona Department of Veterans’ Services—Veterans’ Donations and Military Family Relief Funds Arizona Department of Veterans’ Services and Arizona Veterans’ Service Advisory Commission— Sunset Factors Arizona Board of Regents— Tuition Setting for Arizona Universities Arizona Board of Regents— Sunset Factors Department of Fire, Building and Life Safety Arizona Game and Fish Commission Heritage Fund Arizona Health Care Cost Containment System— Coordination of Benefits 12-02 12-03 12-04 12-05 12-06 12-07 13-01 13-02 13-03 Arizona Health Care Cost Containment System—Medicaid Eligibility Determination Arizona Board of Behavioral Health Examiners Arizona State Parks Board Arizona State Schools for the Deaf and the Blind Arizona Health Care Cost Containment System—Medicaid Fraud and Abuse Prevention, Detection, Investigation, and Recovery Processes Arizona Health Care Cost Containment System—Sunset Factors Department of Environmental Quality—Compliance Management Arizona Board of Appraisal Arizona State Board of Physical Therapy Future Performance Audit Division reports Department of Environmental Quality—Underground Storage Tanks Financial Responsibility