Arizona Families F.I.R.S.T. Program Annual Evaluation Report State Fiscal Year 2014 Prepared for Department of Child Safety Phoenix, Arizona Prepared by Center for Applied Behavioral Health Policy College of Public Service & Community Solutions Arizona State University ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Acknowledgements This report was prepared by the Arizona State University, Center for Applied Behavioral Health Policy (CABHP), under contract number DES060718-001 with the Department of Child Safety (DCS), in partnership with the Arizona Department of Health Services, Division of Behavioral Health Services (AZDHS/DBHS) through the Joint Substance Abuse Treatment Fund. This report represents a reissuance of a report originally provided to the Department of Child Safety (DCS) in November 2014. DCS requested that ASU accept additional data from their largest contracted service provider due to inaccuracies in the data submitted by that provider. The authors wish to thank the following staff of the DCS and the AZDHS/DBHS for their cooperation and assistance: Jenna Shroyer, Jeanine Diaz, Nicolas Espadas, William Aldrich, of the Arizona Department of Child Safety (DCS); Patrick Birmingham, and Li Kuohsiung of the Arizona Department of Economic Security (DES), and Anne C. Dye of Arizona Department of Health Services, Division of Behavioral Health Services (AZDHS/DBHS). We would also like to thank Gary Elias, Marisol Cortez, and Jessica Mueller for their contributions to an earlier version of this year’s report. Finally, the authors wish to express appreciation to the staff of the contracted Arizona Families F. I. R. S. T. (AFF) provider agencies throughout the state: Terros, Southeastern Arizona Behavioral Health Services (SEABHS) and Arizona Partnership for Children (AzPaC). Their insights and recommended strategies for improving the utility and quality of the information contained in this report are appreciated. Most notably, their dedication to the families and children served through the AFF program is recognized and honored. Points of view represented in this report are those of the authors, and do not necessarily represent the official position of the DCS.. Suggested citation: Shafer, M. S., Mendoza, N. S., Rivera, R., Sayrs, L., Harootunian, G., Janich, N., Gururajan, S., and Chithambaran, A.K.(2015). Arizona Families F. I. R. S. T. Program: Annual Evaluation Report for the Period July 1, 2013 – June 30, 2014. Phoenix, AZ. Arizona State University. Center for Applied Behavioral Health Policy Arizona State University | 2 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 TABLE OF CONTENTS Executive Summary………………………………………………………………………... 4 Timelines, Availability and Accessibility of Services……………………………... 5 Child Safety and Reduction of Child Abuse and Neglect…………………………..5 Children of Parents in AFF………………………………………………………… 5 Recovery from Alcohol and Drug Problems............................................................. 5 Section 1: Introduction…………………………………………………………………….. 6 1.1 System Context……………………………………………………………….... 6 1.1.1 Transformation of the State Agency Authority for Child Welfare… 6 1.1.2 Regional Behavioral Health Authority Transition…………………. 7 1.1.3 Implementation of the Patient Protection & ACA…………………. 8 1.2 AFF Program Model…………………………………………………………… 8 Section 2: Evaluation Framework and Data Sources……………………………………… 11 2.1 Analytical Approach…………………………………………………………… 12 Section 3: AFF Individuals and Services Received………………………………………... 13 3.1 AFF Total Referrals & Unique Individuals……………………………………. 13 3.2 Disposition of Total Referrals to the Program…………………………………. 15 3.3 AFF Referral to Selected Events……………………………………………......16 3.4 Total Individuals Assessed……………………………………………………...17 3.5 Substance Used by AFF Individuals 30 Days Prior to Assessment…………….18 3.6 Total Unique Individuals Served and Funding Source………………………… 19 3.7 DCS and RBHA Funded Service Patterns……………………………………... 20 3.8 Level of Care……………………………………………………………………22 3.9 Patterns of AFF Closure and Length of Service……………………………….. 23 Section 4: AFF Program Outcomes……………………………………………………… 24 4.1 Child Safety……………………………………………………………………. 24 4.2 Permanency Achieved by Children of Parents in AFF………………………… 25 4.3 AFF Drug Test Data Reported by DCS/AFF Providers……………………….. 26 4.4 Employment Outcomes for Jobs-referred AFF Individuals…………………… 27 Section 5: Key Findings and Program Implications……………………………………….. 28 5.1 AFF Performance………………………………………………………………. 28 5.1.1 Increases in Timeliness, Availability, and Accessibility……………28 5.1.2 Child Safety and Reduction of Child Abuse and Neglect…………. 28 5.1.3 Children of Parents in AFF………………………………………… 28 5.1.4 Recovery from Alcohol and Drug Problems………………………. 29 5.2 AFF Implications………………………………………………………………. 29 Center for Applied Behavioral Health Policy Arizona State University | 3 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Executive Summary Arizona Families F. I. R. S. T. (Families in Recovery Succeeding Together; AFF) was established in 2000 to address adverse conditions related to alcohol and drug abuse among child welfare-involved families in which allegations of child maltreatment were associated with parents’ abuse of substances. The AFF program provides a variety of treatment and supportive services designed to reduce or eliminate abuse of and dependence on alcohol and other drugs within family systems. Interventions are provided through the Department of Child Safety (DCS), contracted community providers in outpatient and residential settings, and/or through the Regional Behavioral Health Authority (RBHA) provider network under the supervision of the Department of Health Services, Division of Behavioral Health Services (DBHS). Key elements of the AFF program include an emphasis on face-to-face outreach and engagement at the time of program referral, assessments, supportive services (e.g., transportation and housing), counseling, and recovery maintenance services. The service delivery model incorporates essential elements based on family needs, such as culturally responsive services, gender-specific treatment, family-involved treatment services, and motivational enhancement strategies to assist the entire family in its recovery. Timeliness, Availability, and Accessibility of Services  Overall, the number of unique individuals who were referred to the AFF program in SFY 2014 was 17.7% higher than SFY 2013. During SFY 2014, 7,272 referrals were made to the AFF program, representing 6,516 unique individuals.  Nearly all referrals to the AFF program (94.8%) received at least one recorded outreach attempt, and 43% of the referred individuals accepted services.  Providers outreached to referred individuals in less than one day after receipt of the referral (0.6 business days).  The reported rate of referred clients formally accepting AFF services dropped significantly, from 64% in SFY 2013 to 45.5% in SFY 2014.  On average, AFF services were initiated a just over two weeks (17.2 days) from initial referral1.  The total number of unique individuals that engaged in services during SFY 2014 was 5,464.  Slightly less than ¼ (23.5%) of the individuals served in the AFF program successfully completed the program.  The average length of service for individuals completing the AFF program was 124 days (just under 4 months). 1 See service definition in Fig. 1. Billable services are provided after the referral and once the ROI is signed. Center for Applied Behavioral Health Policy Arizona State University | 4 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Child Safety and Reduction of Child Abuse and Neglect  Eighty-eight percent of all individuals referred to the AFF program had a documented allegation of child maltreatment pre-referral; 57.6% of those allegations were substantiated.  Eighty-eight percent of individuals referred to the AFF program had no subsequent maltreatment allegation filed during the reporting period. Children of Parents in AFF  The number of children in out of home care that were associated with AFF clients in SFY 2014 6,196 and represented a 26% increase relative to SFY 2013.  Just under a third (30.3%) of the children in out of home placement achieved permanency, up 3.7% relative to SFY 2013.  Of those who achieved permanency, 89.6% did so through reunification, 4.1% through guardianship, and 6.3% through adoption. Recovery from Alcohol and Drug Problems  A total of 2,815 individuals participated in a substance abuse assessment, 84.8% of whom reported use in the past 30 days.  Marijuana (53.7%), methamphetamine (51.7%), and alcohol (44.9%), continued to be the more commonly reported substances of use.  Among the 5,464 unique individuals served by the AFF program, 61.9% were referred for drug testing.  38.1% of the individuals receiving AFF services were not referred for drug tests.  Of those individuals with reported drug tests, they were tested, on average, 2.2 times per month during their AFF program participation.  Among individuals receiving AFF services, 45.5% of the drug tests were negative, indicating no drug use. Center for Applied Behavioral Health Policy Arizona State University | 5 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 SECTION 1 INTRODUCTION Arizona Families F. I. R. S. T. (AFF) was established as a community substance use disorder prevention and treatment program by Senate Bill 1280, which passed in the 2000 legislative session. Under the requirements of the Joint Substance Abuse Treatment Fund that was established under the legislation, an annual evaluation of the AFF program is required. This evaluation of AFF examines the implementation and outcomes of community substance use disorder treatment services delivered by DCS-contracted providers. AFF is a program that provides contracted family-centered, strengths-based, substance abuse treatment and recovery support services to parents or caregivers whose substance abuse is a significant barrier to maintaining or reunifying the family or is a barrier to maintaining employment. Individuals are referred by Child Safety Specialists at DCS and by the Jobs program (i.e., mandatory employment and training program for work-eligible individuals in households receiving cash assistance). The goal of AFF is to reduce or eliminate abuse of and dependence on alcohol and other drugs, and to address other adverse conditions related to substance abuse. Interventions are provided through the Arizona Department of Child Safety (DCS), contracted community providers with services provided in outpatient and residential settings, and/or through the RBHA network of providers. In addition to traditional service, the AFF program places an emphasis on face-to-face outreach and rapid engagement at the beginning of services, supportive services to remove barriers (e.g., transportation and housing), and recovery maintenance to support ongoing sobriety and recovery. Service delivery incorporates essential elements based on family needs in conjunction with culturally responsive services, gender-specific treatment, motivational enhancement strategies, and collaboration with child service providers to assist the entire family in its recovery. 1.1 System Context During the past year, several system-wide changes occurred that significantly altered the context in which the AFF program operates. These changes, summarized below, impact the structure and organization of child welfare services, as well as the availability and funding of substance abuse treatment and related behavioral health services. 1.1.1 Transformation of the State Agency Authority for Child Welfare On January 13, 2014, during Governor Jan Brewer’s State of the State address, she announced the signing of an Executive Order to create a new Department of Child Safety (DCS), which will eliminate the Division of Children, Youth and Families. She also asked the legislature to act in Center for Applied Behavioral Health Policy Arizona State University | 6 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 order to create a new executive cabinet-level state agency from this Department. On Thursday, May 29, 2014, the Governor signed into law the legislation establishing the Department of Child Safety. This is an historic opportunity to develop the strongest possible Arizona child welfare system. The Director of the DCS is committed to working collaboratively with all public and private organizations to implement the legislation as smoothly as possible to ensure the safety and protection of Arizona’s children and families. The Director will report directly to the Governor on all administrative and policy matters involving child welfare, including foster care, adoption, the array of prevention and intervention services and the Comprehensive Medical and Dental Program. As a result of this transformation in administrative structure and the controversy around child welfare in the months leading up to this change, there has been significant upheaval in the state agency organization and representatives related to communication to the DCS (formerly DCYF) field offices and personnel, and contracted service providers. Nonetheless, throughout this process, the state agency program staff overseeing the AFF program has remained largely unchanged, although many of them had to take on additional duties and responsibilities, not associated with the AFF program, during this time. Subsequent to this administrative change, ASU has been directly engaged in redesign planning and implementation of the new organization, with particular emphasis on new employee orientation and training, continuing education, and general workforce and programmatic development initiatives. 1.1.2 Regional Behavioral Health Authority Transition In July 2013, the Arizona Department of Health Services announced that they were terminating their contract with Magellan Health, as the RBHA for Maricopa County and awarding that contract to a newly established entity, Mercy Maricopa Integrated Care. Magellan Health legally challenged the award decision, delaying the implementation of the new contract award until April 1, 2014. This drawn out process, stretching back into SFY 2013 with the release of the RFP and extending throughout the duration of the current reporting period, has created a great deal of instability within the treatment community of Maricopa County and uncertainty of general direction and focus of Medicaid and related state funded substance abuse services. At the time of this report, there remains significant issues and concerns regarding the new RBHA’s implementation on a variety of issues, most notably related to claims processing and provider reimbursement. On July 7, 2014, ADHS released RFPs for new RBHA contracts for northern and southern Arizona, with the state intended to contract with only two, rather than three entities as they do currently. These contracts are expected to be awarded during the 2015 SFY with implementation of new RBHA contracts set for October 1, 2015. It can be anticipated that this RFP process and Center for Applied Behavioral Health Policy Arizona State University | 7 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 ensuing alteration in the current RBHA structure will create instability in community behavioral health services, including the provision of substance abuse prevention and treatment. 1.1.3 Implementation of the Patient Protection & Affordability Care Act Open enrollment under the new Patient Protection & Affordability Care Act (ACA) occurred October 1, 2013 and March 31, 2014. The implementation of the ACA is significant for the AFF program on a number of fronts. Most notably, the ACA requires that health insurance plans being sold in Arizona provide access to substance abuse treatment services, not deny coverage to someone because of their substance abuse problem, and do not place a differential spending cap on substance abuse treatment, relative to caps imposed for other health conditions. The ACA is a significant development for the AFF program because it will offer an alternative means for families to access care beyond Arizona Health Care Cost Containment system (AHCCCS)funded RBHA network treatment services and DCS funded treatment services. Some families referred to the AFF program will qualify for subsidized insurance plans offered through the state’s health insurance market place or through the expanded AHCCCS program. During the ACA open enrollment period that occurred in SFY 2014, more than 300,000 Arizonans enrolled, with 60% qualifying for AHCCCS/Medicaid insurance and the balance qualifying for a subsidized Health Insurance marketplace plan2. One in five (21%) of those enrolled for health insurance during this period were children. It is unclear what the impact of this expansive access to health insurance has been upon those families served through DCS and the AFF program. Nonetheless, it is reasonable to conclude that more families involved in the child welfare system and the AFF program now have access to health insurance and that this expanded access not only includes AHCCCS, but also subsidized health insurance policies purchased through the state’s health insurance market place. 1.2 AFF Program Model The diagram on the accompanying page depicts the flow of AFF program services during SFY 2014. 2 http://slhi.org/covering-arizona/ Center for Applied Behavioral Health Policy Arizona State University | 8 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Figure 1 Overview of the AFF Program Model; SFY 2014 Center for Applied Behavioral Health Policy Arizona State University | 9 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Exhibit 1 summarizes the county, DCS provider agency, and associated RBHA within each of the five DCS regions. Exhibit 1 List of DCS Regions, Counties, DCS Providers, and RBHAs SFY 2014 DCS Provider 2014 DCS Regions County RBHA 2014 Central Maricopa East Magellan (07/01/13-3/30/14) Mercy Maricopa Integrated Care (MMIC) (> 04/01/14) Pinal Cenpatico Pima Community Partnership of Southern Arizona (CPSA) Maricopa West Magellan (07/01/13-3/30/14) Mercy Maricopa Integrated Care (MMIC) (> 04/01/14) Pima Southwest Southeast Northern Yuma La Paz Gila Cochise Graham Greenlee Santa Cruz Coconino Yavapai Apache Navajo Mohave Center for Applied Behavioral Health Policy Terros Central Terros Pima Terros Southwest South Eastern Arizona Behavioral Health Services (SEABHS) Cenpatico Northern Arizona Regional Behavioral Health Authority (NARBHA) Arizona Partnership for Children (AzPaC) Arizona State University | 10 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 SECTION 2 EVALUATION FRAMEWORK AND DATA SOURCES This evaluation report responds to the legislatively-mandated performance indicators of the AFF program. The data provided within this report are drawn from administrative data submitted to the evaluation team directly, or obtained from administrative information files maintained by DCS and DBHS. These data, like those reported in previous reports, include:     Client characteristics and service utilization data obtained directly from the DCScontracted providers; Child maltreatment allegation and child out of home placement information obtained through the DCS CHILDS (Children’s Information Library and Data Source); Enrollment and service utilization information for services provided through the RBHA network of providers obtained through the ADHS/DBHS CIS (Client Information System) 3 Jobs participation and TANF benefits information obtained through the DES/JAS/AZTEC (Jobs Automated System/Arizona Technical Eligibility Computer System). DCS providers use a common data reporting format developed by ASU in November 2008. These data are either uploaded or manually entered into a web-portal that was developed and maintained by ASU. The data captured through the AFF portal include patient identifiable information along with service related information associated with outreach efforts, assessment information, drug testing results and service provision. The categories of service provision that the providers report on emulates the categories of services and service levels specified by DCS in their contracts with the providers. On an annual basis, ASU provides a roster of clients, as reported by the providers to the DCS, ADHS/DBHS, and DES. These three state agencies use matching algorithms to identify individuals referred to AFF providers that also appear in their respective data systems. Where matches occur, the state agency extracts an agreed upon set of data elements and transmits that information to ASU in a secured format. ASU personnel then integrate these data from these four data sources (providers, DCS, ADHS/DBHS, DES) to create an interoperable data set that serves as the basis for this report. As with any evaluation study that relies upon administrative data, such as those used in this report, there are varying degrees of data quality, including missing data elements that are inconsistent and uneven, vagaries in reporting, and logical consistency. 3 DBHS encounters data are entered into the CIS within 210 days of service provision. Consequently, DBHS data presented in this report may not fully reflect all services provided during the reporting period. Center for Applied Behavioral Health Policy Arizona State University | 11 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Nonetheless, the use of such data provide a cost effective approach that is commonly used in program evaluation, especially for large scale evaluations such as the one reported herein. 2.1 Analytic Approach The data analysis and data query processes utilized in this report remains largely unchanged from the approach in the SFY 2013, with one major exception. In contrast to previous Annual Reports, the timeframe for these analyses is 12 months (July 1, 2013 – June 30, 2014) rather than the 9 month timeframe previously utilized (July 1, 2013 – March 30, 2014). This expansion in reporting period was made possible as a result of some innovations and efficiencies created by the ASU evaluation team. The timeframe between the end of the fiscal year (June 30) the state agencies’ transmittal of external data to ASU (beginning August 4, 2014 with the final transmittal being received on September 11, 2014) is quite short. However, the current report includes additional data uploaded from Terros in January 2015. A variety of data elements, particularly service encounter data reported by ADHS/DBHS takes a relative long period to be fully captured, due to the multi-layering of reporting systems. As such, some of the data contained within this report, particularly that associated with service utilization is likely to underrepresent the true extent or volume of services provided. The extent of this under-representation is not known at this time. Center for Applied Behavioral Health Policy Arizona State University | 12 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 SECTION 3 AFF INDIVIDUALS AND SERVICES RECEIVED 3.1 AFF Total Referrals & Unique Individuals As depicted in Exhibit 2, there were 7,272 total referrals to the AFF program during SFY 2014. During this period of time, 6,516 unique individuals were referred to the program, with a small proportion of individuals referred more than once. The number of total referrals to the AFF program increased by 13% for SFY 2014 as compared to SFY 2013 while the number of unique individuals referred to the AFF program increased by nearly 18%. The greatest peak in referrals was observed during quarter four (22%). Exhibit 2 AFF Total Referrals & Unique Individuals SFY 2014 2013 2013 2014 Change Changec a b (Reported ) (Updated ) # # % July-Sep (Q1) 1,599 1,586 1,700 114 7.2 Oct-Dec (Q2) 1,556 1,513 1,664 151 10.0 Jan-March (Q3) 1,664 1,636 1,852 216 13.2 Apr-June (Q4) 1,656 1,682 2,056 374 22.2 Total Referrals 6,475 6,417 7,272 855 13.3 Unique Individuals 5,541 5,537 6,516 979 17.7 a These numbers were reported in the SFY 2013 AFF Annual Evaluation. b These numbers represent data that have been updated by the providers since the completion of the AFF Annual Evaluation SFY 2013, which accounts for any lag in data entry from AFF providers and incorporates the new approach (See Section 2.1.1) c Calculated as ([SFY2014 - SFY 2013 updated]/SFY 2013 updated). Center for Applied Behavioral Health Policy Arizona State University | 13 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Figure 2 displays the number of total referrals and unique individuals referred by quarter from SFY 2010 through SFY 2014. Figure 2 AFF Total Referrals and Unique Individuals, by Quarter, SFY 2010 – SFY 2014 Center for Applied Behavioral Health Policy Arizona State University | 14 ARIZONA FAMILIES F.I.R.S.T. 3.2 Annual Evaluation Report SFY 2014 Disposition of Total Referrals to the Program Exhibit 3 provides a comparison of client outreach and engagement patterns across the four quarters in SFY 2014 and SFY 2013. Disposition of referrals is categorized according to: individuals who providers reported had accepted services through consent and Release of Information or ROI (i.e., a signed statement allowing the provider agency to share information with specific individuals or groups), closed prior to providing consent or ROI or refusing services refused, and referrals still in process (the provider has not yet provided additional information about services or filed a closure report). The proportion of referrals that were reported as accepting services decreased by 19% in SFY 2014, relative to 2013. It is not clear from the information provided to ASU if this represents an actual decrease in service acceptance, or inaccurate reporting by the service provider. The proportion of referrals that were closed prior to the client consenting to receive services (indicated by signing of the ROI) decreased by 7% relative to the previous year. Finally, the proportion of referrals still process, jumped from 862 to 2,842 in SFY 2014. Alternatively, this increase in referrals in process, accompanied by the drop in clients reported to have accepted services may reflect inconsistencies in the data. While 3,309 referrals were reported to have accepted services, in fact, it appears that 3,871 (the number of individuals reported to receive AFF services) accepted services, since individuals cannot receive services without providing a release of information and formally agreeing to services. Exhibit 3 Disposition of Total Referrals to the Program SFY 2014 SFY 2013a SFY 2014 (All four quarter data) (All four quarter data) Total # of Referrals Total Accepting Services (Signed ROI) Total Closed Referrals (w/out Signed ROI) Total Referrals Still in Process b a 6,417 7,272 n 4,109 1,446 % 64.0 22.5 n 3,309 1,121 862 13.4 2,842 % 45.5 15.4 39.1 These numbers represent updated SFY 2013 data according to current analysis. b “Still in process” refers to situations such as going through the process of outreach attempts, request of DCS to delay outreach, etc. Center for Applied Behavioral Health Policy Arizona State University | 15 ARIZONA FAMILIES F.I.R.S.T. 3.3 Annual Evaluation Report SFY 2014 AFF Referral to Selected Events - Duration in Business Days As reflected in Exhibit 4, the vast majority (96.6%) individuals referred to the AFF program were contacted by their local AFF provider on the same day the provider received their referral. For those individuals for whom the provider had reported they had signed a release of information, service acceptance occurred 17.2 days4 following the referral. Providers reported that 21.1% of referrals they received were subsequently sent to a RBHA with those referrals occurring, on average, about 15.5 days after AFF referral. Slightly more than half (53.2 %) of all referrals did receive AFF services, with their first day of service coming on average, 13.4 days after their referral. This discrepancy between the proportion of referrals that received services (53.2%) and the proportion reported to have accepted services (45.5%) suggests inaccuracies in provider reporting of this latter event, since individuals cannot receive services without providing written consent and a release of information. Providers reported closing files on 1,115 referrals (pre-service), representing 15.3% of all referrals they received. Exhibit 4 AFF Referral to Selected Events - Duration in Business Days (Number of Total Referrals n = 7,272) SFY 2014 First AFF First Outreach Accepting AFF Sent to Attempt Services RBHA Servicea 7,027 3,304 1,531 3,871 96.6% 45.4% 21.1% 53.2% n % Mean # Daysb Pre-Service Closure 1,115 15.3% 0.6 17.2 15.5 13.4 52.6 Standard Deviation 1.3 14.9 13.8 15.2 25.9 b 0.0 0.0 0.0 0.0 2.0 0.0 14.0 13.0 8.0 46.0 16.0 69.0 63.0 64.0 126.0 b Minimum # Days Median # Days b Maximum # Days b a First service refers to the first event of receiving treatment service, regardless of RBHA or AFF provider and after the AFF referral date. b Records in which the date of event occurrence was considered to be an outlier (calculated as greater than 2 standard deviations from the mean) have not been included in the calculations of descriptive statistics. The number of records removed were as follows: First Outreach (131), Accepting AFF Services (153) Sent to RBHA (72), First AFF Service (198), Pre-Service Closure (59). Client acceptance of AFF services has traditionally been reported using the contractor’s interpretation of when a client accepted services. To ensure consistency, starting in SFY 2011, and continuing in SFY 2014, acceptance of AFF services was to be reported using the date a client signed the Release of Information (ROI). 4 Center for Applied Behavioral Health Policy Arizona State University | 16 ARIZONA FAMILIES F.I.R.S.T. 3.4 Annual Evaluation Report SFY 2014 Total Individuals Assessed Among those individuals referred to AFF about thirty-three percent (n =2,815) were assessed in SFY 2014. Assessments were conducted by a contracted DCS provider and/or a RBHA contracted provider, depending on the referred individual’s eligibility status for RBHA services. As summarized in Exhibit 5, among those individuals who were assessed, 48% of the individuals were assessed by DCS contracted providers only, while 37% were assessed by RBHA contracted providers only. Relative to SFY 2013, the proportion of referrals assessed by DCS providers decreased by 4.6% concomitant with a 5.1% increase in the proportion of individuals assessed by RBHA providers. The proportion of individuals assessed by both systems remained relatively unchanged from SFY 2013 (SFY13 = 12% v. SFY14 = 14%). Exhibit 5 Total Individuals Assessed SFY 2014 n % DCS Only 1,362 48.4 DCS & RHBA 402 14.3 RBHA Only 1,051 37.3 a Totals 2,815 100.0 a This figure includes individuals who had been referred to the AFF program in SFY 2013, but not assessed until SFY 2014, along with individuals who were referred and assessed during SFY 2014. Center for Applied Behavioral Health Policy Arizona State University | 17 ARIZONA FAMILIES F.I.R.S.T. 3.5 Annual Evaluation Report SFY 2014 Substance Used by AFF Individuals 30 Days Prior to Assessment Exhibit 6 provides a summary of primary substance use and all substances individuals reported using in the 30 days prior to their initial assessment. Among the individuals assessed (2,815), 85% provided information about any substances used in the preceding 30 day period, while 77% also reported their primary substance of choice. Marijuana (54%), alcohol (45%), and methamphetamine (52%) continued to be the more commonly reported substances of use. However, for more than 1/3 of the individuals assessed (35%), methamphetamine was the most commonly reported primary substance of use, with marijuana (29%) and alcohol (20%) identified as primary substances of use by smaller proportions of the assessed individuals. Exhibit 6 Substance Used by AFF Individuals 30 Days Prior to Assessment (Total Assessed Individuals: 2,815) SFY 2014 All Substance Use Reports Primary Substance Use Reports n % n % Individuals Reporting Use a Marijuana Alcohol Methamphetamine Cocaine/crack Other narcotics Heroin/Opioids Other drugs Hallucinogens Benzodiazepines Other stimulants Other sedatives Inhalants 2,388 1,282 1,072 1,234 309 224 268 115 36 44 12 7 4 84.8 53.7 44.9 51.7 12.9 9.4 11.2 4.8 1.5 1.8 0.5 0.3 0.2 2,172 618 426 762 62 82 188 16 6 6 5 1 0 77.2 28.5 19.6 35.1 2.9 3.8 8.7 0.7 0.3 0.3 0.2 0.0 0.0 a A total of 427 assessment records did not indicate any substance use; 643 assessment records did not include any primary substance use Center for Applied Behavioral Health Policy Arizona State University | 18 ARIZONA FAMILIES F.I.R.S.T. 3.6 Annual Evaluation Report SFY 2014 Total Unique Individuals Served and Funding Source A total of 5,464 unique individuals received AFF services in SFY 2014. These individuals included those who were referred in SFY 2013 and received services in SFY 2014 (n =2,013) and those individuals who were referred and received services in SFY 2014 (n =3,451). Approximately equal proportions of clients received their services funded exclusively by DCS (31.7%), exclusively by RBHA (37.4%), or a combination of DCS and RBHA sources (30.9%). The proportion of clients funded exclusively by the RBHA jumped from 23.6% in SFY 2013 to this year’s rate of 37.4%. Concomitantly, the proportion of clients with services funded by DCS (either exclusively or in combination with RBHA sources) dropped to 62.6% this year, compared to last year’s observed rate of 76.4%. Exhibit 7 Total Unique Individuals Served and Funding Source SFY 2014 n % New and Continuing AFF Individuals 5,464 100.0 New Individuals 3,451 63.2 Continuing Individuals 2,013 36.8 # of Individuals by Service Funding Source DCS Individuals Shared Individuals RBHA Individuals Total Individuals Funded by DCS and RBHAa DCS Funded Individuals RBHA Funded Individuals a 1,731 1,689 2,044 5,464 31.7 30.9 37.4 100.0 3,420 3,733 62.6 68.3 Percentages include individuals who received funding from both sources. Center for Applied Behavioral Health Policy Arizona State University | 19 ARIZONA FAMILIES F.I.R.S.T. 3.7 Annual Evaluation Report SFY 2014 DCS and RBHA Funded Service Patterns As previously noted, 3,420 individuals received AFF services funded by DCS (either exclusively or in tandem with DBHS funds) in SFY 2014. About 40% of individuals who received DCS funded services engaged in a counseling service, with the most common service being group counseling (37.3%). Relative to SFY 2013, the proportion of clients receiving DCS funded individual counseling dropped from 32.9% (SFY 2013) to 17.7%. Exhibit 8 Individuals Receiving DCS-Funded Services a SFY 2014 Unique Individuals DCS Funded Services n = 3,420b n % Substance Abuse Treatment Services Counseling Family 45 1.3 Group 1,275 37.3 Individual 605 17.7 Mental Health Services Medication 0 0.0 Medication Monitoring 6 0.2 Psychiatric Evaluation 4 0.1 Auxiliary Services Case Management 3,015 88.2 Drug Testc Re-engagement Living Skills Training Parenting Skills Concrete Supportive Services Clothing Assistance Food Assistance Housing/Rent Transportation Utilities Assistance Other 2,627 137 4 31 76.8 4.0 0.1 0.9 0 0 6 1,213 13 127 0.0 0.0 0.2 35.5 0.4 3.7 a These data do not capture RHBA funded services or other services clients may be accessing. b Ten cases (.29%) have service date but no service category c The data file for drug tests was different from the data file for rest of the DCS-Funded Services. Center for Applied Behavioral Health Policy Arizona State University | 20 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Almost all (97.6%) of the individuals who received DCS funded services received at least one auxiliary service, with case management and drug testing being the more commonly reported auxiliary services. Over 1/3 (37.7%) of individuals received at least one concrete supportive service, the most frequent being transportation, which increased from 8.8% of clients with DCS funded services in SFY 2013 to 35.5% in SFY 2014. As previously noted, 3,733 individuals received AFF services funded by RBHA sources (either exclusively or in tandem with DCS funds) in SFY 2014. The more frequently reported RBHA funded services consisted of support services (including case management) (88%) and treatment services (75.8%). Exhibit 9 Individuals Receiving RBHA-Funded Services a SFY 2014 RBHA-Funded Services Service Domain Treatment Services Rehabilitation Services Medical Services Support Services Crisis Intervention Services Inpatient Services Residential Services Behavioral Health Day Programs Unique individuals n = 3,733 n 3,614 2,829 1,014 1,058 3,286 376 104 206 15 % 96.8 75.8 27.2 28.3 88.0 10.1 2.8 5.5 0.4 a These data do not capture other services funded by DCS or other systems to which clients may have access. Center for Applied Behavioral Health Policy Arizona State University | 21 ARIZONA FAMILIES F.I.R.S.T. 3.8 Annual Evaluation Report SFY 2014 Level of Care A previously noted, 5,464 new and continuing individuals received AFF services in SFY 2014. According DCS contract specifications, DCS providers are supposed to report the levels of care for each client throughout their AFF program, regardless of the source of their service funding. In SFY 2014, 37.5% of AFF clients had no level of care reported for the entire duration of their AFF program (up from 31.2% in SFY 2013). Among unique individuals who had at least one level of care reported, nearly ½ (46.2%) were reported in the outpatient level of care, with 24% reported in the intensive outpatient level. Just under 10% (7.6%) were reported to have received services in the recovery maintenance level of care. Exhibit 10 Level of Care SFY 2014 Unique Individuals n = 5,464 n % Level of Carea Substance Abuse Awareness Outpatient Intensive Outpatient Residential - Adult Residential - Child Recovery Maintenance 3,415 62.5 276 2,524 1,310 55 416 5.1 46.2 24.0 1.0 0.0 7.6 No Level of Care Identified 2,049 37.5 a LOC categories are not mutually exclusive. Individuals can be assigned to multiple levels of care throughout their AFF program experience. Individuals assigned to the ‘No Level of Care Identified’ had no record of LOC assignment at any time during their AFF program experience. Center for Applied Behavioral Health Policy Arizona State University | 22 ARIZONA FAMILIES F.I.R.S.T. 3.9 Annual Evaluation Report SFY 2014 Patterns of AFF Closure and Length of Service Among the 5,464 new and continuing individuals reported in Exhibit 7, 58.5% had a closure reported during SFY 2014. Closures reflect individuals who exited the program at any point after a referral had been received and is inclusive of clients who dropped out, absconded, or were forcibly terminated by the provider for program noncompliance. As reflected in Exhibit 11, the most commonly reported reason for closure was discontinued, representing 48.2% (n=1,518) of all reported closures. Current reporting mechanisms do not allow for a determination of whether the provider or the client discontinued the participation. Fewer than 1 in 4 clients (23.5%) with a closure code were reported to have completed their AFF program. For these individuals, their average length of program participation was just over 4 months (124 days), although the median program length (92 days) suggests a much briefer period of AFF program participation. Exhibit 11 Patterns of AFF Closure and Length of Service SFY 2014 Standard n Closure Reason Completed Discontinued Refused services Unable to locate No SA Problem Incarcerated Moved out of Area Death Total c a a 741 1,518 82 544 99 93 67 7 3,151 % 23.5 48.2 2.6 17.3 3.1 3.0 2.1 0.2 100.0 b b Median Mean 92.0 85.0 20.5 25.5 18.0 48.5 62.0 51.0 64.0 123.5 101.0 23.9 36.9 22.3 67.1 75.4 88.3 90.4 Deviationb 114.9 77.1 23.9 40.5 21.0 66.7 66.1 76.0 87.5 51 records were not included because they had an unknown closure reason code. b To compute descriptive statistics for this exhibit, 346 records were not included in the calculations because either the last service date occured after case close date (n =210) or the length of service was an outlier (n =136 records). Length of service was considered to be an outlier when it was greater than 2 standard deviations from the mean. The number of records (treatment date > closure date & outliers) removed were as follows: Completed (43 & 29), Discontinued (52 & 77), Refused Services (24 & 2), Unable to Locate (43 & 19), No SA Problem (28 & 2), Incarcerated (8 & 5), Moved out of Area (17 & 2). c Among the 3,151 records and 2,800 records, there were 3146 and 2,795 unique inidividuals, respectively. Center for Applied Behavioral Health Policy Arizona State University | 23 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 SECTION 4 AFF PROGRAM OUTCOMES This section highlights the outcomes achieved by individuals who participated in the AFF program. Outcomes are assessed in the following legislatively-specified domains: child safety, family stability and permanency, recovery from alcohol and drug abuse, and self-sufficiency as reflected by employment. 4.1 Child Safety Among the 5,464 new and continuing individuals that received AFF services in SFY 2014, approximately 88.2% had at least one allegation of child maltreatment prior to their referral to the AFF program. The majority of these individuals (57.6%) had at least one allegation that rose to the level of substantiation. For those the individuals with a maltreatment report at the time of AFF program referral, those with substantiated findings showed a recurrence rate of approximately 12.6%5, while those with unsubstantiated findings at AFF referral demonstrated a slightly higher rate of recurrence of 20.7%6. Overall, regardless of maltreatment allegation status at the time of AFF referral, 87.7% of all AFF individuals had no subsequent filing of an allegation of child maltreatment, consistent with previously reported recurrence patterns. Substantiated Proposed Unsubstantiated Other No Report Total Exhibit 12 Pre-AFF and Post-AFF Referral Report Findings SFY 2014 Post Referral Allegation Finding Pre-Referral Unsubstantiated Other Allegation Finding Substantiated Proposed n % n % n % n % n % 3,195 57.6 93 2.9 78 2.4 169 5.3 64 2.0 793 14.3 9 1.1 18 2.3 26 3.3 10 1.3 856 15.4 40 4.7 25 2.9 95 11.1 17 2.0 54 1.0 0 0 1 1.9 0 0 7 13.0 653 11.8 9 1.4 6 0.9 12 1.8 5 0.8 5,551a 100.0 151 2.7 128 2.3 302 5.4 103 1.9 No Report n % 2,791 87.4 730 92.1 679 79.3 46 85.2 621 95.1 4,867 87.7 a Eighty-seven duplicate records were included in the data provided by DCS and ASU was unable to discern which record to retain so both were retained. 5,551-87=5464 (continuing & new clients) 5 6 This is the sum of post-referral allegations across those that were substantiated at pre-referral. This is the sum of post-referral allegations across those that were identified as unsubstantiated at pre-referral. Center for Applied Behavioral Health Policy Arizona State University | 24 ARIZONA FAMILIES F.I.R.S.T. 4.2 Annual Evaluation Report SFY 2014 Permanency Achieved by Children of Parents in AFF Among new and continuing AFF clients served in 2014, a total of 6,196 children whose parent was the named perpetrator and AFF client were in out of home placement at some point during the reporting period. Compared to SFY 2013, the number of children in out of home placement has increased by 26%. Among these children, more than 2/3 (68.6%) continued to be in out of home placement at the end of the reporting period. Just under 1/3 (30.3%) of these children had achieved permanency, up 3.7% relative to SFY 2013. For those children achieving permanency, reunification represented the majority of cases (89.6%) with adoption (6.3%) and guardianship (4.1%) reflecting balance of permanency decisions. The number of days in out of home placement ranged from a median of 188 days for reunifications to 483 days for adoption. Exhibit 13 Permanency Achieved by Children of Parents in AFF SFY 2014 n % Total Children 6,196 100 Still in Care 4,249 68.6 Other 71 1.1 Achieved Permanency 1,876 30.3 Days in Out of Home Care Among Children Achieving Permanency Achieved Permanency Reunification Guardianship Adoption Center for Applied Behavioral Health Policy n 1,876 1,681 77 118 % Median Days Mean Days 100 89.6 175 188 4.1 346 305 6.3 497 483 Arizona State University | 25 ARIZONA FAMILIES F.I.R.S.T. 4.3 Annual Evaluation Report SFY 2014 AFF Drug Test Data Reported by DCS/AFF Providers Among the 5,464 new and continuing clients that received AFF services in SFY 2014, just under 2/3 (61.9%) had been referred for drug testing at least once during the period of their AFF program participation. In contrast, no records of drug test referrals were reported by DCS providers for 2,083 clients (38.1% of clients served). Nearly 30% (28.4%) of all AFF clients were reported to have been referred twice or more for each month of their AFF program participation. On average, AFF clients with drug test referrals were referred for testing 2.2 times per month. Nearly 40,000 (39,565) drug test results were reported for those clients with reported drug test referrals (regardless of the frequency of such referrals). The relative rates of drug detection from these results and drug abstinence were comparable, with negative test results observed for 45.5% of the tests and positive results observed for 46.4% of the tests. Exhibit 14 AFF Drug Test Data Reported by DES/AFF Providers SFY 2014 Annual report Unique AFF Individuals with Services n Total Unique Individuals 5,464 With Referrals for Drug Tests 3,380 Without Referrals for Drug Tests 2,083 With at least two (2) referrals per month 1,554 % 100.0 61.9 38.1 28.4 Standard Deviation 2.19 1.88 Mean Drug referrals per month a Drug Test Referral Outcomes Negative Positive Pending Refused / Altered Cancelled Total n 17,989 18,345 1,443 1,756 32 39,565 % 45.5 46.4 3.6 4.4 0.1 100.0 a Descriptive statistics were computed for individuals (n =3,078) with at least a month (30 days) of elapse time. Median = 1.75. Center for Applied Behavioral Health Policy Arizona State University | 26 ARIZONA FAMILIES F.I.R.S.T. 4.4 Annual Evaluation Report SFY 2014 Employment Outcomes for JOBS-referred AFF Individuals Among the 5,464 new and continuing clients served by the AFF program in SFY 2014, 235 (4.3%) were also enrolled in JOBS at some time during the year. Unfortunately, employment status was reported by providers for only 60 of these concurrently enrolled clients at both intake and closure, allowing for a comparison of employment status prior to and at the conclusion of AFF program participation. For these 60 clients who were concurrently enrolled in AFF and JOBs, 28.3% (n=17) were reported to be employed at the time of their AFF assessment, with 38.3% (n=23) reporting employment at the time of their AFF program closure. Among those individuals served by the AFF program who were not concurrently enrolled in JOBs (n=5,229), employment status was not reported at assessment and/or closure for 77% of the cases. Among those individuals for whom employment status was reported (n = 1,205), 40.9% (n=493) were reported to be employed at assessment, with 45.8% (n=552) reported employed at AFF closure. SECTION 5 KEY FINDINGS AND PROGRAM IMPLICATIONS This report summarizes the key processes and outcomes of the Arizona Families F. I. R. S. T. program, now in its 14th year of operation. The continued commitment of the legislature to examine the processes and outcomes of this innovative program has afforded the opportunity to systematically evaluate the effectiveness and impact of a program unique in its scope and focus. As part of this annual evaluation, independently conducted by Arizona State University, the performance of the AFF program, in relation to each of the goals articulated by the legislature, was addressed by utilizing information from a variety of sources, including administrative data and service utilization records. 5.1 AFF Performance 5.1.1 Increases in Timeliness, Availability, and Accessibility of Services. In SFY 2014, 7,272 total referrals, representing 6,516 unique individuals, were made to the AFF program statewide. This represents a 17.7% increase in referrals relative to SFY 2013. For the individuals referred and their families, the AFF program continues to provide services in a manner consistent with the program design. During SFY 2014, AFF participants received outreach, assessment, engagement, and substance abuse treatment services in a timely manner. AFF providers made efforts to contact referred individuals within one business day. AFF services were accepted by participants, on average, within 17.2 days of referral to AFF. Center for Applied Behavioral Health Policy Arizona State University | 27 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 Those individuals who were engaged in AFF services typically found themselves receiving services from their local AFF provider and/or a RBHA contracted treatment provider in their community, depending upon their program. In SFY 2014, 48% of AFF individuals received AFF services with funding solely provided by DCS, compared to about 37% of individuals who received services funded by RBHA. Fourteen percent of AFF individuals received services through a combination of DCS and RBHA sources. 5.1.2 Child Safety and Reduction of Child Abuse and Neglect. Most individuals (88.2%) served by the AFF program had at least one allegation of child maltreatment prior to enrolling in the program; 87.7% of all AFF individuals had no subsequent maltreatment report filed during this report period. 5.1.3 Children of Parents in AFF. Among children reported to be under DCS care during the reporting period of July 1, 2013 to June 30, 2014, 68.6% were reported to still be in care, while the permanency rate was reported at 30.3%. Of those who achieved permanency, the majority (89.6%) did so through reunification. 5.1.4 Recovery from Alcohol and Drug Problems. Consistent with previous years, the overwhelming majority of AFF individuals assessed in SFY 2014 self-reported use of marijuana, alcohol, and methamphetamine. The results of drug screens conducted with AFF individuals to detect continued drug use indicated that 45.5% of AFF individuals were drug free throughout their AFF participation. Individuals receiving AFF services were drug tested, on average, 2.2 times per month during AFF program participation. The results of this evaluation of the Arizona Families F. I. R. S. T. program provide continuing evidence of the efficiency with which this program engages referred clients into treatment, the consistency and quality of the program components that individuals receive, and the impact that those services have upon the reduction of parental substance abuse and child neglect, family reunification, and permanency planning. The AFF program places a strong emphasis upon the utilization of evidence-based substance abuse practices, an emphasis on family-focused and recovery-oriented supportive services, and a focus on quality management and program monitoring. 5.2 AFF Implications Research on the intersection of child maltreatment and substance abuse indicates that one-third to two-thirds of all child maltreatment cases involve substance use7. Substance abuse and child maltreatment occur across cultural, socioeconomic, and geographical boundaries. In Arizona, there are no consistent patterns of substance use across geographical service areas, suggesting a 7 Office on Child Abuse and Neglect, Children's Bureau. Goldman, J., Salus, M. K., Wolcott, D., Kennedy, K. Y. (2003) A Coordinated Response to Child Abuse and Neglect: The Foundation for Practice, Chapter 5. Center for Applied Behavioral Health Policy Arizona State University | 28 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 need for services tailored to the needs of the population in a given area8. However, recent statistics reported by the University of Arizona indicated that in 2013 substance abuse treatment admissions were stable or increase from prior years9. Similarly, AFF referrals are on the rise, demonstrating a clear, increasing trend over time. Ultimately, substance abuse among vulnerable, child welfare-involved families remains a formidable problem. To illustrate the growing problem of substance abuse among child welfare-involved families, we reported that in the AFF Program, 84.8% of all individuals used substances in the 30 days prior to their referral. In the 2012 Arizona Substance Abuse Prevention and Treatment Services Capacity Report, the proportion of individuals (from the general population) reporting alcohol use in the past 30 days ranged from 50-55% and between 10-14% for illicit drugs. Treatment episode data for Maricopa County indicated 27% related to alcohol abuse, 23% related to methamphetamine abuse, and 19% related to opioid abuse10. With respect to individuals involved with criminal justice11, 69% reported alcohol use in the preceding 30 days, with nearly 50% reporting methamphetamine, marijuana, or other illicit substance use. All told, AFF-involved families represent a population at- risk exceeding the substance use of other treatment-involved populations. Compounding risk associated with substance use are the familial issues related to child well-being. National studies show that the rate of maltreatment occurs at a total incidence of 17.1 per 1,000 children in the general population12. This rate is equivalent to 1.71 per 100 children, further suggesting that, in the United States, one child out of every fifty-eight children has experienced maltreatment13. In Arizona, the number of children under DCS care continues to increase. The Child Welfare Reporting Requirements14, Semi-annual Report indicated that, of 22,547 reports meeting statutory criteria for maltreatment, 7% were substantiated with an additional 1,190 proposed for substantiation. While data may not be directly compared across information sources in these reports, we may assume that families associated with AFF clients make up a 8 Wolfersteig, W.L., Fernandez, K.M., & Hoffman, K. (2012, September). 2012 Arizona Substance Abuse Prevention and Treatment Services Capacity Report. Phoenix, AZ: Southwest Interdisciplinary Research Center, Arizona State University. 9 Cunniningham, J. K. (2014, June). 2013 Drug Trends in Phoenix and Arizona. Department of Family and Community Medicine and the Native American Research and Training Center, College of Medicine, The University of Arizona, Tucson, Arizona. 10 Ibid. 11 Choate, David E. (2012). Arizona Arrestee Reporting Information Network: 2012 Maricopa County Manager’s Office Report on Substance Use and Public Health Concerns among Arrestees. Phoenix, AZ: Center for Violence Prevention & Community Safety, Arizona State University. 12 Sedlak, A.J., Mettenburg, J., Basena, M., Petta, I., McPherson, K., Greene, A., & Li, S. (2010). Fourth National Incidence Study of Child Abuse and Neglect (NIS–4): Report to Congress. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families. 13 Ibid. 14 Department of Economic Security (DES), Division of Children, Youth and Families (DCYF). (2013, October 1). Child Welfare Reporting Requirements, Semi-annual Report for the Period of October 1, 2013 Through March 31, 2014. Retrieved from https://www.azdes.gov/InternetFiles/Reports/pdf/semi_annual_child_welfare_report_oct_2013_mar_2014.pdf Center for Applied Behavioral Health Policy Arizona State University | 29 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2014 considerable portion of substantiated cases (7,272 total AFF referrals in 2014). However, encouraging information indicates that most (87.7%) of the families served in the AFF program had no recurrence of maltreatment allegation during their AFF program participation during the reporting period. Moreover, permanency rates in 2014, compared to SFY 2013, suggest an increase in goals achieved including increase in reunification, guardianship, and adoption rates. It is clear that the AFF program continues to target a high risk population, consistent with the legislative intent of the program. These data also provide continuing evidence of the program’s impact in reducing parental substance abuse child maltreatment, consistent with the legislative goals of the program. What remains unclear at this time are the long-term impacts of the AFF program upon the program participants and their families with regard to family reunification, parental sobriety, and parenting. DCS-initiated changes in the AFF program design, in consultation with the ASU evaluation team, will allow for more systematic and rigorous assessment of the long term impacts of this program in the future. Center for Applied Behavioral Health Policy Arizona State University | 30