Janice K. Brewer Governor Clarence H. Carter Director The Honorable Andy Biggs President of the Senate Arizona State Senate 1700 West Washington Phoenix Arizona 85007 Dear President Biggs: Pursuant to A.R.S. § 8-884, the Arizona Department of Economic Security (DES) is pleased to provide the Arizona Families F.I.R.S.T. (AFF) program evaluation report for the state fiscal year 2013 (SFY 2013). This annual evaluation report prepared by the independent evaluator, Arizona State University/Center for Applied Behavioral Health Policy, provides analysis of program implementation, service utilization and program outcome data. Highlights contained within the enclosed report include:  During the SFY 2013, 5,541 unique individuals were referred to AFF program, representing a 10 percent decrease from SFY 2012.  During the first three quarters of SFY 2013, a total of 5,011 clients were served by the AFF program, representing a 34 percent increase over the previous year.  Outreach efforts continued to occur in a timely manner, averaging .70 business days which is a slight increase from SFY 2012.  In SFY 2013, almost 89 percent of AFF individual had no subsequent maltreatment report filed during the first nine months of the reporting period of this report. If you have any questions, please contact Flora Sotomayor, Assistant Director, Division of Children, Youth and Families, at (602) 542-3598. Sincerely, Clarence H. Carter Director Enclosure cc: Speaker Andy Tobin, Arizona State House of Representatives Members of the Joint Legislative Audit Committee Joan Clark, Director, Arizona State Library, Archives and Public Records _____________________________________________________________________________________________ 1717 W. Jefferson, S/C 010A, Phoenix, AZ 85007  P.O. Box 6123, Phoenix, AZ 85005 Telephone (602) 542-5678  Fax (602) 542-5339  www.azdes.gov Arizona Families F.I.R.S.T Program Annual Evaluation Report State Fiscal Year 2013 Prepared for Arizona Department of Economic Security Division of Children, Youth and Families Phoenix, Arizona Prepared by Center for Applied Behavioral Health Policy College of Public Programs Arizona State University ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 Acknowledgements This report was prepared by the Center for Applied Behavioral Health Policy (CABHP), College of Public Programs, Arizona State University, under contract number DES060718-001 with the Arizona Department of Economic Security, Division of Children, Youth and Families (ADES/DCYF), in partnership with the Arizona Department of Health Services, Division of Behavioral Health Services (AZDHS/DBHS) through the Joint Substance Abuse Treatment Fund. The authors wish to thank the following staff of the ADES/DCYF and the AZDHS/DBHS for their ongoing cooperation and assistance: Jenna Shroyer, Leanne Hawkins, Jeanine Diaz, Nicolas Espadas, William Aldrich, and John Blankeship of ADES; and Michael Sheldon of AZDHS/DBHS. We would also like to thank Jessica Mueller, Mayra Lozano Porras, and Ashley Watson, CABHP students that contributed to this year’s report through research support. Finally, the authors wish to express appreciation to the staff and directors of the contracted AFF provider agencies throughout the state. Their insights and recommendations of strategies for improving the utility and quality of the information contained in this report are appreciated. Most notably, their dedication to the families and children served through the AFF program is recognized and honored. Points of view represented in this report are those of the authors, and do not necessarily represent the official position or policies of either the ADES or the ADHS. Suggested citation: Shafer, M.S., Elias, G., Rivera, R. Mendoza, N. & Cortez, M. (2013). Arizona Families F.I.R.S.T. Program: Annual Evaluation Report for the Period July 1, 2012 – March 31, 2013. Phoenix, AZ. Arizona State University. Center for Applied Behavioral Health Policy Arizona State University | 2 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 TABLE OF CONTENTS Acknowledgements ..........................................................................................................................2 Table of Contents .............................................................................................................................3 Executive Summary .........................................................................................................................4 Section 1: Introduction .....................................................................................................................6 Figure 1: Overview of the AFF Program Model; SFY 2013 ...............................................7 Exhibit 1: List of DES Regions, DES Counties, and RBHA Providers ..............................8 Section 2: Evaluation Framework and Data Sources .......................................................................9 Section 3: Arizona Families F.I.R.S.T. Individuals and Services Received ..................................11 Exhibit 2: AFF Total Referrals & Unique Individuals ......................................................11 Figure 2: AFF Total Referrals by Quarter, SFY 2009-SFY 2013 .....................................12 Exhibit 3: Disposition of Total Referrals to the Program ..................................................13 Exhibit 4: AFF Referrals to Selected Events – Duration in Business Days ......................14 Exhibit 5: Total Individuals Assessed for SFY 2013.........................................................15 Exhibit 6: Substances Used by AFF Individuals 30 Days Prior to Enrollment .................16 Exhibit 7: Total Individuals Served and Funding Source ..................................................17 Exhibit 8: Individuals Receiving DES Funded Services ...................................................18 Exhibit 9: Individuals Receiving RBHA Funded Services ................................................19 Exhibit 10: Level of Care ...................................................................................................20 Exhibit 11: Patterns of AFF Closure and Length of Service .............................................21 Section 4: AFF Program Outcomes ...............................................................................................22 Exhibit 12: Pre-AFF and Post-AFF Referral Report Findings ..........................................22 Exhibit 13: Permanency Achieved by Children of Parents in AFF ...................................23 Exhibit 14: AFF Drug Test Data Reported by DES/AFF Providers ..................................24 Section 5: System Context and Summary......................................................................................25 Center for Applied Behavioral Health Policy Arizona State University | 3 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 Executive Summary Arizona Families F.I.R.S.T. (Families in Recovery Succeeding Together; AFF) was established in 2000 to address adverse conditions related to alcohol and drug abuse among child welfareinvolved families in which allegations of child maltreatment were associated with parents’ abuse of substances. The AFF program provides a variety of treatment and supportive services designed to reduce or eliminate abuse of and dependence on alcohol and other drugs within family systems. Interventions are provided through the Department of Economic Security, Division of Children, Youth and Families (DES/DCYF), contracted community providers in outpatient and residential settings, and/or through the Regional Behavioral Health Authority (RBHA) provider network under the supervision of the Department of Health Services, Division of Behavioral Health Services (DBHS). Key elements of the AFF program include an emphasis on face-to-face outreach and engagement at the time of program referral, assessments, supportive services (e.g., transportation and housing), counseling, and recovery maintenance services. The service delivery model incorporates essential elements based on family needs, such as culturally responsive services, gender-specific treatment, family-involved treatment services, and motivational enhancement strategies to assist the entire family in its recovery. Similar to last year1, beginning with Exhibit 4, this year’s annual evaluation report is restricted to the first nine months of the state fiscal year, in order to provide more accurate and more detailed analysis; key findings for SFY 2013 are as follows: Timeliness, Availability, and Accessibility of Services  Overall, unique individuals who were referred to the AFF program for all four quarters of SFY 2013 decreased by 10% compared to SFY 2012, this can be attributed to the contract changes among providers. During all four quarters of SFY 2013, 5,541 unique individuals were referred to the AFF program, constituting 6,475 total referrals, averaging 1,619 total referrals per quarter.  Nearly all referrals to the AFF program (91.7%) received at least one recorded outreach attempt, and over half of the individuals referred (57.9%) accepted services, representing a decrease from last year in both recorded outreach attempts and accepted services.  During the first three quarters of SFY 2013, outreach occurred in a timely manner, averaging 0.7 business days, meaning that outreach occurred on the same day that the referral was received.  On average, AFF services were initiated about two weeks (15.0 days) after receiving a referral2. 1 Throughout this report, reference will be made to SFY 2012; these references will be to the first three quarters of the SFY (i.e. for the period of July 1, 2011 – March 31, 2012), unless noted otherwise. 2 See service definition in Fig. 1. Billable services are provided after the referral and once the ROI is signed. Center for Applied Behavioral Health Policy Arizona State University | 4 ARIZONA FAMILIES F.I.R.S.T.   Annual Evaluation Report SFY 2013 The total number of unique individuals that engaged in services during the first three quarters of SFY 2013 was 4,531. The average length of service for individuals completing the AFF program was 182 days (about 6 months); among individuals whose cases were closed, 25% completed the program. Child Safety and Reduction of Child Abuse and Neglect  Almost 89% of all AFF individuals had no subsequent maltreatment report filed during the first nine months of the reporting period of this report. Children of Parents in AFF  The number of children in out of home care that were associated with AFF clients increased by nearly 15% relative to SFY 2012 to 4,705.  More than one-fourth (26.6%) of the children in out of home placement achieved permanency, up 5% relative to SFY 2012.  Of those who achieved permanency, 92.1% did so through reunification, 5.4% through guardianship, and 2.5 through adoption. Recovery from Alcohol and Drug Problems  A total of 2,185 individuals participated in a substance abuse assessment, 91% of whom reported use in the past 30 days.  Of the 91% who reported use in the past 30 days, marijuana (56.5%), alcohol (53.9%), and methamphetamine (47.9%) continued to be the most commonly reported substances of use.  Among the 4,531 unique individual served by the AFF program in the first three quarters, 57.3% were referred for drug testing.  On average, AFF individuals were tested 1.7 times per month during the first three quarters of SFY 2013.  Among individuals receiving AFF services, 80% of the drug tests were negative, indicating no drug use. Center for Applied Behavioral Health Policy Arizona State University | 5 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 SECTION 1 INTRODUCTION Arizona Families F.I.R.S.T. (AFF) was established as a community substance use disorder prevention and treatment program by Senate Bill 1280, which passed in the 2000 legislative session. Under the requirements of the Joint Substance Abuse Treatment Fund that was established under the legislation, an annual evaluation of the AFF program is required. This evaluation of AFF examines the implementation and outcomes of community substance use disorder treatment services delivered by DES-contracted providers. AFF is a program that provides contracted family-centered, strengths-based, substance abuse treatment and recovery support services to parents or caregivers whose substance abuse is a significant barrier to maintaining or reunifying the family or is a barrier to maintaining employment. Individuals are referred by Child Protective Services (CPS) and by the Jobs program (i.e., mandatory employment and training program for work-eligible individuals in households receiving cash assistance). The goal of AFF is to reduce or eliminate abuse of and dependence on alcohol and other drugs, and to address other adverse conditions related to substance abuse. Interventions are provided through the Arizona Department of Economic Security, Division of Children, Youth and Families (ADES/DCYF), contracted community providers with services provided in outpatient and residential settings, and/or through the RBHA provider network. In addition to traditional services, AFF includes an emphasis on face-to-face outreach and engagement at the beginning of services, supportive services to remove barriers (e.g., transportation and housing), and recovery maintenance to support ongoing sobriety and recovery. Service delivery incorporates essential elements based on family needs in conjunction with culturally responsive services, gender-specific treatment, motivational enhancement strategies, and collaboration with child service providers to assist the entire family in its recovery. At the beginning of the third quarter, DES awarded new contracts for the AFF program. The transition period occurred during the months of February 1, 2013 to April 30, 2013. During this period, old providers were not to accept new clients and were to transition clients to new providers. As a result, the impact of the transition can attribute to challenges in client transfers and data tracking. Center for Applied Behavioral Health Policy Arizona State University | 6 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 The following diagram shows the flow of AFF program services beginning when the individual was initially referred by his/her case manager under the new contract requirements beginning February 1, 2013. Figure 1 Overview of the AFF Program Model; SFY 2013 Center for Applied Behavioral Health Policy Arizona State University | 7 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 Exhibit 1 summarizes the county, DES provider agency, and associated RBHA within each of the five DES regions. Also depicted in Exhibit 1 is the transition that took effect February 1, 2013, in which new program contracts were awarded to providers. The report will reference providers between 2012 and 2013 as old versus new. Exhibit 1 List of DES Regions, Counties, DES Providers, and RBHAs SFY 2012 – 2013 DES Regions Central Pima County DES Provider 2012 RBHA 2012 DES Provider 2013 RBHA 2013 Maricopa East TERROS Magellan Magellan Pinal No provider Cenpatico Cenpatico Pima Community Partnership of Southern Arizona (CPSA) Community Partnership of Southern Arizona (CPSA) Community Partnership of Southern Arizona (CPSA) Maricopa West TERROS Magellan Southwest Yuma La Paz TERROS Magellan Arizona Partnership for Children (AzPaC-Yuma) WestCare Arizona Cenpatico Gila Southeast Cochise Graham Greenlee Santa Cruz Coconino Yavapai Northern No provider Southern Arizona Behavioral Health Services (SEABHS) Southern Arizona Behavioral Health Services (SEABHS) Arizona Partnership for Children (AzPaC-Coconino) Navajo Old Concho Community Assistance Center Mohave WestCare Arizona Apache Cenpatico Center for Applied Behavioral Health Policy Northern Arizona Regional Behavioral Health Authority (NARBHA) Arizona Partnership for Children (AzPaC-Northern) Northern Arizona Regional Behavioral Health Authority (NARBHA) Arizona State University | 8 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 SECTION 2 EVALUATION FRAMEWORK AND DATA SOURCES This evaluation report responds to the legislatively-mandated performance indicators of the AFF program. The data provided within this report are drawn from administrative data submitted to the evaluation team directly, or obtained from administrative information files maintained by DES and DBHS. These data, like those reported in previous reports, include:     Service utilization data obtained directly from the DES-contracted providers Enrollment and encounter data provided by the Arizona Department of Health Services, Division of Behavioral Health Services (ADHS/DBHS) for services provided through the local RBHA networks DES CHILDS (Children’s Information Library and Data Source) data, which provides child welfare information DES JAS/AZTEC (Jobs Automated System/Arizona Technical Eligibility Computer System) data, which provides employment services information DES providers use a common data reporting format deployed by the Arizona State University Center for Applied Behavioral Health Policy (CABHP) in November 2008. These data are either manually entered into the AFF data entry web-portal or uploaded by providers to the web-portal. The data captured through the AFF web-portal include information regarding outreach efforts, assessment information, drug testing results and service provision using a service matrix that emulates the categories of services utilized by DES as required by contract. For those client services that are funded through DBHS/RBHA, enrollment and service encounter data are provided by DBHS. These data are derived from the DBHS Client Information System (CIS)3. Two additional data sets used for this evaluation include: the DES CHILDS information system, which provides child welfare data on allegations of child maltreatment and permanency outcomes and the DES JAS/AZTEC information system, which provides employment services data. 3 DBHS encounters data are entered into the CIS within 210 days of service provision. Consequently, DBHS data shown may not fully contain all services provided in these time frames. Center for Applied Behavioral Health Policy Arizona State University | 9 ARIZONA FAMILIES F.I.R.S.T. 2.1 Annual Evaluation Report SFY 2013 Analytic Approach In consultation with DES representatives, the information presented throughout this report is based on either the 9 months of the state fiscal year or the full 12 months of the state fiscal year. Data reconciliation processes at the provider and agency (DES, RBHA) occur for a number of months following initial data submission. The data analysis and data query processes utilized in the construction of this report remain unchanged from the approaches taken in the SFY 2012 Annual Report. Minor changes have occurred in the terminology used in a number of the exhibits in this year’s report, these changes resulting from the new contract of services that DES issued in the third quarter of SFY 2013. 2.1.1 Terminology Changes. Substance abuse education is now known as substance abuse awareness. Supportive services are now broken into two categories: auxiliary and concrete services. Aftercare is now defined as recovery maintenance. 2.1.2 Formatting Changes. In order to provide clarity to this report, formatting changes were implemented to depict comparisons and also to be consistent with program changes. Beginning with this year, Figure 2 has been modified to provide comparisons between total referrals and unique individuals served for SFY 2009 through SFY 2013. New to this year, Exhibit 8 now depicts the new structure and terminology of the DES service taxonomy for the AFF program, revised with the new service contract. Center for Applied Behavioral Health Policy Arizona State University | 10 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 SECTION 3 AFF INDIVIDUALS AND SERVICES RECEIVED Sections 3.1 and 3.2 present data for all 12 months of SFY 2013; all subsequent sections beginning with section 3.3 are restricted to data for the first three quarters (9 months) of SFY 2013. 3.1 AFF Total Referrals & Unique Individuals As depicted in Exhibit 2, there were 6,475 total referrals to the AFF program during all four quarters of SFY 2013, averaging approximately 1,619 total referrals per quarter. During this period of time, 5,541 unique individuals were referred to the program, with a small proportion of individuals referred more than once. The number of total referrals to the AFF program decreased by 6.5% for SFY 2013 as compared to SFY 2012, while the number of unique individuals referred to the AFF program decreased by 10.4%. The service provider contract transition that occurred during the third quarter of the state fiscal year (Jan-Mar) should be taken into consideration as one potential contributing factor to this decrease. The greatest year-over-year decrease in referrals was observed during this quarter (12.3%). Exhibit 2 AFF Total Referrals & Unique Individuals SFY 2013 July-Sep (Q1) Oct-Dec 9 (Q2) Jan-March (Q3) Apr-June (Q4) Total Referrals Unique Individuals 2012 (Reported4) 1,742 1,557 1,884 1,685 6,868 6,154 2012 (Updated5) 1,744 1,566 1,899 1,716 6,925 6,180 2013 # 1,599 1,556 1,664 1,656 6,475 5,541 Change # (145) (10) (235) (60) (450) (639) Change6 % -8.3 -0.6 -12.3 -3.5 -6.5 -10.4 4 These numbers were reported in the SFY 2012 AFF Annual Evaluation. These numbers represent data that have been updated since the completion of the AFF Annual Evaluation SFY 2012, which accounts for any lag in data entry from AFF providers and incorporates the new approach (See Section 2.1.1) 6 Calculated as (SFY 2012 updated-SFY2013/SFY 2012 updated). 5 Center for Applied Behavioral Health Policy Arizona State University | 11 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 Figure 2 displays the number of total referrals and unique individuals referred by quarter from SFY 2009 through SFY 2013. Decreases in 2009 and increases of referrals in 2012 can be attributed and contextualized to fiscal and inter-agency shifts (i.e., adjusting to budget cuts as a result of the 2009 financial crisis). The trend of unique individual referrals remained parallel to that of total referrals up until the beginning of the SFY 2012, when the trend began to grow apart. Third quarter in 2013 depicts the greatest dip in unique individuals compared to total referrals; this change may be attributed to the contractual transition that occurred during this quarter. 2000 1800 1600 1400 1200 1000 800 600 Total Referrals 400 Unique Individuals 200 Figure 2 AFF Total Referrals and Unique Individuals, by Quarter, SFY 2009 – SFY 2013 Center for Applied Behavioral Health Policy Arizona State University | 12 2013-QQ4 2013-QQ3 2013-QQ2 2013-QQ1 2012-QQ4 2012-QQ3 2012-QQ2 2012-QQ1 2011-QQ4 2011-QQ3 2011-QQ2 2011-QQ1 2010-QQ4 2010-QQ3 2010-QQ2 2010-QQ1 2009-QQ4 2009-QQ3 2009-QQ2 2009-QQ1 0 ARIZONA FAMILIES F.I.R.S.T. 3.2 Annual Evaluation Report SFY 2013 Disposition of Total Referrals to the Program Exhibit 3 provides a comparison of client outreach and engagement patterns across all four quarters of SFY 2013 and all four quarters of SFY 2012. Disposition of referrals is categorized according to individuals who: accepted services through consent and Release of Information or ROI (i.e., a signed statement allowing the provider agency to share information with specific individuals or groups); were closed prior to ROI being signed or services refused; or had referrals still in process at the end of the SFY 2013. Compared to SFY 2012, the proportion of referrals that were reported to accept services decreased by 15.6% during the same period of time in SFY 2013. Similarly, the proportion of referrals that were closed without a signed ROI also decreased in SFY 2013, relative to 2012, but by a much smaller percentage (2.0%). Referrals still in process at the end of SFY 2013 had increased by 17.6% relative to the same period of time in SFY 2012. This may be attributed to the transition period as new providers established program services in their respective communities. Exhibit 3 Disposition of Total Referrals to the Program SFY 2013 SFY 20127 SFY 2013 (All four quarter data) (All four quarter data) Total # of Referrals Total Accepting Services (Signed ROI) Total Closed Referrals (w/out Signed ROI) Total Referrals Still in Process8 6,925 n 4,809 1,284 832 6,475 % 69.4 18.5 12.0 n 3,487 1,073 1,915 % 53.8 16.6 29.6 All subsequent analyses to be reported are restricted to the first three quarters of SFY 2013 (July 1, 2012 – March 30, 2013). 7 These numbers represent updated SFY 2012 data according to current analysis. Still in process refers to situations such as going through the process of outreach attempts, request of CPS to delay outreach, etc. 8 Center for Applied Behavioral Health Policy Arizona State University | 13 ARIZONA FAMILIES F.I.R.S.T. 3.3 Annual Evaluation Report SFY 2013 AFF Referral to Selected Events - Duration in Business Days As evidenced in Exhibit 4, the average duration from referral to first outreach attempt, during the first three quarters of SFY 20139, was less than one day (mean 0.7), suggesting this event occurred during the same business day on which the referral was received. The average number of days from referral to acceptance of AFF services was 15 days10. One-fourth (25.6%) of all referrals to the AFF program were subsequently referred to a RBHA for Title XIX and/or Substance Abuse and Prevention Treatment (SAPT) eligibility determination, with these referrals occurring, on average, 11 days after AFF referral. For individuals that received AFF services, (57.8% of all referrals), the first AFF service occurred on average, 15 days following AFF referral. The average number of days from referral to closure for those individuals that did not accept or receive services was 44 days, demonstrating an increase from last year (35 days). Exhibit 4 AFF Referral to Selected Events - Duration in Business Days (Number of Total Referrals n = 4,816) SFY 2013: Q1-Q3 First Outreach Accepting AFF Sent to First AFF Attempt Services11 RBHA Service12 n13 4,416 2,787 1,234 2,773 % 91.7% 57.9% 25.6% 57.6% 0.7 14.8 11.0 15.0 Mean # Days 1.0 12.0 10.2 13.7 Standard Deviation 0.0 0.0 0.0 0.0 Minimum # Days 11.0 54.0 50.0 62.0 Maximum # Days Pre-Service Closure 549 11.4% 43.5 13.9 2.0 78.0 9 Throughout this report, reference will be made to SFY 2013; these references will be to the first three quarters of the SFY, unless noted otherwise. 10 Client acceptance of AFF services has traditionally been reported using the contractor’s interpretation of when a client accepted services. To ensure consistency, starting in SFY 2011, and continuing in SFY 2013, acceptance of AFF services was to be reported using the date a client signed the Release of Information (ROI). 11 Accepting AFF Services refers to the event of first date of receiving service. 12 First service refers to the first event of receiving treatment service. 13 Exhibit 4, sample n, is excluding outliers. Center for Applied Behavioral Health Policy Arizona State University | 14 ARIZONA FAMILIES F.I.R.S.T. 3.4 Annual Evaluation Report SFY 2013 Total Individuals Assessed Forty-six percent (n = 2,18514) of all individuals referred to AFF were assessed during the first three quarters of SFY 2013. Assessments were conducted by a contracted DES provider and/or a RBHA contracted provider, depending on the referred individual’s eligibility status for RBHA services. As summarized in Exhibit 5, approximately 55% of the individuals were assessed by DES contracted providers only, while 33% were assessed by RBHA contracted providers only. Relative to SFY 2012, the proportion of referrals assessed by DES providers increased by 8% in SFY 2013. A relatively small proportion (12%) of referred individuals were assessed by both a DES-contracted and a RBHA-contracted provider. The factors accounting for these multiple assessments cannot be determined on the basis of the information available and could reflect a number of situations, including separate and distinct episodes of care, as well as separate mental health and substance abuse assessments. DES Only DES & RHBA RBHA Only Exhibit 5 Total Individuals Assessed SFY 2013: Q1-Q3 n 1,195 268 722 Totals 2,185 % 54.7 12.3 33.0 100.00 14 This figure includes individuals who had been referred to the AFF program in SFY 2012, but not assessed until SFY 2013, along with individuals who were referred and assessed during SFY 2013. Center for Applied Behavioral Health Policy Arizona State University | 15 ARIZONA FAMILIES F.I.R.S.T. 3.5 Annual Evaluation Report SFY 2013 Substance Used by AFF Individuals 30 Days Prior to Assessment Exhibit 6 provides a summary of primary substance use and all substances individuals reported using just prior to their initial assessment. Among the individuals assessed (2,185), 91% provided information about any substances used in the preceding 30 day period, while 83% also reported their primary substance of choice. Marijuana (56%) continues to be the most commonly reported substance, with 33% of the individuals identifying this drug as their primary substance of choice. Alcohol and methamphetamine were also commonly reported substances of use, accounting for 53.9% and 47.9%, respectively, of those reporting any substance use. Nearly one -third (32.6%) of those individuals assessed identified methamphetamine as their primary substance of choice, while 20% of the individuals identified alcohol as their primary substance of choice. Exhibit 6 Substance Used by AFF Individuals 30 Days Prior to Assessment (Total Assessed Individuals: 2,185) SFY 2013: Q1-Q321 Primary All Substance Substance Use Use Reports Reports n n % % Individual Reporting Use 1,997 91.4 1,663 83.3 Marijuana 1,128 56.5 543 32.7 Alcohol 1,076 53.9 333 20.0 Methamphetamine 957 47.9 542 32.6 Cocaine/crack 288 14.4 66 4.0 Other narcotics 207 10.4 83 5.0 Heroin/Opioids 147 7.4 66 4.0 Other drugs 75 3.8 11 0.7 Hallucinogens 50 2.5 5 0.3 Benzodiazepines 38 1.9 4 0.2 Other stimulants 26 1.3 8 0.5 Other sedatives 15 0.8 2 0.1 Inhalants 13 0.7 0 0.0 Center for Applied Behavioral Health Policy Arizona State University | 16 ARIZONA FAMILIES F.I.R.S.T. 3.6 Annual Evaluation Report SFY 2013 Total Unique Individuals Served and Funding Source A total of 4,531 unique individuals received AFF services during the first three quarters of SFY 2013. As depicted in Exhibit 7, 44.7% (n = 2,027) of the unique individuals served in SFY 2013 were continuing individuals who had begun AFF services in SFY 2012. In comparison, 36.6% of the individuals served in SFY 2012 were continuing clients carried over from SFY 2011. Slightly more than one-third (36.8%) of all individuals received services that were funded by both DES and RBHA sources, reflecting a 5% decrease in the proportion of individuals with shared service funding. In SFY 2013, 76.4% of individuals received AFF services that were funded by DES, either exclusively (39.6%), or, as previously noted, in concert with RBHA sources (36.8%). In comparison, 60.4% of individuals received AFF services that were funded in part by a RBHA, with 23.6% reported as receiving RBHA funded services exclusively. The factors contributing to the reported absence of any reported DES funding for this group of AFF clients are not clear with the information currently available to the evaluation team. Exhibit 7 Total Unique Individuals Served and Funding Source SFY 2013: Q1-Q3 n 4,531 2,504 2,027 % 100.0 55.3 44.7 # of Individuals by Service Funding Source DES Individuals Shared Individuals DBHS Individuals Total15 1,794 1,666 1,070 4,530 39.6 36.8 23.6 100.0 Individuals Funded by DES and DBHS16 DES Funded Individuals DBHS Funded Individuals 3,460 2,736 76.4 60.4 New and Continuing AFF Individuals New Individuals Continuing Individuals 15 The difference between the totals for New and Continuing AFF Individuals compared to the total sum under # of Individuals by Service Funding Source relates to a client who was not identified due to incorrect information such as ss#. 16 Percentages include shared individuals who received funding from either source. Center for Applied Behavioral Health Policy Arizona State University | 17 ARIZONA FAMILIES F.I.R.S.T. 3.7 Annual Evaluation Report SFY 2013 Individuals Receiving DES & RHBA Funded Services Exhibit 8 summarizes the number of individuals who received AFF services funded by DES (either exclusively or in tandem with RBHA funds) during the first three quarters of SFY 2013. As evidenced in Exhibit 8, case management was provided to nearly two-thirds of all individuals who received DES funded services, while approximately one-third received group counseling (37.1%) and/or individual counseling (32.9%) that was funded by DES. Exhibit 8 Individuals Receiving DES-Funded Services17 SFY 2013: Q1-Q3 Unique Individuals DES-Funded Service n = 3,460 n % Substance Abuse Treatment Services Counseling Family 21 0.6% Group 1,282 37.0% Individual 1,140 32.9% Mental Health Services Medication 1 0.03% Medication Monitoring 27 0.8% Psychiatric Evaluation 35 1.0% Auxiliary Services Case Management 2,221 64.2% Drug Testing 2,243 64.8% Re-engagement 37 1.1% Living Skills Training 2 0.1% Parenting Skills 2 0.1% Concrete Supportive Services Clothing assistance 10 0.3% Food assistance 22 0.6% Housing/Rent 28 0.8% Transportation 286 8.3% Utilities Assistance 3 0.1% Other 276 8.0 % 17 These data do not capture RHBA funded services or other services clients may be accessing. Center for Applied Behavioral Health Policy Arizona State University | 18 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 Exhibit 9 provides summary information on the number and percentage of individuals with RBHA funded services provided during the first three quarters of SFY 2013. As these data reveal, support services, which include case management and treatment, such as individual and group counseling, were the more commonly reported services provided by RBHA funded providers. Exhibit 9 Individuals Receiving RBHA-Funded Services18 SFY 2013: Q1-Q3 Unique individuals RBHA-Funded Services n = 2,736 Service Domain n % Treatment Services 1219 44.55 Rehabilitation Services 304 11.11 Medical Services 352 12.87 Support Services 1302 47.59 Crisis Intervention Services 155 5.67 Inpatient Services 27 0.99 Residential Services 73 2.67 Behavioral Health Day Programs 10 0.37 18 These data do not capture other services funded by DES or other systems to which clients may have access.. Center for Applied Behavioral Health Policy Arizona State University | 19 ARIZONA FAMILIES F.I.R.S.T. 3.8 Annual Evaluation Report SFY 2013 Level of Care Exhibit 10 summarizes the levels of care that AFF individuals were assigned to by their AFF provider during their AFF service period. These data reflect only those AFF individuals who were assigned to one or more level of care. Of the 4,531 unique individuals served during the first three quarters of SFY 2013, nearly one-half (49.6%) of individuals were assigned to an outpatient level of care at some point in their AFF services; 22% were also assigned to intensive outpatient care at some point. Substance abuse awareness (12.9%) and recovery maintenance (5.5%) were levels of care assigned to a minority of individuals receiving AFF services. Nearly one-third (33.2%) of individuals receiving AFF services had no level of care assignment for the entire duration of their program participation. Exhibit 10 Level of Care SFY 2013: Q1-Q3 Unique Individuals n = 4,53119 Level of Care Substance Abuse Awareness Outpatient Intensive Outpatient Residential - Adult Residential - Child Recovery Maintenance 583 2,246 1,012 28 247 12.9 49.6 22.3 0.6 0.0 5.5 No Level of Care Identified 1,413 31.2 19 Individuals can be assigned to multiple levels of care throughout their AFF program experience; therefore the totals are greater than 100%. Individuals assigned to the ‘No Level of Care Identified’ had no record of LOC assignment at any time during their AFF program experience. Center for Applied Behavioral Health Policy Arizona State University | 20 ARIZONA FAMILIES F.I.R.S.T. 3.9 Annual Evaluation Report SFY 2013 Patterns of AFF Closure and Length of Service During the first three quarters of SFY 2013, 2,802 individual cases were closed by AFF providers (see Exhibit 11), representing a significant decrease compared to SFY 2012 (3,258). The closure data reflects individuals who exited the program at any point after a referral had been received. Among those individuals for whom AFF cases were closed, 25.2% were reported as having successfully completed their AFF treatment program. In contrast, 47.7% of the reported individual cases were closed as discontinued, reflecting a termination of services prior to a service completion. Relative to SFY 2012, the rate of successful treatment completion decreased slightly (1%), while the rate of treatment discontinuance increased significantly (12.6%). This increase may be reflective of new contract case transitions that occurred during the third quarter of the reporting period. Exhibit 11 Patterns of AFF Closure and Length of Service SFY 2013: Q1-Q3 n Closure Reason Completed Discontinued Refused services Unable to locate No SA Problem Incarcerated Moved out of Area Death Total Center for Applied Behavioral Health Policy 705 1,338 48 443 167 58 38 5 2,802 % 25.2 47.7 1.7 15.8 5.9 2.1 1.4 0.2 100.00 Median 171 112 1 18 4 72 77 84 91 Mean 182.3 134.8 16.8 46.8 26.5 114.3 107.5 198.4 123.7 Standard Deviation 124.5 104.1 38.2 74.3 52.3 121.3 93.7 177.3 114.9 Arizona State University | 21 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 SECTION 4 AFF PROGRAM OUTCOMES This section highlights the outcomes achieved by individuals who have participated in the AFF program. Outcomes are assessed in the following legislatively-specified domains: child safety, family stability and permanency, recovery from alcohol and drug abuse, and self-sufficiency as reflected by employment. 4.1 Pre-AFF and Post-AFF Referral Report Findings Among the 4,531 unique individuals that received AFF services, 3,943 (86.1%) had at least one allegation of child maltreatment prior to their referral to the AFF program. Of these pre-referral maltreatment allegations, 57.8 % were substantiated and 16.6 % were unsubstantiated. Of the individuals with a maltreatment report at the time of their referral to the AFF program, those with substantiated findings showed a recurrence rate of approximately 12%, while those with unsubstantiated findings at AFF referral demonstrated a slightly higher rate of recurrence of 18%. Overall, regardless of maltreatment allegation status at the time of AFF referral, 89% of all AFF individuals had no subsequent filing of an allegation of child maltreatment, consistent with previously reported recurrence patterns. Exhibit 12 Pre-AFF and Post-AFF Referral Report Findings SFY 2013: Q1-Q3 Pre-Referral Post Referral Allegation Finding Allegation Substantiated Proposed Unsubstantiated Other Finding n % n % n % n % n % Substantiated No Report n % 2,650 57.8 82 3.1 60 2.3 115 4.3 63 2.4 2,330 87.9 Proposed 484 10.6 0 0.0 11 2.3 6 1.2 4 0.8 463 95.7 Unsubstantiated 761 16.6 28 3.7 12 1.6 72 9.5 18 2.4 631 82.9 48 1.1 0 0.0 0 0.0 0 0.0 3 6.3 45 93.8 639 13.9 12 1.9 4 0.6 13 2.0 6 0.9 604 94.5 20 100.0 122 2.7 87 1.9 206 4.5 94 2.0 4,073 88.9 Other No Report Total 20 4,582 Fifty-one duplicate records were included in this value, attributable to unresolved data transition issues. Center for Applied Behavioral Health Policy Arizona State University | 22 ARIZONA FAMILIES F.I.R.S.T. 4.2 Annual Evaluation Report SFY 2013 Permanency Achieved by Children of Parents in AFF As depicted in Exhibit 13, a total of 4,705 children associated with AFF individuals in the first three quarters of SFY 2013 were in CPS out of home placement at some point during the reporting period. This accounts for a 15.4% increase as compared to SFY 2012 in which 3,982 children were previously reported. Out of the total children under CPS care the majority (71.8%) continue to be under CPS care while 26.6% have been reported to have achieved permanency. The rate of permanency (26.6%) has slightly increased this year compared to last year, where the reported rate was 21.7%. Among children who achieved permanency, 92.1% were reunified with a family and/or a legal guardian. The number of days in out of home placement ranged from a median of 158 days for reunifications to 460 days for adoption. Exhibit 13 Permanency Achieved by Children of Parents in AFF SFY 2013: Q1-Q3 n % Total Children 4,705 100 21 Still in Care 3,377 71.8 22 Other 77 1.6 23 Achieved Permanency 1,251 26.6 Days in Out of Home Care Among Children Achieving Permanency Achieved Permanency Reunification Guardianship Adoption n 1,251 1,152 68 31 % 100 92.1 5.4 2.5 Median Days 158 263 460 Mean Days 174.4 289.4 452.8 21 Removals 07/01/2011 - 03/31/2013, No placement as of 03/31/2013 Removals 07/01/2011 - 03/31/2013, Discharge by 03/31/2013 23 Removals 07/01/2011 - 03/31/2013, Permanency by 03/31/2013 22 Center for Applied Behavioral Health Policy Arizona State University | 23 ARIZONA FAMILIES F.I.R.S.T. 4.3 Annual Evaluation Report SFY 2013 AFF Drug Test Data Reported by DES/AFF Providers According to AFF program requirements, DES-contracted providers are required to verify that AFF individuals are tested for drug use at least twice per month during each month of AFF program participation once the individual has signed the Release of Information (ROI). Among the 4,531 unique individuals who received AFF services in SFY 2013, about 57.3% had been referred for drug testing at least once during the period of their AFF program participation. On average, those AFF individuals who were referred for drug testing were tested 1.7 times per month, which is close to the AFF program requirements for testing. For individuals with known drug test results (16,677), 80% were negative, indicating no drug use during the time frame of AFF program participation. Exhibit 14 AFF Drug Test Data Reported by DES/AFF Providers SFY 2013: Q1-Q3 Unique AFF Individuals with Services n Total Unique Individuals 4,531 With Referrals for Drug Tests 2,596 Without Referrals for Drug Tests 1,935 779 With at least two (2) referrals per month Mean 1.7 Drug referrals per month Drug Test Referral Outcomes Negative Positive Pending Refused / Altered Cancelled Total Center for Applied Behavioral Health Policy n % 100.0 57.3 42.7 30.0 Standard Deviation 1.5 % 13,371 2,040 951 295 20 16,677 80.2 12.2 5.7 1.8 0.1 100.0 Arizona State University | 24 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 SECTION 5 KEY FINDINGS AND PROGRAM IMPLICATIONS This report summarizes the key processes and outcomes of the Arizona Families F.I.R.S.T. program, now in its 13th year of operation. The continued commitment of the legislature to examine the processes and outcomes of this innovative program has afforded the opportunity to systematically evaluate the effectiveness and impact of a program unique in its scope and focus. As part of this annual evaluation, independently conducted by Arizona State University, the performance of the AFF program, in relation to each of the goals articulated by the legislature, was addressed by utilizing information from a variety of sources, including administrative data and service utilization records. 5.1.1 Increases in Timeliness, Availability, and Accessibility of Services. In SFY 2013, 6,475 total referrals, representing 5,541 unique individuals, were made to the AFF program statewide. This represents a 10% reduction in referrals relative to SFY 2012. For the individuals referred and their families, the AFF program continues to provide services in a manner consistent with the program design. During SFY 2013, AFF participants received outreach, assessment, engagement, and substance abuse treatment services in a timely manner. AFF providers made efforts to contact referred individuals within one business day. AFF services were accepted by participants, on average, within 18 days of referral to AFF. Those individuals who were engaged in AFF services typically found themselves receiving services from their local AFF provider and/or a RBHA contracted treatment provider in their community, depending upon their program. During the first three quarters of SFY 2013, nearly 40% of AFF individuals received AFF services with funding solely provided by DES, compared to about 24% of individuals who received services funded by RBHA. Thirty-six percent of AFF individuals received services through a combination of DES and RBHA sources, with each source providing complimentary but non-duplicative services. 5.1. 2 Child Safety and Reduction of Child Abuse and Neglect. Most individuals (86.1%) served by the AFF program had at least one allegation of child maltreatment prior to enrolling in the program. Nearly 89% of all AFF individuals had no subsequent maltreatment report filed during this report period. 5.1.3 Children of Parents in AFF. Among children reported to be under CPS care during the reporting period of July 1, 2012 to March 31, 2013, about 72% were reported to still be in care, while the permanency rate was reported at 26.6%. Of those who achieved permanency, the majority (92.1%) did so through reunification. Center for Applied Behavioral Health Policy Arizona State University | 25 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 5.1.4 Recovery from Alcohol and Drug Problems. Consistent with previous years, the overwhelming majority of AFF individuals assessed in SFY 2013 self-reported use of marijuana, alcohol, and methamphetamine. The results of drug screens conducted with AFF individuals to detect continued drug use indicated that almost 60% of AFF individuals were drug free throughout their AFF participation. Individuals receiving AFF services were drug tested, on average, 1.7 times per month during AFF program participation. The results of this evaluation of the Arizona Families FIRST program provide continuing evidence of the efficiency with which this program engages referred clients into treatment, the consistency and quality of the program components that individuals receive, and the impact that those services have upon the reduction of parental substance abuse and child neglect, family reunification, and permanency planning. During this past state fiscal year, DES completed a nearly two-year process of AFF program redesign that culminated with the awarding of new service provider contracts during the third quarter of the current reporting period. The result of this new program design places a stronger emphasis upon the utilization of evidence-based substance abuse practices, an emphasis on family-focused and recovery-oriented supportive services, and a focus on quality management and program monitoring. The new AFF program design caused modest disruptions in service availability and provision. This disruption was most pronounced in the third quarter of the current reporting period, with lingering impacts that began prior to that and extended throughout the fourth quarter of the reporting period. These disruptions impacted not only service availability and provision, but also program evaluation and monitoring systems. Evidenced through this report are multiple minor reductions in AFF service provision, with overall referrals off by 10% relative to SFY 2012, and numerous service indicators displaying minor to modest reductions in performance efficiency. It is not clear whether the alterations in program performance are reflective of disruptions in data monitoring process, as clients and client data systems were bridged between two providers, or are reflective of true downturns in program performance. It is anticipated that these program performance indicators will display rebound in SFY 2014 as a result of the program requirements contained in the new AFF service contracts. One of the significant outcomes of the new AFF service contracts is the reduction in the number of contract providers from eight to three, as TERROS was awarded service contracts for four counties, while SEABHS’s catchment area was expanded from three to four counties. This reduction in the number of contracted providers affords the State greater efficiency in program and contract monitoring, increases the consistency of program delivery, and reduces program monitoring data gathering complexity. The target population of the AFF program, caregivers who abuse substances and families engaged with the child welfare system, represent a critical atrisk and growing segment of our state’s population. Center for Applied Behavioral Health Policy Arizona State University | 26 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 The intersection of child maltreatment and substance abuse is ubiquitous, occurring across cultural, socioeconomic, and geographical boundaries. The 2012 Arizona Substance Abuse Prevention and Treatment Services Capacity Report measured self-reported alcohol, illegal/illicit drug use, and prescription drug use during the 30 days preceding survey administration. The data were gathered from the general population of AZ (i.e., not a clinical sample). The report indicated that there were no consistent patterns of substance use across geographical service areas, suggesting a need for services tailored to the needs of the population in a given area24. In the aforementioned report, the proportion of individuals reporting alcohol use in the past 30 days ranged from 50-55% and between 10-14% for illicit drugs. In comparison, among arrestees booked into jail in Maricopa County, 69% reported alcohol use in the preceding 30 days, with nearly 50% reporting methamphetamine, marijuana, or other illicit substance use. Among this sample of arrestees, 43% also reported having children living in the home, and conceivably at risk for child neglect and/or abuse25. In contrast, 91% of all individuals in the AFF program had used substances in the 30 days prior to their referral; during their period of program participation these same individuals were tested to be drug free 80% of the time. Likewise, the results of this year’s evaluation provide continuing evidence of the AFF program’s impact upon the recurrence of child maltreatment. Nearly all (91%) of those families served in the AFF program had no recurrence of maltreatment allegation during their AFF program participation during the reporting period. Permanency rates in 2013, compared to SFY 2012, suggest an increase in goals achieved including increase in reunification, guardianship, and adoption rates. Nonetheless, the number of children under CPS care also increased. Consistent with other reports, permanency rates continue to be unstable and show slight changes since 2009, at which point trends showed the lowest rates of children entering the CPS system26. The Child Welfare Reporting Requirements, Semi-annual Report indicated that, of 20,253 reports meeting statutory criteria for maltreatment, 53.9 percent were closed, 6.3 percent were sustained, and 5.2 percent were in the process of appeal as they were recommended for sustention. While data may not be compared across information sources in these reports, we may assume that families associated with AFF clients make up a substantial portion of sustained cases (6,925 total referrals in 2012)27. 24 Wolfersteig, W.L., Fernandez, K.M. , & Hoffman, K. (2012, September). 2012 Arizona Substance Abuse Prevention and Treatment Services Capacity Report. Phoenix, AZ: Southwest Interdisciplinary Research Center, Arizona State University. 25 Choate, David E. (2012). Arizona Arrestee Reporting Information Network: 2012 Maricopa County Manager’s Office Report on Substance Use and Public Health Concerns among Arrestees. Phoenix, AZ: Center for Violence Prevention & Community Safety, Arizona State University. 26 Department of Economic Security (DES), Division of Children, Youth and Families (DCYF). (2013, August 8). Child Welfare Reporting Requirements, Semi-annual Report for the Period of October 1, 2012 Through March 31, 2013. Retrieved from https://www.azdes.gov/InternetFiles/Reports/pdf/semi_annual_child_welfare_report_ oct_2012_mar_2013.pdf 27 Imbd. Center for Applied Behavioral Health Policy Arizona State University | 27 ARIZONA FAMILIES F.I.R.S.T. Annual Evaluation Report SFY 2013 According to the Child Welfare Reporting Requirements, Semi-annual Report, children entering the CPS system began to increase gradually during the reporting period of October 2012-March 2013 compared to previous years. According to AFF evaluation data however, numbers of referrals have significantly decreased from 2012 through the current year. Identifying reasons underlying fewer referrals with increasing numbers of involved children will become more critical moving forward. In contrast, national studies show that the rate of maltreatment occurs at a total incidence of 17.1 per 1,000 children in the general population. This rate is equivalent to 1.71 per 100 children, further suggesting that, in the United States, one child out of every fifty-eight children has experienced maltreatment28. As such, it is clear that the AFF program continues to target a high risk population, consistent with the legislative intent of the program. These data also provide continuing evidence of the program’s impact in reducing parental substance abuse child maltreatment, consistent with the legislative goals of the program. What remains unclear at this time are the long-term impacts of the AFF program upon the program participants and their families with regard to family reunification, parental sobriety, and parenting. DES-initiated changes in the AFF program design, in consultation with the ASU evaluation team, will allow for more systematic and rigorous assessment of the long term impacts of this program in the future. 28 Sedlak, A.J., Mettenburg, J., Basena, M., Petta, I., McPherson, K., Greene, A., & Li, S. (2010). Fourth National Incidence Study of Child Abuse and Neglect (NIS–4): Report to Congress. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families. Center for Applied Behavioral Health Policy Arizona State University | 28