Arizona Department of Child Safety Douglas A. Ducey Governor Gregory McKay Director November 30, 2016 The Honorable Andy Biggs President of the Senate Arizona State Senate 1700 West Washington Phoenix Arizona 85007 Re: Arizona Families F.I.R.S.T. 2016 Annual Evaluation Report Dear President Biggs: Pursuant to A.R.S. § 8-884, the Arizona Department of Child Safety (DCS) is pleased to provide the Arizona Families F.I.R.S.T. (AFF) program evaluation report for the state fiscal year 2016 (SFY 2016). This annual evaluation report prepared by the independent evaluator, Wellington Group, provides analysis of program implementation, service utilization and program outcome data. If you have any questions, please contact my office at (602) 255-2500. Sincerely, Gregory McKay Director P.O. Box 6030  Site Code C010-23  Phoenix, AZ 85005-6030 Telephone (602) 255-2500 The Honorable Andy Biggs Arizona Families F.I.R.S.T. 2016 Annual Evaluation Report Page 2 Enclosures cc: Speaker David M. Gowan, Speaker, Arizona State House of Representatives Representative John Allen, Chairperson, Joint Legislative Audit Committee Representative Regina Cobb, Member, Joint Legislative Audit Committee Representative Debbie McCune Davis, Member, Joint Legislative Audit Committee Representative Rebecca Rios, Member, Joint Legislative Audit Committee Representative Kelly Townsend, Member, Joint Legislative Audit Committee Senator Judy Burges, Member, Joint Legislative Audit Committee Senator Nancy Barto, Member, Joint Legislative Audit Committee Senator Lupe Contreras, Member, Joint Legislative Audit Committee Senator David Farnsworth, Member, Joint Legislative Audit Committee Senator Lynne Pancrazi, Member, Joint Legislative Audit Committee Holly Henley, Director, Arizona State Library, Archives and Public Records Arizona Families F.I.R.S.T. Annual Evaluation Report Summary for SFY 2016 Arizona Families F.I.R.S.T. Annual Evaluation Report: Summary for SFY 2016 AZ DES Contract Number: ADSPO13-038321:24 September 2016 Reporting Period: SFY 2016: July 1, 2015 – June 30, 2016 Submitted to: Arizona Department of Child Safety Arizona Families F.I.R.S.T. 3003 N. Central Avenue Phoenix, AZ 85012 Submitted by: Jane Dowling, PhD Samantha Martin, PhD Stephanie Heyde, BA Wellington Consulting Group, Ltd. ii Executive Summary Arizona Families F.I.R.S.T. (AFF) was established as a community substance use disorder prevention and treatment program by Senate Bill 1280, which passed in the 2000 legislative session. AFF addresses adverse conditions related to alcohol and drug abuse among childwelfare-involved families in which allegations of child maltreatment were associated with parental substance abuse. The AFF program provides a variety of treatment and supportive services designed to reduce or eliminate abuse of and dependence on alcohol and other drugs in family systems. The AFF program emphasizes face-to-face outreach and rapid engagement at the time of program referral, assessments, supportive services to remove barriers (e.g., employment, transportation, case management, and housing services), individual and group treatment, counseling individual and group treatment, and recovery maintenance to support ongoing sobriety and recovery. Key Findings The AFF program data is presented in two contexts: 1) Unique individuals, and 2) Referrals. A unique individual is an individual who is referred to the AFF program for substance use disorder treatment services. A unique individual can have multiple referrals to the AFF program. Therefore, the amount of data presented on referrals will always be greater than the unique individual data. Throughout the body of the AFF Annual Evaluation Report, the data is presented in terms of clients and referrals. Each referral initiates a new flow of service to the client. Therefore, the referral data documents the overall services AFF provided during SFY 2016 while the client data details the number of unique individuals served during the same time period. iii In this Executive Summary, all data is presented in the context of referrals. Presenting a single data source (referrals) simplifies the data presentation in the Executive Summary by focusing on the services provided during SFY 2016. AFF Client Demographic Characteristics During State Fiscal Year (SFY) 2016, a total of 12,261 unique individuals received a referral to the Arizona Families FIRST (AFF) program. Out of the 12,261 unique individuals, there were 8,131 individuals referred to the AFF program between July 1, 2015 and June 30, 2016. An additional 4,130 individuals were referred to the AFF program prior to July 1, 2015 and continued to receive services during SFY 2016. More than one half (55.6%) of the individuals in SFY 2016 were between the ages of 25 and 35 year old. Three out of five AFF participants (62.6%) were female. Referrals, Outreach, Acceptance of Services, and Assessments In SFY 2016, there were a total of 9,611 new referrals to the AFF program (i.e., referrals received between July 1, 2015 and June 30, 2016) and 5,030 continuing referrals that opened prior to July 1, 2015. The 12,261 unique individuals served during SFY 2016 received a total of 14,641 referrals which includes the continuing referrals from SFY 2015. As a single individual can receive multiple referrals to the AFF program, the data summarized here focuses on the 14,641 referrals. Of the 14,641 referrals, AFF providers completed some form of outreach for 14,051 of the referrals (96%), with 13,345 referrals (95%) having an outreach attempt within one day or less from when the AFF provider received the referral. In accordance with AFF model fidelity, 67.2% of referrals received at least three (3) outreach attempts within five (5) business days. Of the 14,641 referrals, a total of 9,079 referrals (62%) resulted in a signed Release of Information (ROI) form, which indicates client acceptance of AFF services. While less than two-thirds of referrals resulted in a signed ROI form and thus client acceptance of AFF services, the AFF providers showed an increase from SFY 2015 when only 50.9% of referrals resulted in a signed ROI. After acceptance of AFF services, a substance abuse assessment is conducted to assess the level and impact of a clients’ use and abuse of alcohol and drugs. Out of the 9,079 referrals with a signed ROI, the substance abuse assessment was completed for 8,747 referrals (96.3%). The 8,747 referrals with a completed assessment resulted in 7,939 referrals (90.8%) where the individual was identified as needing substance abuse treatment services. Overall, 7,939 of the 14,641 referrals or 54.2% had a completed assessment that resulted in an identified need for substance abuse treatment services. iv Alcohol and Substance Use Among AFF Clients In SFY 2016, the top three substances AFF clients reported using in the past 30 days during their assessment were: 1) Marijuana/Hashish (30.7%), 2) Methamphetamine/Speed (27.9%), and 3) Alcohol (17.9%). The high percentage of AFF clients who reported using Methamphetamine/Speed during the past 30 days should be noted. In the Annual Report on Substance Abuse Treatment Programs Fiscal Year 2015 released by the Arizona Department of Health Services/Division of Behavioral Health Services, 17.9% of individuals receiving substance abuse treatment services cited methamphetamine as their primary substance, which is 10% less than the percentage of AFF clients. Data included for the first time in the AFF Annual Report is data on AFF client compliance with drug testing policies. Clients enrolled in the AFF program must complete routine drug screenings on a schedule determined by how long they have been enrolled in the program. Clients enrolled between 0 and 60 days are expected to complete at least two drug tests per week. When the client is enrolled between 61 and 120 days, s/he should complete at least two drug tests each month. Finally, when a client has been enrolled greater than 120 days, s/he should complete at least one drug test per month. The data currently shows that approximately 25% of clients in each of the three drug testing schedules were compliant with AFF drug testing standards. Services used by AFF Clients During SFY 2016, 70.6% of AFF clients who received substance abuse treatment began in Outpatient services, and 21.8% of AFF clients started their treatment in Intensive Outpatient services. Among clients who received substance abuse treatment and closed in SFY 2016, 15.6% of clients exited the program after being in Recovery Maintenance/Aftercare services and successfully completing the program. A total of 53.9% clients left the program while they were in Outpatient services, 22.1% left while they were in Intensive Outpatient services, and 8.2% of clients did not have a level of care documented at closure. In addition to substance abuse treatment services, AFF providers provide clients with other services including: 1) Parenting skills training, 2) Job readiness/employment training, 3) Mental health services, 4) Medical services, 5) Domestic violence services, 6) Crisis services, 7) Basic life needs, and 8) Other services. During SFY 2016, 7,448 clients received a total of 117,460 services in these eight areas. This equates to nearly 16 services per client. Parenting services and mental health services accounted for more than 65% of the services received by AFF clients. Child Safety and the Reduction of Child Abuse and Neglect To prepare data on child safety and the reduction of child abuse and neglect, data from the Department of Child Safety’s CHILDS database is utilized to track allegation findings individuals received before and after being referred to the AFF program. Among individuals who successfully completed the AFF program, 56.8% had no reports to DCS after being referred to the AFF program. Among individuals who exited the AFF program before v program completion, 61.5% had no reports to DCS after being referred to the AFF program. It should be noted people who did not complete services may have done so because reasons that could limit new reports being received such as incarceration, moving out of state, or deceased. This would reduce their likelihood of a subsequent report. It is the intention of the Department and Wellington Group to continue to identify alternative methods to evaluate the effectiveness of the AFF services in relation to preventing repeated reports of child abuse or neglect. Permanency for Children Through Reunification Data related to permanency for children through reunification also comes from the Department of Child Safety’s CHILDS database. More than 50% of the children associated with a referral to the AFF program between April 30, 2011 through June 30, 2016 had achieved permanency by the end of SFY 2016. Out of the children who had achieved permanency, 47.9% of the children were reunified with their family, and 42.8% of the children were adopted. It should be noted that children associated with and entered care in CY 2016 would not reasonably have been expected to have achieved permanency. Table 21 illustrates that the average number of days in out of home care totaled 524.5 days which is more than 12 months. vi Table of Contents Executive Summary .............................................................................................................................. iii Key Findings ....................................................................................................................................................... iii AFF Client Demographic Characteristics ..................................................................................................... iv Referrals, Outreach, Acceptance of Services, and Assessments ............................................................. iv Alcohol and Substance Use Among AFF Clients ........................................................................................... v Services used by AFF Clients ............................................................................................................................... v Child Safety and the Reduction of Child Abuse and Neglect ..................................................................... v Permanency for Children Through Reunification ....................................................................................... vi Introduction .............................................................................................................................................. 1 Brief Description of the AFF Program and Client Flow ........................................................................... 3 Overview .................................................................................................................................................... 5 Evaluation Framework and Data Sources.................................................................................................... 5 New Evaluator Hired ............................................................................................................................................. 5 Challenges ................................................................................................................................................................. 6 AFF Program Funnel Diagram.......................................................................................................................... 7 AFF Demographics.................................................................................................................................. 9 AFF Referrals ........................................................................................................................................ 12 AFF Referral Sources ...................................................................................................................................... 14 AFF Outreach .................................................................................................................................................... 15 AFF Accepted Services .................................................................................................................................... 16 AFF Substance Abuse Assessment ............................................................................................................... 18 AFF Funding Source at Assessment............................................................................................................. 19 AFF Level of Care at Initial Assessment..................................................................................................... 20 AFF Level of Care at Closure ........................................................................................................................ 22 AFF Services .......................................................................................................................................... 24 AFF Closures ......................................................................................................................................... 27 Closure Reason Pre-AFF Services ................................................................................................................ 27 Closure Reason Post-AFF Services .............................................................................................................. 28 AFF Funding Source at Closure ................................................................................................................... 29 Employment Status at Closure ...................................................................................................................... 30 AFF Substance Use ............................................................................................................................... 32 AFF Drug Test ....................................................................................................................................... 34 AFF Drug Test Compliance ........................................................................................................................... 34 AFF Drug Test Referral Outcome ................................................................................................................ 35 AFF Referral Report Findings and Permanency Outcomes ...................................................... 37 AFF Referral Report Findings (Maltreatment) ........................................................................................ 37 AFF Permanency Outcomes .......................................................................................................................... 43 Recommendations and Next Steps .................................................................................................... 45 Conclusion .............................................................................................................................................. 47 vii List of Exhibits EXHIBIT 1. LIST OF DCS REGIONS, COUNTIES, DCS PROVIDERS, AND RBHAS, SFY 2016 ___________________ 2 EXHIBIT 2. MAP OF AFF PROVIDERS AND RBHA HEALTH AGENCY REGIONS 2016 _________________________3 EXHIBIT 3. AZ FAMILIES F.I.R.S.T. FLOW OF SERVICE __________________________________________________________4 List of Tables TABLE 1: AFF DEMOGRAPHICS AT INITIAL ASSESSMENT ___________________________________________________ 10 TABLE 2: AFF REFERRALS _______________________________________________________________________________________ 13 TABLE 3: AFF REFERRAL SOURCES_____________________________________________________________________________ 14 TABLE 4: AFF OUTREACH _______________________________________________________________________________________ 16 TABLE 5: AFF ACCEPTED SERVICES ____________________________________________________________________________ 17 TABLE 6: AFF SUBSTANCE ABUSE ASSESSMENT ______________________________________________________________ 18 TABLE 7: AFF FUNDING SOURCE AT ASSESSMENT ___________________________________________________________ 19 TABLE 8: AFF LEVEL OF CARE AT INITIAL ASSESSMENT ____________________________________________________ 21 TABLE 9: AFF LEVEL OF CARE AT CLOSURE ___________________________________________________________________ 23 TABLE 10: AFF SERVICES1 _______________________________________________________________________________________ 25 TABLE 11: CLOSURE REASON PRE-AFF SERVICES1‡ __________________________________________________________ 28 TABLE 12: CLOSURE REASON POST-AFF SERVICES1‡ _________________________________________________________ 29 TABLE 13: AFF FUNDING SOURCE AT CLOSURE ______________________________________________________________ 30 TABLE 14: EMPLOYMENT STATUS AT CLOSURE ______________________________________________________________ 31 TABLE 15: AFF SELF-REPORTED SUBSTANCE USE ___________________________________________________________ 33 TABLE 16: AFF DRUG TEST ______________________________________________________________________________________ 35 TABLE 17: AFF DRUG TEST REFERRAL OUTCOME ____________________________________________________________ 36 TABLE 18: AFF REFERRAL REPORT FINDINGS (COMPLETED AFF PROGRAM) _____________________________ 40 TABLE 19: AFF REFERRAL REPORT FINDINGS (DID NOT COMPLETE AFF PROGRAM) ____________________ 42 TABLE 20: AFF PERMANENCY STATUS ________________________________________________________________________ 43 TABLE 21: AFF PERMANENCY OUTCOMES ____________________________________________________________________ 44 List of Figures FIGURE 1: AFF SELF REPORTED SUBSTANCE USE ____________________________________________________________ 33 viii Introduction Arizona Families F.I.R.S.T. (AFF) was established as a community substance use disorder prevention and treatment program by Senate Bill 1280, which passed in the 2000 legislative session. AFF addresses adverse conditions related to alcohol and drug abuse among childwelfare-involved families in which allegations of child maltreatment were associated with parental substance abuse. The AFF program provides a variety of treatment and supportive services designed to reduce or eliminate abuse of, and dependence on, alcohol and other drugs in family systems. Interventions are provided through the Arizona Department of Child Safety (DCS) contracted community providers with services provided in outpatient and residential settings, and/or through the Regional Behavioral Health Authority (RBHA) network of providers under contract with the Arizona Department of Health Services, Division of Behavioral Health Services (ADHS/DBHS). This evaluation examines the implementation and outcomes of community substance use disorder treatment services delivered by providers contracted with DCS 1. The following exhibit provides a list of DCS regions, counties, DCS providers, and RBHAs (Exhibit 1). The map provided in Exhibit 2 shows the AFF provider regions and RBHA service areas. 1 DCS providers are contracted with the Arizona Department of Child Safety to deliver substance use disorder treatment services through the Arizona Families F.I.R.S.T. (AFF) program. Clients are referred to the AFF program through two sources: 1) the Arizona Department of Child Safety (DCS), and 2) the TANF/Jobs program operated by the Arizona Department of Economic Security. 1 Exhibit 1. List of DCS Regions, Counties, DCS Providers, and RBHAs, SFY 2016 DCS Region County Central Maricopa East Pinal Pima Southwest Southeast Northern Pima Maricopa West Yuma La Paz Gila Cochise Graham Greenlee Santa Cruz Coconino RBHA 2016 Mercy Maricopa Integrated Care (MMIC) DCS Provider 2016 Terros Central Cenpatico Cenpatico Mercy Maricopa Integrated Care (MMIC) Cenpatico Health Choice Integrated Care (HCIC) Terros Pima Terros Southwest Southern Arizona Behavioral Health Services (SEABHS) Arizona Partnership for Children (AzPaC) 2 Exhibit 2. Map of AFF Providers and RBHA Health Agency Regions 2016 Brief Description of the AFF Program and Client Flow AFF is a program that provides contracted family-centered, strengths-based, substance abuse treatment and recovery support services to parents or caregivers whose substance abuse is a significant barrier to maintaining or reunifying the family. The goal of the program is to reduce or eliminate abuse of and dependence on alcohol and other drugs, and to address other adverse conditions related to substance abuse. Interventions are provided through the Department of Child Safety’s contracted community providers in outpatient and residential settings or through the RBHA provider network. In addition to traditional services, AFF includes an emphasis on face-to- face outreach and engagement at the time of program referral and the beginning of treatment, concrete supportive services to remove barriers (e.g., employment, transportation, case management, and housing services), and an aftercare phase to manage relapse occurrences. Essential elements based on family and community needs, such as culturally responsive services, gender- specific treatment, services for children, and motivational enhancement strategies to assist the entire family in its recovery, are incorporated into the service delivery. 3 The Exhibit on the following page shows the flow of clients through various stages of the AFF program (Exhibit 3). 4 Overview The Arizona Families F.I.R.S.T. Annual Evaluation Report: Summary for SFY 2016 summarizes the data provided to Wellington Consulting Group, Ltd. in July 2016 by the providers. This annual report encompasses the data collected from Arizona Partnerships for Children (AzPAC), Southern Arizona Behavioral Health Services (SEABHS), and Terros from May 2011 through July 2016, specifically the referrals, services, and outcomes associated with Arizona Families F.I.R.S.T (AFF) for the State Fiscal Year 2016 (July 1, 2015 – June 30, 2016). Evaluation Framework and Data Sources This evaluation report responds to the legislatively mandated performance indicators of the AFF program. New Evaluator Hired In March 2016, Wellington Consulting Group, Ltd. (Wellington Group) was hired by DCS as the new evaluator of the AFF program. In May 2016, AFF providers began submitting data to Wellington Group through a new data portal, Arizona Families F.I.R.S.T. Data Collection Portal. The new portal, created by Wellington Group, follows the data specifications and criteria negotiated with, and approved by, the contracted service providers and DCS. The new portal allows providers to upload their in-house data directly into the portal in a secured format. It also provides DCS and the providers the ability to access web-based data look-ups on their specific individual and aggregate clients. Wellington Group provided trainings to the contracted service providers 5 and DCS to ensure the data was properly entered and that it aligned with AFF reporting requirements. The data provided herein are drawn from administrative data submitted to the evaluator directly via the Arizona Families F.I.R.S.T Data Collection Portal. The new data upload specifications were finalized in April 2016 and required all providers to upload data in eight data tables (Referral, Outreach, Client, Level of Care, Service, Drug Test, Past 30 Day Use, and Closure) using data file formats that would ensure cross-agency consistency and lead to better data integrity. These data uploads occur by the 15th of each month. Upon receipt of a data upload, Wellington Group evaluates the file structure and the data in each table to ensure that it meets the standards specified in the AFF Data Transmittal Specifications and the Data Definitions for AFF Data Tables. Wellington Group maintains communication with the providers and DCS to ensure the data is accurately interpreted and to correct errors identified in the data table formats. This annual report summarizes the data imported into the Arizona Families F.I.R.S.T. Data Collection Portal. As this is the first report prepared by Wellington Group, the ability to compare the SFY 2016 annual report for AFF to previous annual reports is limited because of differing data collection and analysis methods. This report captures a 12-month period (July 1, 2015 – June 30, 2016) that includes: 1) Data on clients who received referrals in the specified time period, and 2) Data on clients who received referrals prior to SFY 2016 and received services within the specified time period Challenges Two primary changes took place during the State Fiscal Year: 1) a new evaluator was hired and 2) a new data collection portal was developed. The new portal included new programming and reporting requirements for the evaluator, DCS and the providers. As with any change, there is a period of adaption and transition that takes place. Accordingly, DCS, the providers, and the evaluator worked together with ongoing communication to foster good working relationships among all parties. The evaluator continues to work with the providers to ensure the integrity of the data. As noted earlier, the transition to a new evaluator limits the ability to compare the SFY 2016 Annual report with previous annual reports. 6 AFF Program Funnel Diagram The diagram below provides a snapshot of the services provided in the SFY 2016 (Exhibit 3). The AFF Funnel Diagram tracks how clients referred to the AFF program move through and exit from services. The AFF Program Funnel summarizes the data for unique individuals served during SFY 2016. 7 Exhibit 3. AFF Program Funnel 8 AFF Demographics The demographic information reported in this section refers to individuals who: 1) received new referrals for AFF services reported in the SFY 2016 (July 1, 2015 – June 30, 2016), 2) received referrals for AFF services prior to SFY 2016 and continued to receive AFF services in SFY 2016, and 3) received referrals for AFF services prior to SFY 2016, continued receiving services in SFY 2016 and then closed in SFY 2016 (July 1, 2015 – June 30, 2016). A client who was referred to the AFF on July 1, 2015 would be included as a “new referral.” Another client who was referred to AFF on May 1, 2015 and continued to receive AFF services from during SFY 2016 would be included as a “continuing referral.” Finally, a client referred to the AFF program on June 1, 2015 and closed on August 30, 2015 would also be included as a “continuing” referral because this client received AFF services during SFY 2016. A total of 8,131 unique individuals were the subjects of the 9,611 new referrals for AFF services received in SFY 2016. In addition, 4,130 individuals represent the referrals opened prior to SFY 2016 and continued to receive AFF services or closed in the SFY 2016. Accordingly, 12,261 unique individuals were either newly referred and/or served in SFY 2016. More than one half (55.6%) of the individuals in SFY 2016 were between the ages of 25 and 35 year old. Three out of five AFF participants (62.6%) were female. For Marital Status, Race, and Educational Achievement, a large percentage of the data is “Unknown” (49.5%, 76.8%, and 42.9% respectively). The missing information reflects challenges providers face with collecting this data in their existing data collection systems. Several providers are updating their databases to provide more complete and accurate information to the AFF evaluators. Among the clients for whom the information was reported, the majority were single (65.8%) and white (81.0%). Among those individuals for whom information was reported, a little over two thirds (69.3%) had no higher than a high school diploma/GED level of education. Additionally, the vast majority of clients identified as non-Hispanic (93.8%). 9 Table 1: AFF Demographics at Initial Assessment Age # % <18 28 0.2% 18 - 24 2,259 18.4% 25 – 30 3,978 32.4% 31 – 35 2,843 23.2% 36 – 45 2,475 20.2% 46 – 55 588 4.8% > 55 90 0.7% Unknown 0 0% Total 12,261 100% Gender # % Male 4,584 37.4% Female 7,675 62.6% Other 0 0% Refused 0 0% Unknown 2 0% Total 12,261 100% Marital Status # % Married 1,069 8.7% Single, never married 4,078 33.3% Widowed 61 0.5% Domestic Partner/ Cohabitation 39 0.3% Divorced/Separated 853 7.0% Refused 96 0.8% Unknown 6,065 49.5% Total 12,261 100% Race (Not Mutually Exclusive) # % White 2,322 18.8% American Indian/Alaska Native 180 1.5% Asian 9 0.1% Black/African American 255 2.1% Native Hawaiian or Other Pacific 23 0.2% Islander Other 79 0.6% Unknown 9,475 76.8% Total 12,343 100% Hispanic/Latino Ethnicity # % Yes 765 6.2% No 11,496 93.8% Other 0 0% Refused 0 0% Unknown 0 0% Total 12,261 100% 10 Education Less than 1 year Grades 1 to 12 (No HS Diploma/GED) High School Graduate or GED Vocational/Technical School Some College, No Degree College – AA/BA Degree Graduate or Post Graduate Degree Refused Unknown Total Employment Employed Full-Time Employed Part-Time Unemployed Volunteer Vocational Rehabilitation Homemaker Student Retired Disabled Inmate of Institution Work Adjustment Training Transitional Employment Placement Refused Unknown Total # 5 % 0.0% 3,075 25.1% 1,768 353 1,399 383 14.4% 2.9% 11.4% 3.1% 18 0.1% 0 5,260 12,261 # 2,150 841 3,444 8 44 51 14 2 46 1 1 0% 42.9% 100% % 17.5% 6.9% 28.1% 0.1% 0.4% 0.4% 0.1% 0.0% 0.4% 0.0% 0.0% 1 0.0% 0 5,658 12,261 0.0% 46.1% 100% Note: 1. An individual may have more than one referral, at least one of which closed during SFY 2016. 2. An individual may select more than one race; therefore, the total for the Race categories is greater than 12,261. 3. The client’s age was calculated using the date of birth and the end of the fiscal year (June 30, 2016). 11 AFF Referrals This section on AFF Referrals presents data on how referrals and the clients who received each referral move through the AFF program. Data is presented in two ways: 1) referrals to the AFF program, and 2) unique individuals referred to the AFF program. As a single individual may receive multiple referrals, the total number of referrals will be greater than the unique individuals. Referrals data is presented by quarter and as an annual total. The quarterly counts contain new referrals for that quarter and continuing referrals (i.e., referrals which began before SFY 2016). The total count (aka the annual count) of referrals includes all referrals (both new and continuing) for those clients who received services in SFY 2016. 2 The data presented in Table 2 through Table 9 provides an overview of how individuals enter the program, progress through services, and exit from the AFF program. Exhibit 3 in this report diagrams the flow of AFF Services. The following table (Table 2) displays the values for: 1) New Referrals: New referrals for AFF services reported in the SFY 2016 (July 1, 2015 – June 30, 2016) 2) Continuing Referrals a. Referrals for AFF services prior to SFY 2016 and still open at the end of SFY 2016 b. Referrals for AFF services prior to SFY 2016 that closed in SFY 2016 3) Total Number of Unique Individuals: Unique individuals that have received: a. New referrals for AFF services reported in the SFY 2016 b. Referrals for AFF services prior to SFY 2016 and not closed by the end of SFY 2016 c. Referrals for AFF services prior to SFY 2016 that closed in SFY 2016 2 In Table 2, the Total column for Continuing Referrals is an unduplicated count. A referral may be counted in multiple quarters as long as the referral is open. For example, a referral opened on June 1, 2015 and closed on December 30, 2015 would be counted as a continuing referral in Quarter 1 and Quarter 2 because the client received AFF services during each quarter. 12 4) Number of Individuals with more than one referral: Individuals that have received referrals/services in SFY 2016 that have received more than one referral regardless of SFY A total of 9,611 new referrals were received by AFF providers during SFY 2016, averaging 2,400 referrals per quarter. The 5,030 continuing referrals were referrals that opened prior to SFY 2016 and continued into the SFY 2016. In addition, a total of 12,261 unduplicated individuals received AFF services during SFY 2016. Approximately one-quarter (27.5%) of the unique individuals received more than one referral for AFF services. Table 2: AFF Referrals Total State Fiscal Year 2016 N New Referrals 9,611 Continuing1 Referral s Total Referrals2 Total # of Unique Individuals # of Individuals with more than one referral Quarter 4 % 65.6 % 34.4 % 5,030 14,64 1 100% 12,26 1 100% 3,376 27.5% N 2,25 7 1,88 0 4,13 7 % 54.6 % 45.4 % 100 % Quarter 3 N 2,55 0 2,18 6 4,73 6 % 53.8 % 46.2 % 100 % Quarter 2 N 2,26 2 2,35 7 4,61 9 % 49.0 % 51.0 % 100 % Quarter 1 N 2,54 2 3,17 9 5,72 1 % 44.4 % 55.6 % 100 % 1 Continuing referrals are referrals opened pre SFY 2016 and not closed as of start of SFY 2016 or referrals opened pre SFY 2016 and closed in SFY 2016. 2 The Total Referrals value of 14,641 includes 468 referrals where the evaluators were unable to match the referral to a specific client. These 468 referrals are excluded from the Total # of Unique Individuals. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. In Table 2, the number of referrals for each quarter includes continuing referrals for clients who began receiving AFF services prior to SFY 2016 and continued to receive services during SFY 2016. Therefore, the total value for all four quarters will exceed the total for SFY 2016. The total number of new referrals AFF providers received during SFY 2016 varies slightly each quarter. The highest number of new referrals occurred in Quarter 3 (2,550) and the lowest number of new referrals occurred in Quarter 4 (2,257) in Quarter 4. The number of continuing referrals has a higher variation between the four quarters as retention decreases each quarter. In Quarter 1, there were 3,179 continuing referrals while in Quarter 4 there were 1,880 continuing referrals. The decline in the number of continuing referrals from Quarter 1 to Quarter 4 is anticipated as clients who began receiving AFF services prior to July 1, 2015 exit the program because they either completed their treatment or withdrew from services. 13 AFF Referral Sources Table 3 displays the values for the origin of the referrals. There are two possible sources: the Department of Child Safety and TANF/Jobs Program. As shown in Table 3, the majority of the referrals (99.9%) originate from the Department of Child Safety. Of the 14,641 referrals, 12 came from the TANF/Jobs program. However, limitations in the data from the TANF and JOBS programs do not allow a full assessment of the AFF program in providing services to persons receiving TANF. Table 3: AFF Referral Sources Total State Fiscal Year 2016 Total Referrals1,2 Referred from Department of Child Safety Referred from TANF/ JOBS Program Quarter 4 Quarter 3 Quarter 2 Quarter 1 N % N % N % N % N % 14,641 100% 4,137 100% 4,736 100% 4,619 100% 5,721 100% 14,629 99.9% 4,131 99.9% 4732 99.9% 4613 99.9% 5713 99.9% 12 0.1% 6 0.1% 4 0.1% 6 0.1% 8 0.1% Total # of Unique 12,261 100% Individuals3 Referred from Department of Child 12,260 100% Safety Referred from TANF/ JOBS 14 0% Program 1 Total Referrals from AFF Referrals Table 2 Exhibit 2 The Total Referrals value of 14,641 includes 468 referrals where the evaluators were unable to match the referral to a specific client. These 468 referrals are excluded from the Total # of Unique Individuals. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. 3 Total Individuals from AFF Referrals Table 2 4 Out of the 12 referrals from the TANF/JOBS program, only one referral contained information to identify the Unique Individual. The remaining 11 referrals from the TANF/JOBS program are part of the referrals missing information on the specific client and are excluded from the information on Unique Individuals. In Table 3, the number of referrals for each quarter includes continuing referrals for clients who began receiving AFF services prior to SFY 2016 and continued to receive services during SFY 2016. Therefore, the total value for all four quarters will exceed the total for SFY 2016. 14 Individual clients may have received a referral from more than one source, for example, the first referral to AFF may have originated with DCS while the second referral originated with the TANF/JOBS program. When a client has more than one referral from more than one source, the most recent referral is used in the Unique Individual Count in Table 3. AFF Outreach Table 4 shows that the majority of all referrals to the AFF program (96.0%) received some form of outreach, and the majority of these referrals (95.0%) reported their first outreach attempt within one day or less. An outreach attempt is counted if: 1) The AFF outreach staff was able to contact the client 2) The AFF outreach staff attempted to contact the client, but no contact occurred As noted, 95.0% of the outreach attempts occurred within one business day of the AFF provider receiving the referral. An AFF provider may not be successful in reaching the referred individual on the first attempt. The Scope of Work for each AFF provider specifies that a minimum of three outreach attempts must be completed within five business days to engage the client. This means that if an AFF provider did not contact the referred client on the first attempt, then the AFF provider should make at least two additional attempts to contact the referred client within five business days. When the referred client has been successfully contacted within five business days whether on the first, second, or third outreach attempt, the referral (or unique individual) is included in the count for three outreach attempts within five business days. Additionally, if three outreach attempts with at least one in person outreach attempt are made and all three outreach attempts are unsuccessful, then the referral (or unique individual) is included in the count for three outreach attempts within five business days. Of the 14,641 referrals received in SFY 2016, 14,054 received an outreach attempt.Three outreach attempts (one of which was in person) were made within five business days for 9,449 referrals (67.2%). The 9,449 referrals with three outreach attempts (one of which was in person) made within five business days includes referrals where the client was successfully contacted and referrals where the client was not successfully contacted. Table 4 also shows the outreach data for the unique individuals who received AFF services in SFY 2016. Out of the 12,261 unique individuals referred to AFF services in SFY 2016, the first outreach attempt was made within one business day for 96.6%. A total of 8,517 unique individuals (70.5%) had three outreach attempts in five business days. The 8,517 unique individuals who received three outreach attempts (including one in person attempt) within five business days includes clients who were successfully contacted as well as clients who AFF providers were unable to contact. 15 Table 4: AFF Outreach Total State Fiscal Year 2016 N % Total Referrals1,2 With Outreach Attempt With a first outreach attempt within one business day from referral With a first outreach attempt greater than one business day With three outreach attempts within five business days 14,641 14,054 13,345 709 9,449 100% 96.0% 95.0% 5.0% 67.2% Total # of Unique Individuals3 With Outreach Attempt With a first outreach attempt within one business day from referral With a first outreach attempt greater than one business day With three outreach attempts within five business days 12,261 12,088 11,673 415 8,517 100% 98.6% 96.6% 3.4% 70.5% 1 Total Referrals from AFF Referrals Table 2 The Total Referrals value of 14,641 includes 468 referrals where the evaluators were unable to match the referral to a specific client. These 468 referrals are excluded from the Total # of Unique Individuals. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. 3 Total Individuals from AFF Referrals Table 2 2 AFF Accepted Services Once the AFF provider has successfully contacted the client through outreach efforts, providers attempt to engage the client into services. This is reflected by the client’s signature on a release of information (ROI) form. This form authorizes the treatment provider to gain access to the client’s clinical records as well as authorize the treatment provider to share their treatment information with DCS. The ROI also indicates the parent has voluntarily requested AFF services. Table 5 displays whether or not the client signed the ROI. As shown in Table 5, approximately three-fifths (62.0%) of all referrals and over two-thirds (71.7%) of unique individuals referred for AFF services signed the ROI to accept and participate in AFF services. The SFY 2016 data shows an increase in the number of referrals that resulted in a signed ROI (62.0%) compared to the SFY 2015 data where only 50.9% of referrals resulted in a signed ROI. 16 Table 5: AFF Accepted Services State Fiscal Year 2016 Total Referrals1, 2 With acceptance of services (with a signed ROI) Without acceptance of services (no signed ROI) Total # of Unique Individuals3 With acceptance of services (with a signed ROI) Without acceptance of services (no signed ROI) Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N 14,641 % 100% N 4,137 % 100% N 4,736 % 100% N 4,619 % 100% N 5,721 % 100% 9,079 62.0% 2,507 60.6% 3,244 68.5% 3,194 69.1% 3,719 65.0% 5,562 38.0% 1,630 39.4% 1,492 31.5% 1,425 30.9% 2,002 35.0% 12,261 100% 8,795 71.7% 3,466 28.3% 1 Total Referrals from AFF Referrals Table 2 2 The Total Referrals value of 14,641 includes 468 referrals where the evaluators were unable to match the referral to a specific client. These 468 referrals are excluded from the Total # of Unique Individuals. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. 3 Total Individuals from AFF Referrals Table 2 In Table 5, the number of referrals for each quarter includes continuing referrals for clients who began receiving AFF services prior to SFY 2016 and continued to receive services during SFY 2016. Therefore, the total value for all four quarters will exceed the total for SFY 2016. As seen in Table 5, there is a difference in percentages among the four quarters. The data for Quarter 2 indicates the highest percentage of ROI signers (69.1%). Data for Quarter 4 show the lowest percentage of ROI signers (60.6%). The lower percentage of referrals in Quarter 4 may reflect two separate issues. First, the quarterly data presented in Table 5 is a combination of “new” and “continuing” referrals. As seen in Table 2, the continuing referrals decreased with each quarter as clients referred to the AFF program before July 1, 2015 exited or completed the program. The decline in “continuing” referrals is contributing to the reduction in the number and percentage of referrals with a signed ROI. Second, “new” referrals that AFF providers received in Quarter 4 may not have had sufficient time for the client to sign an ROI before the end of the state fiscal year on June 30, 2016. Any clients who sign an ROI after June 30, 2016, will be included in next year’s annual report as part of the “continuing” referral population. 17 AFF Substance Abuse Assessment Once an individual agrees to participate in AFF services, the AFF Provider conducts a comprehensive psychosocial assessment, including an assessment of substance abuse patterns, prior to developing a treatment plan. Table 6 presents both: 1) The number of referrals that resulted in a signed Release of Information form (N=9,079) 2) The number of unique individuals who signed the Release of Information form (N=8,795) Out of 9,079 referrals, the substance abuse assessment was completed for 8,747 (96.3%). There was no substance abuse problem indicated for 808 (9.2%) of the 8,747 referrals at assessment. The 808 referrals with a substance abuse assessment that showed no need for substance abuse treatment were closed by providers. Out of the 8,795 unique individuals who signed the Release of Information form, 8,248 (93.8%) of the individuals were assessed for substance abuse. There was no substance abuse problem indicated for 774 (9.4%) of the 8,248 unique individuals at assessment; therefore, the clients did not meet AFF program requirements and were closed by providers. Table 6: AFF Substance Abuse Assessment State Fiscal Year 2016 Total referrals with acceptance of services (with a signed ROI)1, 2 Total Assessments With Substance Abuse Assessment With Substance Abuse Assessment and found not to need SA Treatment Total # of Unique Individuals with acceptance of services (with a signed ROI)3 Total Assessments With Substance Abuse Assessment With Substance Abuse Assessment and found not to need SA Treatment Total N 9,079 8,747 7,939 % 100.0% 96.3% 90.8% 808 9.2% 8,795 8,248 100.0% 93.8% 7,474 90.6% 774 9.4% 1 Total referrals with acceptance of services (with a signed ROI) from AFF Acceptance of Services Table 5 2 The Total referrals with acceptance of services value of 9,079 includes 127 referrals where the evaluators were unable to match the referral to a specific client. These 127 referrals are excluded from the Total # of Unique Individuals with acceptance of services. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. 3 Total individuals with acceptance of services (with a signed ROI) from AFF Acceptance of Services Table 5 While preparing the unique individual count data for Table 6, the evaluators identified a total of 51 individuals who received multiple referrals and completed multiple assessments with different results during SFY 2016. At least one assessment indicated that the client needed substance abuse treatment services and other assessments completed at a second time point indicated that 18 the client did not need substance abuse treatment services. For these 51 clients, the result of their most recent assessment was utilized to assign them to the category of “With Substance Abuse Assessment” or “With Substance Abuse Assessment and found not to need SA Treatment.” For 26 clients who had multiple referrals to the AFF program and completed multiple assessments during SFY 2016, their most recent assessment showed they needed substance abuse treatment services. Another 25 clients also received multiple referrals during SFY 2016 and completed multiple assessments. For these 25 clients, their most recent assessment indicated that they did not need substance abuse treatment. AFF Funding Source at Assessment AFF program policies allow AFF treatment providers to make use of substance abuse assessments by other providers or systems if the assessments occurred within the six-month period immediately preceding the referral for AFF services. Table 7 displays the possible funding sources: 1) Arizona Department of Child Safety/Arizona Families F.I.R.S.T. 2) Arizona Department of Health Services/Division of Behavioral Health 3) Private insurance 4) Tribal funding 5) Federal funding for veterans 6) Medicare It is important to note that Table 7 only includes referrals with a signed Release of Information and a completed substance abuse assessment (N=8,747). 3 For 35 referrals (0.4%), a funding source was not identified; therefore, the funding source is listed in Table 7 as “Unknown.” Among the referrals with an identified assessment funding source, the data indicate that more than half (57.4%) of assessments are funded by the Arizona Department of Health Services/Division of Behavioral Health. The Arizona Department of Child Safety/Arizona Families F.I.R.S.T. program funded 37.9% of the assessments in SFY 2016. Table 7: AFF Funding Source at Assessment State Fiscal Year 2016 Total N % DCS/AFF DBHS Private Insurance Tribal Funded Veteran Medicare Unknown1 3,319 5,025 262 28 4 74 35 37.9% 57.4% 3.0% 0.3% 0.0% 0.8% 0.4% Total # of Referrals 8,747 100% 1 Unknown represents empty Assessment Funding Source field. 3 Table 7 references the 8,747 Total Assessments documented in Table 6 completed for referrals with a signed ROI. Referrals without a signed ROI did not complete a substance abuse assessment are excluded from Table 6 and Table 7. 19 AFF Level of Care at Initial Assessment AFF program policies require AFF providers to report levels of care for AFF clients throughout the course of their treatment. Table 8 displays the Levels of Care at the time of initial assessment. The options for Level of Care are: 1) Outpatient 2) Intensive Outpatient 3) Residential Treatment Adult 4) Residential Treatment Child 5) Recovery Maintenance/Aftercare It should be noted that Table 8 displays referrals (N=7,939) which had a signed the Release of Information form, and identified a need for a Substance Abuse Treatment. Out of 7,939 referrals, 92.2% had a reported level of care assignment while 7.8% of those referrals had no reported Level of Care assignment. Referrals with a missing Level of Care at Initial Assessment reflect two different scenarios: 1) New clients who have not been assigned to a Level of Care at Initial Assessment by the end of the report period 2) Cases where the client completed the assessment but exited from the program before receiving services associated with assigning a Level of Care The most commonly reported levels of care at initial assessment were Outpatient (70.2%) and Intensive Outpatient (21.4%). The very low rate of the “Residential Treatment Child” Level of Care (0.0%) in Table 8 indicates that it is uncommon for clients to enter a treatment facility with their children. Instead, most individuals in AFF services utilize residential treatment facilities where they cannot bring their children. The very low rate of Recovery Maintenance/Aftercare (0.4%) seen in Table 8 is not surprising because Table 8 focuses on the Level of Care where clients started after their initial assessment. Since individuals were just beginning to receive substance abuse treatment service, it is logical to assume that very few clients would begin in the Recovery Maintenance/Aftercare level of care. Out of 7,474 unique individuals, 93% had a reported level of care assignment while 7.0% of those individuals had no reported level of care assignment for the entire duration of their AFF treatment. 20 Table 8: AFF Level of Care at Initial Assessment State Fiscal Year 2016 Total referrals with acceptance of services (with a signed ROI)1, 2 Outpatient Intensive Outpatient Residential Treatment Adult Residential Treatment Child Recovery Maintenance/ Aftercare No Level of Care Identified Total # of Unique Individuals with acceptance of services (with a signed ROI)3 Outpatient Intensive Outpatient Residential Treatment Adult Residential Treatment Child Recovery Maintenance/ Aftercare No Level of Care Identified Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N % N % N % N % N % 7,939 5,574 100% 70.2% 455 186 100% 40.9% 1,012 522 100% 51.6% 1,205 651 100% 54.0% 1,611 947 100% 58.8% 1,701 21.4% 85 18.7% 207 20.5% 322 26.7% 386 24.0% 15 0.2% 0 0.0% 3 0.3% 4 0.3% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 33 0.4% 89 19.6% 161 15.9% 163 13.5% 141 8.8% 616 7.8% 95 20.9% 119 11.8% 65 5.4% 137 8.5% 7,474 5,277 100% 70.6% 1,627 21.8% 15 0.2% 0 0.0% 33 0.4% 522 7.0% 1 Total referrals with acceptance of services (with a signed ROI) and identified a need for a Substance Abuse Treatment Table 6 2 The Total referrals with acceptance of services value of 7,939 includes 12 referrals where the evaluators were unable to match the referral to a specific client. These 12 referrals are excluded from the Total # of Unique Individuals with acceptance of services and identified a need for Substance Abuse Treatment. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. 3 Total individuals with acceptance of services (with a signed ROI) and identified a need for a Substance Abuse Treatment Table 6 21 AFF Level of Care at Closure As noted above, AFF program policies require AFF providers to report levels of care for AFF clients throughout the course of their treatment. Table 9 displays the possible Levels of Care at the time of closure: 1) Outpatient 2) Intensive Outpatient 3) Residential Treatment Adult 4) Residential Treatment Child 5) Recovery Maintenance/Aftercare It should be noted that Table 9 displays only the referrals that closed during SFY 2016 for unique individuals who signed the Release of Information (ROI) form and were identified as needing Substance Abuse Treatment (N=4,101). Similar to the data in Table 8: AFF Level of Care at Initial Assessment, Outpatient (53.3%) and Intensive Outpatient (22.8%) are the more commonly reported level of care among those individuals who were closed in SFY 2016 for whom a level of care was reported. It is also important to note that a level of care was not entered for 8.7% of the referrals closed after the client signed the Release of Information. As Table 9 documents the Level of Care at Closure, the referrals without a Level of Care reflect situations where the referral closed before a Level of Care was assigned. For the 3,932 unique individuals with a signed ROI, the percentages for each Level of Care category at closure mirror the data seen for each referral. The majority of these individuals exited the program while in Outpatient care (53.9%) or Intensive Outpatient care (22.1%). However, the percentage in Recovery/Maintenance Aftercare increased to 15.6% from only 0.4% at Initial Assessment. 22 Table 9: AFF Level of Care at Closure State Fiscal Year 2016 Total closed referrals with acceptance of services (with a signed ROI)1, 2 Outpatient Intensive Outpatient Residential Treatment Adult Residential Treatment Child Recovery Maintenance/ Aftercare No Level of Care Identified Total closed Individuals with acceptance of services (with a signed ROI)3 Outpatient Intensive Outpatient Residential Treatment Adult Residential Treatment Child Recovery Maintenance/ Aftercare No Level of Care Identified Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N % N % N % N % N % 4,101 2,184 100% 53.3% 393 154 100% 39.2% 956 479 100% 50.1% 1,171 624 100% 53.3% 1,581 927 100% 58.6% 937 22.8% 71 18.1% 193 20.2% 303 25.9% 370 23.4% 7 0.2% 0 0.0% 3 0.3% 4 0.3% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 615 15.0% 100 25.4% 173 18.1% 185 15.8% 157 9.9% 358 8.7% 68 17.3% 108 11.3% 55 4.7% 127 8.0% 3,932 2,121 100% 53.9% 868 22.1% 7 0.2% 0 0.0% 613 15.6% 323 8.2% 1 Total closed referrals with acceptance of services (with a signed ROI) and identified a need for a Substance Abuse Treatment 2 The Total closed referrals with acceptance of services value of 4,101 includes three referrals where the evaluators were unable to match the referral to a specific client. These three referrals are excluded from the Total # of Unique Individuals with acceptance of services and identified a need for Substance Abuse Treatment. The evaluator is working with the AFF providers to identify the discrepancy and resolve the issue. 3 Total closed individuals with acceptance of services (with a signed ROI) and identified a need for a Substance Abuse Treatment. 23 AFF Services The AFF Services section transitions from documenting how clients referred to the AFF program enter, move through, and complete the program to providing a broader overview of the services and supports clients received. Table 10 addresses additional services provided to clients that fall outside the following fields: Level of Care, Case Coordination with Auxiliary Services, Concrete Supportive Services, or Substance Abuse Service. There are eight additional service options: 1) Parenting 2) Job Readiness/Employment 3) Mental Health Services 4) Medical Services 5) Domestic Violence Services 6) Crisis Services 7) Basic Life Needs 8) Other Services These eight services are provided to clients by the AFF providers and documented by the caseworkers. Table 10 displays only the services provided in SFY 2016 for the clients who identified a need for Substance Abuse Treatment. The 7,448 clients identified in Table 6 as needing Substance Abuse Treatment in SFY 2016 received a total of 117,460 services (or nearly 16 services per client). The 117,460 services provided to clients in SFY 2016 are detailed in Table 10. Parenting and Mental Health Services account for more than 65% of the services received by AFF clients. 24 Table 10: AFF Services1 State Fiscal Year 2016 Parenting Job Readiness/ Employment Mental Health Services Medical Services Domestic Violence Services Crisis Services Basic Life Needs Other Services Total 1 Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N 6,841 % 5.8% N 2,037 % 6.2% N 1,683 % 6.3% N 1451 % 5.7% N 1419 % 5.4% 953 0.8% 162 0.5% 8 0.0% 7 0.0% 3 0.0% 71,101 60.5% 20,223 61.5% 17,006 63.7% 15356 60.6% 16211 61.1% 0 0.0% 0 0.0% 5 0.0% 4 0.0% 2 0.0% 0 0.0% 0 0.0% 7 0.0% 4 0.0% 5 0.0% 452 0.4% 135 0.4% 63 0.2% 56 0.2% 42 0.2% 5,100 4.3% 1,247 3.8% 366 1.4% 283 1.1% 418 1.6% 33,013 117,460 28.1% 100% 9,063 32,867 27.6% 100% 7,568 26,706 28.3% 100% 8178 25,339 32.3% 100% 8423 26,523 31.8% 100% Total services provided for clients that have identified a need for a Substance Abuse Treatment The evaluator spent significant time working with the AFF providers to clarify the use of the “Other” services code. Through this communication, the evaluator identified differences in how the individual AFF providers use the “Other Services” code. One provider conducted extensive training with the staff to ensure information entered into the database aligns with the service categories used in the data system. Their efforts resulted in this provider having the lowest percentage of “Other Services” responses. The trainings conducted by this provider are being used as a model for the other providers. Communication with a second AFF provider revealed that the “Other Services” field was being used to document required AFF services such as case coordination, concrete support services, and substance abuse services. The evaluator worked with this AFF provider to separate the documentation of required services from the “Other Services” they provided to clients. Communication with the third AFF provider revealed similar challenges with the inclusion of required AFF services in the “Other Services” field as well as additional training needs and missing data that was not being uploaded to the AFF Data Collection Portal. The evaluator is working with the third provider to ensure the accurate reporting of “Other Services.” It is anticipated that the percentage of “Other Services” will decline as AFF providers refine data collection and reporting processes. Currently, the “Other Services” category captures activities 25 such as the distribution of donated goods to AFF clients (e.g., diapers), clients who only received case management services before exiting the program, and incomplete records where the service provided cannot be identified. 26 AFF Closures The data presented in the AFF Closures section encompasses all new and continuing referrals to AFF providers that closed during SFY 2016. The number of referrals discussed in Table 11 Closure Reasons Pre-AFF Services and Table 12 Closure Reasons Post-AFF Services is greater than the figures reported in Table 8, which presents the Level of Care at Closure for referrals where the client signed an ROI, completed an assessment, and was identified as needing Substance Abuse Services. The data presented in Table 11 Closure Reasons Pre-AFF Services and Table 12 Post-AFF Services provides a broader understanding of when and why referrals were closed during SFY 2016. During SFY 2016, a total of 9,131 referrals were closed; 4,043 referrals (44.3%) closed Pre-AFF Services, and 5,088 referrals (55.7%) closed Post-AFF Services. Closure Reason Pre-AFF Services Table 11 displays the referrals that closed in SFY 2016 before the client received AFF services (N=4,043). Closure Pre-AFF Services includes referrals that were closed because: 1) The outreach attempts to the client were unsuccessful 2) The client did not sign a Release of Information (ROI) form indicating voluntary agreement to participate in the AFF program There are seven closure reasons associated with pre-AFF services: 1) At the time of intake or assessment, the client refused to take part in AFF services 2) Client case was closed because the client was incarcerated by the criminal justice system 3) Client died 4) Client moved out of the area in which they were to receive AFF services 5) No substance abuse problem indicated at assessment 6) Providers were unable to locate the client at outreach 7) Providers were unable to locate the client at intake 27 “Unable to locate for initial outreach” represented the most commonly reported reason a case was closed (45.7%) among the Pre-AFF Service Closures in the SFY 2016. Approximately twofifths of the cases (39.6%) reported the reason of closure was due to “Unable to locate for intake.” “Death” (0.1%), “No Substance Abuse” (1.0%), “Incarcerated” (1.2%) and “Moved out of area” (1.2%) were rarely reported in the SFY 2016. Table 11: Closure Reason Pre-AFF Services1‡ State Fiscal Year 2016 Client refused service at initial referral or assessment Incarcerated Death Moved out of area No Substance Abuse Problem Unable to locate for initial outreach Unable to locate for intake Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N % N % N % N % N % 434 45 4 47 11.3% 1.2% 0.1% 1.2% 95 11 0 10 9.3% 1.1% 0.0% 1.0% 102 11 2 12 10.5% 1.1% 0.2% 1.2% 115 9 0 13 13.0% 1.0% 0.0% 1.5% 122 14 2 12 12.5% 1.4% 0.2% 1.2% 38 1.0% 9 0.9% 11 1.1% 8 0.9% 10 1.0% 1,761 45.7% 346 33.9% 478 49.0% 426 48.0% 511 52.4% 1,528 39.6% 549 53.8% 359 36.8% 316 35.6% 304 31.2% 1 Referrals fall in the Pre-AFF Services category if they closed before signing the Release of Information form. ‡ A total of 186 closures are excluded from this table due to invalid closure reasons entered into the AFF Data Collection Portal. The evaluators continue to work with AFF Providers to minimize errors in the data. Closure Reason Post-AFF Services Table 12 displays the closed referrals in the SFY 2016 that were classified as post-AFF services (N=5,088). Referrals closed after the client signs the Release of Information (ROI) form are classified as “Post-AFF Services” because the client voluntarily agreed to participate in AFF Services. The number of referrals referenced in the Post-AFF Services Closure Reason (Table 12) is greater than the number of referrals in the Table 9: AFF Level of Care at Closure because Table 12 includes clients who exited the program for any reason after signing the ROI form as well as clients who were identified as not needing Substance Abuse Treatment services during their assessment. There are seven closure options associated with post-AFF services: 1) Client case closed because the client was incarcerated by the criminal justice system 2) Client died 3) Client moved out of the area where they were to receive AFF services 4) No substance abuse problem indicated at assessment 5) Unable to locate (post-intake) 6) Client discontinued without completing services (excluding unable to locate) 7) Completed AFF at the conclusion of substance abuse treatment 8) Completed AFF at the conclusion of Recovery Maintenance 28 For the referrals closed in the SFY 2016 and classified in the “Post-AFF Services” category, “Client discontinued without completing services” represented the most commonly reported reason client case was closed (36.8%). Approximately one-quarter of the referrals closed because the client “Completed AFF at the conclusion of Substance Abuse treatment” (31.0%). The least reported closure reasons in the SFY 2016 included: “Incarcerated” (2.1%), “Moved out of area” (1.5%) and “Death” (0.2%). Table 12: Closure Reason Post-AFF Services1‡ State Fiscal Year 2016 Incarcerated Death Moved out of area No Substance Abuse Problem Unable to locate for (postintake) Client discontinued without completing services Completed AFF at the conclusion of Substance Abuse treatment Completed AFF at the conclusion of Recovery Maintenance Unknown Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N % N % N % N % N % 86 8 63 816 2.1% 0.2% 1.5% 19.5% 24 3 14 128 3.2% 0.4% 1.9% 17.3% 19 3 17 242 1.6% 0.2% 1.4% 20.1% 20 2 17 222 1.7% 0.2% 1.5% 19.3% 23 0 15 224 2.1% 0.0% 1.4% 20.5% 198 4.7% 17 2.3% 60 5.0% 62 5.4% 59 5.4% 1,541 36.8% 152 20.5% 432 36.0% 494 43.0% 463 42.3% 1,298 31.0% 386 52.2% 380 31.6% 289 25.2% 243 22.2% 173 4.1% 15 2.0% 48 4.0% 42 3.7% 68 6.2% 1 0.1% 0 0.0% 0 0.0% 0 0.0% 1 0.0% Referrals fall in the Post-AFF Services category when they closed after the client signed the Release of Information form. ‡ A total of 904 closures are excluded from this table due to invalid closure reasons entered into the AFF Data Collection Portal. The evaluators continue to work with AFF Providers to minimize errors in the data. 1 AFF Funding Source at Closure Table 13 displays the funding sources used to pay for the final substance use assessment when a referral closed. Possible funding sources for the substance use assessment at closure include: 1) Arizona Department of Child Safety/Arizona Families F.I.R.S.T. 2) Arizona Department of Health Services/Division of Behavioral Health 3) Private insurance 4) Tribal funding 5) Federal funding for veterans 6) Medicare It should be noted that only the referrals closed in the SFY 2016 (N=9,131) are shown in Table 13. Approximately one-half (55.2%) of the referrals closed in the SFY 2016 indicated the 29 DCS/AFF as the funding source at closure. Approximately two-fifths of the referrals closed (42.2%) indicate the ADHS/DBHS as the funding source at closure. In addition, similar to the funding sources reported at assessment, “Medicare” (0.5%), “Tribal Funded” (0.2%), and “Veteran” (0.0%) were rarely reported as the funding sources when closed in the SFY 2016. Table 13: AFF Funding Source at Closure Total State Fiscal Year 2016 DCS/AFF ADHS/DBHS Private Insurance Tribal Funded Veteran Medicare Unknown1 1 N 5,036 3,850 168 19 2 47 9 % 55.2% 42.2% 1.8% 0.2% 0.0% 0.5% 0.1% Unknown represents when the Closure Funding Source field has an invalid code entered. It should be noted that the most common funding source for referrals switched between Initial Assessment and Closure. As shown in Table 7: AFF Funding Source at Assessment, the most common funding source was ADHS/DBHS, which paid for 57.4% of referrals while the DCS/AFF funded 55.2% of closed referrals. The evaluators communicated with the AFF providers to gain insight into the reasons why the funding source for the substance abuse assessment would change between the Initial Assessment and Closure. Service providers indicated that there could be a multitude of reasons for this change, including situations where clients’ enrollment in AHCCCS lapsed or clients were no longer eligible for AHCCCS because they had obtained employment. Employment Status at Closure Table 14 displays the reported employment status for individuals with a closed referral. Only referrals closed in the SFY 2016 (N=9,131) are shown in Table 14. For nearly half (47.6%) of the closed referrals, the employment status of the individual receiving the referral was “Unknown.” For more than one in five referrals, the individual’s employment status at closure was documented as “Unemployed” (21.9%), while slightly less than one in five referrals had the client’s employment status at closure as “Employed Full-Time” (19.8%). Seven percent (7%) of referrals ended with the client reporting they were employed part-time at closure. The remaining ten categories have a cumulative total of 3.7% of the closed referrals in the SFY 2016. Comparing Employment Status at Closure to the employment information clients referred to AFF reported at intake shows some important changes. For example, at the time of their referral to AFF, more than one in four referrals (28.1%) had the clients’ employment status listed as “Unemployed” compared to 21.9% of referrals where the clients’ employment status is identified as “Unemployed” at closure. The percentage of referrals where the client reported they were “Employed Full-Time” increased from 17.5% at intake to 19.8% at closure. 30 Table 14: Employment Status at Closure State Fiscal Year 2016 Employed Full-Time Employed Part-Time Unemployed Volunteer Vocational Rehabilitation Homemaker Student Retired Disabled Inmate of Institution Work Adjustment Training Transitional Employment Placement Refused Unknown Total # of Referrals Total N 1,806 637 2,003 9 26 72 53 6 159 15 1 1 0 3,897 9,131 % 19.8% 7.0% 21.9% 0.1% 0.3% 0.8% 0.6% 0.1% 1.7% 0.2% 0.0% 0.0% 0.0% 47.6% 100% 31 AFF Substance Use Clients referred to the AFF program, who accept services, complete a drug/ alcohol-screening tool that captures data on their self-reported drug use in the past 30 days. Table 15 displays the substance that referred clients reported using in the past 30 days on the drug/alcohol-use screening tool. Substance abuse data is collected on 12 categories: 1) Alcohol 2) Methamphetamine/Speed (CNS stimulants) 3) Other stimulants (i.e. a stimulant other than methamphetamine/speed or cocaine/crack) 4) Hallucinogens 5) Inhalants 6) Marijuana/Hashish 7) Cocaine/Crack (CNS stimulants) 8) Heroin/Morphine (opiates/narcotics) 9) Benzodiazepines (CNS depressants) 10) Other sedatives/ tranquilizers (CNS depressants) (i.e., a sedative/tranquilizer not represented in the other provided categories) 11) Other Opiates/Synthetics (i.e., an opiate/synthetic drug not represented in the other provided categories) 12) Other Drugs (i.e., a drug not included in the other categories provided) The top eight substances reported by clients are included in the Table 15 and Figure 1. It should be noted that Table 15 displays data from 6,061 drug/alcohol-use screenings completed by clients identified as needing Substance Abuse Treatment services. Out of the 6,061 substance abuse assessments completed in the SFY 2016, eight substances account for 98.5% of the drugs and alcohol clients reported using. The top three substances used among these individuals include: Marijuana/Hashish (30.7%), Methamphetamine/Speed (27.9%), and Alcohol (17.9%). 32 Table 15: AFF Self-Reported Substance Use State Fiscal Year 2016 Total N 1,085 1,690 1,860 423 469 76 290 77 Alcohol Methamphetamine/Speed Marijuana/Hashish Cocaine/Crack Heroin/Morphine Benzodiazepines Other Opiates/Synthetics Other Drugs % 17.9% 27.9% 30.7% 7.0% 7.7% 1.3% 4.8% 1.3% Figure 1: AFF Self-Reported Substance Use Other Drugs State Fiscal Year 2016 1.3% Other Opiates/Synthetics 4.8% Benzodiazepines 1.3% Heroin/Morphine 7.7% Cocaine/Crack 7.0% Marijuana/Hashish 30.7% Methamphetamine/Speed Alcohol 27.9% Methamphetamine/Speed 60% Cocaine/Crack Heroin/Morphine Benzodiazepines Other Opiates/Synthetics Other Drugs Alcohol 0% 10% 17.9% 20% 30% 40% 50% 70% 80% Marijuana/Hashish 90% 100% 33 AFF Drug Test Drug testing is an integral element in the AFF program model, and AFF providers are expected to refer individuals for drug testing and to report the results of these tests on a routine basis. AFF Drug Test Compliance Table 16 displays client compliance with drug testing requirements. The number of required drug tests changes depending on how long the client has been receiving services in the AFF Program. The number of required drug tests by length of enrollment breaks down as follows: 1) At least two drug tests per week for clients in AFF services between 0 to 60 days 2) At least two drug tests per month for clients in AFF services between 61 to 120 days, and 3) At least one drug test per month for clients in AFF services more than 120 days It should be noted that Table 16 displays only the clients who received the referrals, signed the Release of Information form, and indicated a need for a Substance Abuse Treatment. A total of 4,703 unique individuals were enrolled in the AFF Program up to 60 days during SFY 2016. A total of 4,549 unique individuals were enrolled in the AFF Program between 61 to 120 days during SFY 2016. A total of 5,064 unique individuals were enrolled in the AFF Program for more than 120 days during SFY 2016. For all three categories, the percentages for compliance with the drug testing requirements were similar (23.0%, 24.5%, 28.0% respectively). Therefore, about a quarter of the population were compliant to the drug testing requirements. 34 Table 16: AFF Drug Test State Fiscal Year 2016 Compliant Count Drug testing greater than two (>2) times per week (up to 60 days) Drug testing greater than two (>2) times per month (61 to 120 days) Drug testing greater than (>1) time per month (>120 days) Population Compliant N N % 4,703 1,084 23.0% 4,549 1,113 24.5% 5,064 1,420 28.0% Compliance with AFF Drug Testing schedules was analyzed for the first time in SFY 2016. The percentages reported will be used as baseline for future reports with the intent to improve the percentage of clients who are compliant with AFF Drug Testing policies. AFF Drug Test Referral Outcome Table 17 displays the reported results of the drug test administered to the clients. Drug Test Referral data is collected on eight categories: 1) Positive (1 or more substances detected), 2) Negative (No Substances detected), 3) Awaiting Results, 4) Client Refused, 5) Cancelled for reasons beyond client control, 6) Altered specimen/sample, 7) No call/no show for testing, and 8) Test indicates allowable substance. It should be noted that 111,899 drug tests were administered in the SFY 2016 to clients identified as needing Substance Abuse Treatment services. The results of the 111,899 drug tests are displayed in Table 17. The top three reported drug test outcomes are: 1) Negative (No Substances detected) (52.5%), 2) No call/no show for testing (34.2%), and 3) Positive (1 or more substances) (12.8%). The remaining categories only account for 0.5% of the results. It is important to note that in more than half of the drug tests completed in SFY 2016, the results were negative with no substances detected. The evaluator will review the data provided on the drug test results by the AFF providers in future quarters and report whether or not this percentage is maintained or exceeded. 35 Table 17: AFF Drug Test Referral Outcome State Fiscal Year 2016 Positive (1 or more substances detected) Negative (No Substances detected) Awaiting results Client Refused Cancelled for reasons beyond client control Altered specimen/ sample No call/no show for testing Test indicates allowable substance Unknown1 Total 1 Total Quarter 4 Quarter 3 Quarter 2 Quarter 1 N % N % N % N % N % 14,346 12.8% 4,570 14.2% 3,641 13.1% 3,097 11.9% 3,038 11.8% 58,787 119 88 52.5% 0.1% 0.1% 16,285 21 24 50.6% 0.1% 0.1% 15,184 20 17 54.6% 0.1% 0.1% 13,860 19 21 53.1% 0.1% 0.1% 13,458 59 26 52.2% 0.2% 0.1% 160 0.1% 64 0.2% 52 0.2% 23 0.1% 21 0.1% 12 0.0% 1 0.0% 3 0.0% 3 0.0% 5 0.0% 38,266 34.2% 11,199 34.8% 8,858 31.8% 9,056 34.7% 9,153 35.5% 120 0.1% 50 0.2% 38 0.1% 18 0.1% 14 0.1% 1 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.0% 27,813 100% 26,097 100% 25,774 100% 111,899 100% 32,214 100% Unknown represents empty Drug Test Result field. 36 AFF Referral Report Findings and Permanency Outcomes AFF Referral Report Findings (Maltreatment) The AFF Referral Report Findings presented in this section regard those individuals who were referred to the AFF Program and were closed by the end of SFY 2016. The AFF Referral Report Findings on maltreatment present a broader historical view of the AFF program than seen in earlier sections. To prepare the AFF Referral Report Findings data, AFF providers were instructed to provide the evaluators with all data on referrals they received between April 30, 2011 and June 30, 2016. The data presented in this AFF Referral Report Findings section encompasses all unique individuals referred to the AFF program by April 30, 2011 and who closed by June 30, 2016. 4 The evaluator provided the staff at the Department of Child Safety a list of all clients referred to the AFF program between April 30, 2011 and June 30, 2016 (N = 26,215 unique individuals). The 26,215 unique individuals were matched to the data in the CHILDS database to identify the maltreatment report findings associated with that individual. The unique individuals with closed referrals in the AFF program for whom at least one record was identified in the CHILDS database are included in the data presented in this section. Data in this section is presented for unique individuals. As discussed earlier in the report, a unique individual may have multiple referrals to the AFF program. Therefore, the evaluators utilized the following steps to analyze and present the data on maltreatment report findings: 4 Not all of the AFF providers were able to provide historical data going back to April 30, 2011. Some AFF providers did not have historical data going back to April 30, 2011. When AFF providers were unable to provide historical data going back to April 30, 2011, they uploaded the earliest active case through June 30, 2016. 37 Step 1: Data was examined to identify those unique individuals with all AFF referrals closed by June 30, 2016 (the end of SFY 2016). Step 2: A total of 20,169 unique individuals met this criteria and were included in the examination of data on maltreatment findings. Step 3: The 20,169 unique individuals’ AFF referrals were reviewed to determine the closure reason. A total of 4,748 unique individuals had a closure reason of “Completed AFF at conclusion of Substance Abuse Treatment” or “Completed AFF at conclusion of Recovery Maintenance.” Step 4: The Maltreatment Report Findings for these 4,748 unique individuals who successfully completed the AFF program were examined, and the results are presented in Table 18. Step 5: The remaining 15,421 unique individuals had a closure reason that did not indicate successful completion of the AFF program. The Maltreatment Report Findings for these 15,421 unique individuals are presented in Table 19. Table 18 and Table 19 present data in terms of “Pre-AFF referral” maltreatment findings (aka Pre-Referral) and “Post-AFF referral” maltreatment findings (aka Post-Referral). Pre-Referral data includes the maltreatment allegations identified prior to the unique individual receiving a referral to the AFF program. Post-Referral data includes the maltreatment allegations identified after the unique individual was referred to the AFF program. When a unique individual had multiple maltreatment allegations that resulting in different maltreatment findings on or before the AFF referral date, the highest finding level (“Substantiated” being the highest level and “No Report” being the lowest level) was reported in the Pre-Referral section. For example, a unique individual had three maltreatment allegations prior to being referred to AFF with three different findings of unsubstantiated, substantiated, and unsubstantiated. As a result, this individual was included in the “Substantiated” row (i.e., the highest level) in the Pre-Referral section. In situations where a unique individual had multiple maltreatment reports that resulted in different maltreatment findings after the AFF referral date, the highest finding level was reported in the Post-Referral Section. Tables 18 and 19 use the categories of “Substantiated,” “Proposed,” “Unsubstantiated,” and “No Report.” The “Substantiated” row reports unique individuals who received a maltreatment finding of: 1) Substantiated 2) Substantiated Dependency Adjudication The “Proposed” row includes unique individuals who received maltreatment findings of: 1) Proposed Substantiated Perpetrator Deceased 2) Proposed Substantiated Pending Dependency Adjudication 3) Proposed Substantiated 4) Proposed Substantiated – Perpetrator Unknown 5) Request Proposed Substantiated 6) Request Proposed Substantiated Pending Dependency Adjudication The “Unsubstantiated” row includes unique individuals who received a maltreatment finding of “Unsubstantiated”. The “No Report” row includes individuals who did not have a report of abuse, neglect, or maltreatment to the Department of Child Safety. “Unable to Locate” includes individuals who could not be located to investigate the allegation of abuse, neglect, or 38 maltreatment. The “No Matching CHILDS Data” row includes those individuals for whom the AFF information provided could not be matched to the CHILDS database due to human error in data entry; therefore, allegation findings could not be determined. Table 18: AFF Referral Report Findings (Completed AFF Program) includes the 4,748 unique individuals who were referred into the AFF Program, participated in AFF services, and were identified as completing the AFF program by the end of SFY 2016. There are two closure options associated with “Completed AFF Program”: 1) Completed AFF at the conclusion of Substance Abuse Treatment 2) Completed AFF at the conclusion of Recovery Maintenance Table 18 indicates that among individuals having a Substantiated finding prior to being referred to the AFF program, 65% had no reports to DCS after being referred to AFF. Approximately 20% of the group of individuals having a Substantiated finding prior to being referred to the AFF program had an “Unsubstantiated” maltreatment allegation after being referred to the program. Twelve percent (12%) of individuals with prior substantiated findings had another substantiated finding after referral to the AFF program. 39 Table 18: AFF Referral Report Findings (Completed AFF Program) Post-Referral Finding Substantiated Pre-Referral Finding Substantiated (N = 2,907) (61.2% of 4,748 individuals) Proposed (N = 55) (1.2% of 4,748 individuals) Unsubstantiated (N = 851) (17.9% of 4,748 individuals) No Report (N = 395) (8.3% of 4,748 individuals) Unable to Locate (N = 15) (0.3% of 4,748 individuals) No Matching CHILDS Data (N = 525) (11.1% of 4,748 individuals) Post-Referral Total (N = 4,748) (100% of 4,748 unique individuals) Proposed Unsubstantiated No Report Unable to Locate No Matching CHILDS Data Total N % N % N % N % N % N % N % 357 12.3% 71 2.4% 570 19.6% 1,891 65.0% 18 0.6% 0 0% 2,907 100% 3 5.5% 3 5.5% 6 10.9% 43 78.2% 0 0.0% 0 0% 55 100% 115 13.5% 24 2.8% 218 25.6% 488 57.3% 6 0.7% 0 0% 851 100% 46 11.6% 12 3.0% 70 17.7% 266 67.3% 1 0.3% 0 0% 395 100% 2 13.3% 0 0.0% 2 13.3% 11 73.3% 0 0% 0 0% 15 100% 0 0.0% 0 0.0% 0 0.0% 0 0% 0 0% 525 100% 525 100% 523 11.0% 110 2.3% 866 18.2% 2,699 56.8% 25 0.5% 525 11.1% 4,748 100% 40 Table 19: AFF Referral Report Findings (Did not complete AFF Program) includes the 15,421 unique individuals who were referred into the AFF Program and exited the AFF program before completion during SFY 2016. There are nine closure options associated with “Did not complete AFF Program”: 1) At the time of intake or assessment, the client refused to take part in AFF services 2) Client case closed because the client was incarcerated by the criminal justice system 3) Client died 4) Client moved out of the area where they were to receive AFF services 5) No substance abuse problem indicated at assessment 6) Providers were unable to locate the client at outreach 7) Providers were unable to locate the client at intake 8) Unable to locate (post-intake) 9) Client discontinued without completing services (excluding unable to locate) Among the 8,837 individuals who had a substantiated finding before being referred to the AFF program, more than 72.3% of the individuals had no report to DCS after being referred to the AFF program. Approximately 12% had a substantiated finding after referral, and 11.6% had an unsubstantiated finding after referral to the AFF program. 41 Table 19: AFF Referral Report Findings (Did not complete AFF Program) Post-Referral Finding Substantiated Pre-Referral Finding Substantiated (N = 8,837) (57.3% of 15,421 individuals) Proposed (N = 369) (2.4% of 15,421 individuals) Unsubstantiated (N = 2,900) (18.8% of 15,421 individuals) No Report (N = 1,059) (6.9% of 15,421 individuals) Unable to Locate (N = 28) (0.2% of 15,421 individuals) No Matching CHILDS Data (N = 2,228) (14.4% of 15,421 individuals) Post-Referral Total (N = 15,421) (100% of 15,421 unique individuals) Proposed Unsubstantiated No Report Unable to Locate No Matching CHILDS Data Total N % N % N % N % N % N % N % 1,081 12.2% 294 3.3% 1,024 11.6% 6,390 72.3% 48 0.5% 0 0% 8,837 100% 11 3.0% 21 5.7% 31 8.4% 306 82.9% 0 0% 0 0% 369 100% 306 10.6% 83 2.9% 469 16.2% 2,027 69.9% 16 0.6% 0 0% 2,900 100% 101 9.5% 21 2.0% 189 17.8% 741 70.0% 7 0.7% 0 0% 1,059 100% 5 17.9% 0 0.0% 2 7.1% 21 75.0% 0 0% 0 0% 28 100% 0 0% 0 0% 0 0% 0 0% 0 0% 2,228 100% 2,228 100% 1,504 9.8% 419 2.7% 1,715 11.1% 9,485 61.5% 71 0.5% 2,228 14.4% 15,421 100% 42 AFF Permanency Outcomes The AFF Permanency Outcomes presents a broader historical view of the AFF program. As with the AFF Referral Report Findings, AFF providers were instructed to provide the evaluators with all data on referrals they had received between April 30, 2011 and June 30, 2016. 5 The AFF Permanency Outcome section presents data on the children of individuals who were referred to the AFF program by April 30, 2011 and who closed by June 30, 2016. The evaluator provided the staff at the Department of Child Safety with a list of 26,215 unique individuals referred to the AFF program between April 30, 2011 and June 30, 2016. The 26,215 unique individuals were matched to the data in the CHILDS database to identify information on permanency data. A total of 27,092 children 6 were matched in the CHILDS database and are included in the discussion of the Permanency data. The data presented in Table 20: AFF Permanency Status includes the children associated with a parent (aka a unique individual) who received an AFF referral by the end of SFY 2016. There are two options associated with a child’s permanency status: 1) Still in care and 2) Permanency. “Still in care” refers to children who were removed from their home and do not have a return date that indicates they have not been returned to a home through reunification with their parent, placed with a guardian, or adopted or other reason for exiting care. “Permanency” refers to children who were removed from their home and have a return date signifying that they have been reunified with their parent, placed with a guardian, adopted, or exited care for other reasons (e.g. age of majority or transferring jurisdiction). Out of the 27,092 children for whom data was available, more than half of the children (58.6%) achieved permanency by the end of SFY 2016. Table 20: AFF Permanency Status Status Still in care Permanency Total N 11,219 15,873 27,092 % 41.4% 58.6% 100% The data presented in Table 21: AFF Permanency Outcomes includes 15,873 children associated with parents who had received an AFF referral by the end of SFY 2016 and achieved permanency by the conclusion of SFY 2016. The following End of Removal codes provided by DCS are included in the “Guardianship” category: 1) Guardianship by Relative 5 Not all AFF providers were able to provide historical data going back to April 30, 2011. When AFF providers were unable to provide historical data, they uploaded their earliest active case through June 30, 2016. 6 A total of 1,577 unique individuals referred to AFF could not be matched in the Department of Child Safety’s CHILDS database for data on permanency. 43 2) Guardianship by Foster Parent 3) Living with Other Relative 4) Guardianship by Non-Relative The “Adoption” category includes the following End of Removal codes provided by DCS: 1) Adoption by Relative 2) Adoption by Non-Relative 3) Adoption by Foster Parent The “Reunification” category was simply identified as Reunification. Finally, the “Other” category captures children identified with the following End of Removal codes: 1) Runaway 2) Transfer to Another Agency 3) Added in Error 4) Death of Child 5) Age of Majority In cases where the child had multiple removals, the reason for most recent removal was used for the End of Removal code. As shown in Table 21, of the 15,873 children who achieved permanency, nearly half of the children (47.9%) were reunified with their families and 42.8% were adopted. Table 21 also presents the average number of days these children are in Out of Home Care. The children who were eventually reunified with their family had the lowest average for the number of days in out of home care while the children who were adopted had the highest average for the number of days in out of home care. Table 21: AFF Permanency Outcomes Reunification Guardianship Adoption Other Total Children Achieving Permanency Average Number of Days in Out of Home N % Care 7,599 47.9% 358.4 947 6.0% 474.7 6,800 42.8% 715.2 527 3.3% 386.1 15,873 100% 524.5 44 Recommendations and Next Steps As this annual report is the first prepared by the new evaluator hired in March 2016, there are no comparisons with previous annual reports due to modifications in the data collected and analysis methods. Therefore, the recommendations and next steps proposed in this report predominantly focus on correcting data collection methods and reporting challenges identified in the data received from each AFF provider by the evaluator. The continued improvement of data collection methods by the AFF providers and the subsequent transmission of data to the evaluator will increase accuracy of data and reducing missing data. One area identified for further investigation and collaboration between the evaluator and AFF providers is the collection of demographic data on AFF clients. For example, the high percentage of unknown data in the categories of Marital Status, Race, and Education should be explored to understand how data collection practices and client responses impact the data. The AFF providers are discussing modifying their data collection systems with the goal of improving their ability to collect required AFF data such as client demographic information. During the preparation of the annual report, the evaluator worked with the AFF providers to clarify the use of the “Other Services” category when documenting services clients receive (see Table 10: AFF Services). Through this communication, the evaluator and the AFF providers identified data clean-up processes to better capture the services being provided. One AFF provider has identified the need for additional staff training on entering data to ensure it aligns with AFF reporting requirements. This training was conducted for September 2016, and the evaluator will continue to review the data and report back to the AFF provider on the impact of the training. During SFY 2017, the evaluator will use quarterly reports to provide feedback to DCS and AFF providers on the impact on reporting in the “Other Services” category. New information, such as the number and percentage of clients compliant with AFF Drug Testing policies (see Table 16), is presented in the annual report. As SFY 2016 marks the first year this information is available, the percentages identified in Table 16 will be utilized as a 45 baseline for future reports with the intent to increase the percentage of clients who are compliant with AFF Drug Testing policies. One recommendation put forth to the Department of Child Safety is to establish a benchmark for the percentage of referrals that result in a signed Release of Information (ROI) indicating the client accepted AFF services. SFY 2016 showed an increase in the number of referrals with a signed ROI and accepted AFF services compared to SFY 2015 (62.0% in SFY 2016 compared to 50.9% in SFY 2015). Establishing a benchmark would help AFF providers measure their performance against a clearly defined standard. 46 Conclusion This report summarizes the key processes and outcomes of the Arizona Families F.I.R.S.T. program during SFY 2016 (July 1, 2015 – June 30, 2016). The evaluation of the AFF program and the preparation of this report were conducted independently by Wellington Group. The content of this report, and the conclusions contained herein, represent the opinions of Wellington Group and do not necessarily reflect the opinions of or endorsement by DCS. There are limitations to the data available for this report. With the launch of a new data collection portal, the evaluator and the AFF providers are working together to explore how to maximize data collection and the portal. The clean-up of old and new data along with addressing missing data will improve future reports. Further, the evaluator has no direct involvement in the collection of the data. As a result, there are variations among the providers in how data are collected and the extent to which data are missing. Continuous communication between the evaluator, DCS, and the AFF providers will enhance data clean-up in the data collection portal as well as improve the collection and reporting systems. The challenges faced this fiscal year are not surprising as a number of changes occurred in SFY 2016. The changes include the hiring of a new evaluator as well as a new data collection portal, which includes new programming and reporting requirements for the evaluator, DCS and the providers. These challenges will decrease as communication and problem solving continue to occur among DCS, the AFF providers, and Wellington Group. As a legislatively mandated element of the AFF program, this annual evaluation report provides analysis of the performance of DCS and its contracted AFF providers in meeting the legislative mandates of the program. These mandates include: 1) Increasing the availability, timeliness and accessibility of substance abuse treatment to improve child safety, family stability, and permanency for children in foster care or other out-of-home placement, with a preference for reunification with a child's birth family. 47 2) Increasing the availability, timeliness and accessibility of substance abuse treatment to persons receiving temporary assistance for needy families to achieve self-sufficiency through employment. 3) Increasing the availability, timeliness and accessibility of substance abuse treatment to promote recovery from alcohol and drug problems In regards to the first manadate focused on improving child safety, family stability, and permanency for children, the data included in Table 21 reveals that nearly half of the children (47.9%) who have a permanency data have been reunified with their family. Additionally, more than 40% of children achieved permanency through adoption. In regards to the second mandate focused on achieving self-sufficiency through employment, Table 1 documents that 17.5% of the individuals referred to and receiving AFF services during SFY 2016 were employed full time. Among the referrals that closed during SFY 2016, 19.8% of the referrals closed with the individuals’ employment status listed as employed full time. Finally, in regards to the third mandate focusing on promoting recovery from alcohol and drug problems, the data in Table 5 indicates that during SFY 2016 62.0% of referrals resulted in a signed Release of Information. The data for SFY 2016 is an improvement over the percentage of referrals that resulted in a signed ROI (50.9%) during SFY 2015. 48