ARIZONA FAMILIES F.I.R.S.T. PROGRAM Annual Evaluation Report State Fiscal Year 2009 July 1, 2008 – June 30, 2009 Prepared for Arizona Department of Economic Security Division of Children, Youth and Families Phoenix, Arizona Contract No.: DES060718 Prepared by Center for Applied Behavioral Health Policy College of Public Programs Arizona State University © 2009 by the Arizona Board of Regents for and on behalf of Arizona State University and its Center for Applied Behavioral Health Policy Center for Applied Behavioral Health Policy 500 N. Third Street, Suite 200 Phoenix, AZ 85004 email: cabhp@asu.edu website: http://www.cabhp.asu.edu This document may be copied and transmitted freely. No deletions, additions or alterations of contents are permitted without the expressed, written consent of the Center for Applied Behavioral Health Policy. Acknowledgements This report was prepared by the Center for Applied Behavioral Health Policy (CABHP), College of Public Programs, Arizona State University, under contract number DES060718-1 with the Arizona Department of Economic Security, Division of Children, Youth and Families (ADES/DCYF), in partnership with the Arizona Department of Health Services, Division of Behavioral Health Services (ADHS/DBHS) through the Joint Substance Abuse Treatment Fund. Michael S. Shafer, Ph.D. and Brian Arthur are responsible for the content and writing of the report. Appreciation is given to Charles Davis, Lisa Moen, Richard Rivera, and Julie Sauvageot for contributing to the report’s analyses, review, and production. The authors wish to thank staff of the ADES/DCYF and the ADHS/DBHS for their ongoing cooperation and assistance with the evaluation. In particular, Susan Blackburn, Karen Bulkeley, Cathy Hasenberg and Mark Klym of ADES, and Jennie Lagunas and Steve Lazere of ADHS have been very helpful throughout the course of the evaluation. Points of view represented in this report are those of the authors, and do not necessarily represent the official position or policies of either the ADES or the ADHS. Suggested citation: Center for Applied Behavioral Health Policy. (2009). Arizona Families F.I.R.S.T. Program: Annual Evaluation Report for the Period July 1, 2008 – June 30, 2009. Tempe, AZ. Arizona State University. i TABLE OF CONTENTS Acknowledgements .......................................................................................................................... i Table of Contents.............................................................................................................................1 Executive Summary .........................................................................................................................2 Chapter 1: Introduction ...................................................................................................................5 Chapter 2: Evaluation Framework and Data Sources .....................................................................9 Chapter 3: Arizona Families F.I.R.S.T. Clients and Services Received.......................................10 Chapter 4: AFF Program Outcomes .............................................................................................23 Chapter 5: Summary and Conclusions ..........................................................................................27 Appendix A ...................................................................................................................................30 Center for Applied Behavioral Health Policy y Arizona State University 1 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report EXECUTIVE SUMMARY Arizona Families F.I.R.S.T. Program Model Arizona Families F.I.R.S.T. (Families in Recovery Succeeding Together – AFF) was established as a community substance abuse prevention and treatment program by ARS 8-881. AFF is a program that provides family-centered substance abuse and recovery support services to parents or caregivers whose substance abuse is a significant barrier to maintaining or reunifying the family or achieving self-sufficiency. The program provides an array of structured interventions to reduce or eliminate abuse of and dependence on alcohol and other drugs, and to address other adverse conditions related to substance abuse. Interventions are provided through the Department of Economic Security, Division of Children, Youth and Families (DES/DCYF) contracted community providers in outpatient and residential settings, or through the Regional Behavioral Health Authority (RBHA) provider network under the supervision of the Department of Health Services, Division of Behavioral Health Services (DBHS). AFF emphasizes face-to-face outreach and engagement at the beginning of treatment, concrete supportive services, transportation, housing, and aftercare services to manage relapse occurrences. The service delivery model incorporates essential elements based on family and community needs, such as culturally responsive services, gender-specific treatment, services for children, and motivational enhancement strategies to assist the entire family in its recovery. The evaluation of AFF, required by ARS 8-884, focuses on the fidelity of program implementation of the AFF model, performance of service providers, factors that contribute to client success, and the extent to which the legislative outcome goals were met: • • • • • Increases in timeliness, availability and accessibility of services Recovery from alcohol and drug problems Child safety and reduction of child abuse and neglect Permanency for children through reunification Achievement of self-sufficiency through employment This year’s evaluation continues the focus on the documentation of program implementation through the analysis and reporting of client-level service data from DES providers and DBHS, qualitative data gathered from AFF program directors, and selected substance abuse treatment and child welfare outcomes for the period July 1, 2008 through June 30, 2009. Key findings for state fiscal year (SFY) 2009, categorized by the aforementioned legislative outcome goals are as follows: Increases in Timeliness, Availability and Accessibility of Services • A total of 3,954 referrals (representing 3,944 unduplicated individuals1) were received by DES providers during SFY 2009, averaging 988 referrals per quarter. Overall, unique 1 Each referral is valid for a six-month period. If an individual does not engage in services within six months of the initial referral, a new referral may be sent to the AFF provider. Center for Applied Behavioral Health Policy y Arizona State University 2 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report • • • • • • individuals referred to the AFF program for SFY 2009 declined by 15.9% compared to SFY 2008. Nearly all clients referred to the AFF program (91%) received at least one or more recorded outreach efforts on or after the referral date by the DES provider within their community. Outreach efforts occurred in a timely manner, averaging 2.1 days, slightly longer than the 1.8 average days recorded in SFY 2008. The observed rate at which new clients were referred to RBHAs was 79% higher than the rate of RBHA referrals observed in SFY 2008 (17% of referrals). A total of 3,1472 individuals (representing 80% of individuals referred to the AFF program) were assessed3 during SFY 2009. 36% of individuals assessed have records from a DES assessment and a DBHS enrollment, 34% were unique assessments provided by DES providers, and 30% were unique assessments reported from DBHS enrollment data. The total clients served increased between 2007 and 2009, from 4,471 clients to 4,845 clients. Recovery from Alcohol and Drug Problems • Based on the initial assessment information collected on 3,147 AFF clients assessed in SFY 2009, about 85% reported they had used alcohol or one or more illicit substances in the 30 days immediately prior to their assessment. Alcohol (46%), methamphetamine (40%), and marijuana (40%) continue to be the more commonly reported substances. • Based on drug screens reported through the AFF web-based data entry portal, 2,492 clients (51.4% of all clients served) had reported drug screens during their AFF program participation. On average these clients were tested 1.24 times per month. • Among those clients with reported drug screens, 67% of the clients tested negative, indicating no drug use, on all of their drug screens during their AFF program participation. Child Safety and Reduction of Child Abuse and Neglect • Nearly all families (91%) served by the AFF program had at least one report of suspected child maltreatment prior to entering AFF. While there were no reports received from DES for the remaining 9% of clients, this is likely due to data matching issues, such as incomplete SSN, CHILDS person ID, etc. • Nearly all (97%) had no subsequent report filed during this reporting period. • Among those families with a substantiated report at intake, only .5% and 2.2% had a subsequent filing of a substantiated or unsubstantiated report, respectively. Permanency for Children through Reunification • A total of 2,443 children whose parents were AFF clients in SFY 2009 were in Child Protective Services (CPS) out of home placement at some point during the reporting 2 Note: This figure includes individuals that had been referred to the AFF program in SFY 2008, but not assessed until SFY 2009, along with clients who were referred and assessed during SFY 2009. 3 The term “assessed” is defined as individuals having completed the DBHS initial “Core Assessment.” Center for Applied Behavioral Health Policy y Arizona State University 3 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report period. Slightly more than one-third of these children (39%) achieved permanency during SFY 2009. Of those who were discharged from CPS out of home placement and achieved permanency (n=948), the majority (84%) were reunified with their families. Achievement of Self-Sufficiency through Employment • Employment activity is reported only for those clients referred to the AFF program who were already enrolled in the Jobs program. Among those clients who were referred to the AFF program and completed AFF services this year (n=3), one client was reported employed at the time of AFF program completion. Center for Applied Behavioral Health Policy y Arizona State University 4 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report CHAPTER 1 INTRODUCTION Arizona Families F.I.R.S.T. (Families in Recovery Succeeding Together) was established as a community substance use disorder prevention and treatment program by ARS 8-881 (Senate Bill 1280, which passed in the 2000 legislative session). Under the requirements of the Joint Substance Abuse Treatment fund that was established under the legislation, Section 8-884 requires an annual evaluation of the Arizona Families F.I.R.S.T. (AFF) program. The evaluation of AFF examines the implementation and outcomes of community substance use disorder treatment services delivered by DES-contracted providers and the Regional Behavioral Health Authorities (RBHA) network. 1.1 Brief Description of the AFF Program and Client Flow The legislation which created AFF is based on the recognition that substance abuse disorder in families is a major problem contributing to child abuse and neglect, and that substance abuse can present significant barriers for those attempting to reenter the job market or maintain employment. In addition, federal priorities under the 1997 Adoption and Safe Families Act (ASFA) that address child welfare outcomes (such as permanency and shorter time frames for reunification) coupled with time limits established under the TANF block grant were factors behind the legislation. However, the timeframes for substance abuse recovery currently viewed as a chronic recurring illness4 sometimes conflict with the requirements of ASFA and Arizona Juvenile Court guidelines. Currently, states must file a petition to terminate parental rights and concurrently identify, recruit, process, and approve a qualified adoptive family on behalf of any child, regardless of age, that has been in foster care for 15 out of the most recent 22 months. AFF is a program that provides contracted family-centered, strengths-based, substance abuse treatment and recovery support services to parents or caregivers whose substance abuse is a significant barrier to maintaining or reunifying the family. The goal of the program is to reduce or eliminate abuse of and dependence on alcohol and other drugs, and to address other adverse conditions related to substance abuse. Interventions are provided through the Department of Economic Security, Division of Children, Youth and Families (DCYF) contracted community providers in outpatient and residential settings or through the RBHA provider network. In addition to traditional services, AFF includes an emphasis on: face-to-face outreach and engagement at the beginning of treatment; concrete supportive services, such as, transportation and housing; and an aftercare phase to manage relapse occurrences. Essential elements based on family and community needs, such as culturally responsive services, gender-specific treatment, services for children, and motivational enhancement strategies to assist the entire family in its recovery, are incorporated into the service delivery. The diagram on the following page shows the flow of clients through various stages of the AFF program. 4 Lesher, A. (2001). Addiction is a brain disease. Issues in Science and Technology. Center for Applied Behavioral Health Policy y Arizona State University 5 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 1 Overview of the AFF Program Model CPS Case workers Jobs Case workers Referred to AFF Provider • Conduct Outreach • Engage client in services • Determine AHCCCS eligibility No AHCCCS eligible? Yes DES/DCYF DES-funded Services • • • • Continue with AFF provider Conduct core assessment Develop service plan Begin services 9 Substance abuse education 9 Outpatient 9 Intensive outpatient 9 Residential treatment ADHS/DBHS RBHA Funded Services • • • • • AFF Funded Supportive Services Child care Transportation Housing Job training, etc. Connect to RBHA provider Conduct RBHA enrollment Conduct core assessment Develop Service plan Begin services 9 Outpatient 9 Intensive Outpatient 9 Residential Treatment Access AFF Aftercare Services Close Case Center for Applied Behavioral Health Policy y Arizona State University Close Case 6 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 2 summarizes the county, DES provider agency, and associated RBHA within each of six regional DES districts. DES-contracted agencies in bold italics also participate in the RBHA network as either a RBHA or a RBHA network provider. Exhibit 2 List of DES Districts, Counties, DES Providers, and RBHAs DES District Regional Behavioral Health Authority DES Provider Agency I Maricopa TERROS Magellan II Pima Community Partnership of Southern Arizona (CPSA) Community Partnership of Southern Arizona (CPSA) Coconino Arizona Partnership for Children (AzPaC-Coconino) Yavapai Arizona Partnership for Children (AzPaC-Yavapai) Apache and Navajo Old Concho Community Assistance Center Yuma Arizona Partnership for Children (AzPaC-Yuma) III Northern Arizona Regional Behavioral Health Authority (NARBHA) Cenpatico Behavioral Health of Arizona, Inc. La Paz WestCare Arizona Mohave WestCare Arizona Northern Arizona Regional Behavioral Health Authority (NARBHA) V Gila and Pinal Horizon Human Services Cenpatico Behavioral Health of Arizona, Inc VI Cochise, Graham, Greenlee, and Santa Cruz Southern Arizona Behavioral Health Services (SEABHS) Community Partnership of Southern Arizona (CPSA) IV 1.2 County Statewide Context of AFF Program and Substance Use and Treatment In 2008, an estimated 22.2 million persons nationwide (8.9 percent of the U.S. population aged 12 or older) were classified with substance dependence or abuse in the past year based on criteria specified in the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV). Of these, 3.1 million were classified with dependence on or abuse of both alcohol and illicit Center for Applied Behavioral Health Policy y Arizona State University 7 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report drugs, 3.9 million were dependent on or abused illicit drugs but not alcohol, and 15.2 million were dependent on or abused alcohol but not illicit drugs.5 The recent data available on substance use in Arizona6 indicate that 9.2% of Arizonans were classified with alcohol or illicit drug dependence or abuse in the past year, slightly higher than the national average. Twenty-one percent of Arizonans 18-25 years of age and 7% of Arizonans 26 years of age or older used illicit drugs during the past month. Further, past-month binge alcohol abuse was reported by 38% and 22% of individuals within these two age groups respectively. Finally, in a recent report on substance use in the 15 largest metropolitan areas,7 7.6% of persons living in the Phoenix metropolitan area aged 12 or older reported using illicit drugs in the past month, and 24.6% of persons living within the Phoenix metropolitan area reported past-month binge alcohol use, significantly higher than the national average. Abuse and neglect of children is generally believed to be associated with substance abuse. In reports on this issue,8,9,10 data were presented showing that parents who abuse drugs and alcohol generally do not attend to children’s emotional cues, are poor role models, and discipline their children less effectively than other parents. It is within this context that the AFF program is meant to intervene and break the cycle of substance abuse, and the abuse and neglect of children. The significant event affecting the AFF program in State Fiscal Year (SFY) 2009 has been the declining economy. In late January 2009, the Arizona State Legislature was called into special session to address an anticipated $1.6 billion state budget shortfall projected by the Joint Legislative Budget Committee for FY 2008 - 2009. The First Special Session made statutory and session law changes necessary to resolve the projected $1.6 billion shortfall, and SFY 2008-2009 appropriations were reduced. On January 31 2009, the Arizona Legislature passed and Governor Brewer signed a revised budget for SFY 2009. As a result of reductions, fund transfers and internal shortfalls, DES had to reduce its budget by $153 million by the end of SFY 2009 (June 30, 2009). The Arizona Families F.I.R.S.T program was not immune to these cuts – approximately $762,500 (representing 11% of the AFF program services budget) was cut from the program during this 5 Substance Abuse and Mental Health Services Administration. (2009). Results from the 2008 National Survey on Drug Use and Health: National Findings (Office of Applied Studies, NSDUH Series H-36, HHS Publication No. SMA 09-4434). Rockville, MD. 6 Substance Abuse and Mental Health Services Administration. (2008). 2007 State Estimates of Substance Use and Mental Health: Arizona. Office of Applied Studies. Rockville, MD. Retrieved September 15, 2009, from http://www.oas.samhsa.gov/2k7state/arizona.htm 7 Substance Abuse and Mental Health Services Administration. (2007). The NSDUH Report. Office of Applied Studies. Rockville, MD. Retrieved September 15, 2009 from http://www.oas.samhsa.gov/2k7/metro/metro.pdf 8 U.S. Department of Health and Human Services (1999). Blending Perspectives and Building Common Ground: A Report to Congress on Substance Abuse and Child Protection. Washington, DC: US Department of Health and Human Services. 9 U.S. General Accounting Office (1994). Foster Care: Parental Drug Abuse Has Alarming Impact on Young Children. GAO/HEHS-94-89. 10 DePanfilis, D. (2006). Child Neglect: A Guide for Prevention, Assessment, and Intervention. Washington, DC: US Department of Health and Human Services, Administration on Children, Youth and Families, Children’s Bureau. Retrieved September 15, 2009, from http://www.childwelfare.gov/pubs/usermanuals/neglect/neglect.pdf Center for Applied Behavioral Health Policy y Arizona State University 8 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report period. Similarly, the Arizona Department of Health Services/Division of Behavioral Health Services was also cut. In response to the Legislative and Governor-approved actions, ADHS announced cuts of $11,624,500 to program services, of which $3,125,000 were directed toward non-SMI adults, including but not exclusively AFF program recipients. Collectively, these midyear budget reductions to both DES and DBHS resulted in reductions in AFF services as are noted throughout this report. In addition to the reduction in funding for service provision from DES and DBHS, the contract for the evaluation of the AFF program was reduced by 27%. This reduction has resulted in significant modifications to this evaluation report that will be apparent to readers of previous annual evaluation reports. Most notably, in many instances DES provider and/or DES district level comparisons have been eliminated and replaced with overall, statewide performance reporting. Likewise, onsite visits to each of the DES programs have been eliminated, along with any in-person interviewing of program staff or clients. The onsite visits have been replaced by semi-structured telephonic interviews conducted with DES provider agency management personnel and program supervisors. The following chapters summarize the methods and findings of the AFF program evaluation for State Fiscal Year 2009 (July 1, 2008 – June 30, 2009). CHAPTER 2 EVALUATION FRAMEWORK AND DATA SOURCES The evaluation design developed for the AFF program focuses on program implementation to determine whether DES provider agencies implemented the services in a manner consistent with their scope of work. The design also addresses whether the AFF outcome goals and performance measures, as well as other outcomes in the areas of substance abuse recovery, family stability, safety, permanency, and self-sufficiency were achieved. The evaluation design is not a longitudinal study of AFF clients using data collected from individual client interviews, nor does it use any comparison group. Rather, the design uses primarily administrative data covering points in time. This year’s report again draws upon information from multiple data sources. Data sets included: • Service utilization data obtained directly from the nine DES providers; • Enrollment and encounter data provided by the Arizona Department of Health Services, Division of Behavioral Health Services (DBHS) for services provided through the local RBHA network; • DES CHILDS information system, which provides child welfare information, and the DES JAS/AZTEC information system, providing employment services information; and • Qualitative information obtained from DES provider agency management personnel and program supervisors. Comments or findings from program managers are provided throughout the report in “text box” format. These comments are from qualitative semi-structured interviews in August and September of 2009. Center for Applied Behavioral Health Policy y Arizona State University 9 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report DES providers use a common data reporting format, revised by CABHP in November 2008. These data are either directly entered into the AFF web-based data entry portal or uploaded by the provider to the portal. The data entered through the AFF web-portal include information regarding outreach efforts, assessment information, drug testing results, and service provision, using a service matrix that emulates the categories of service utilized by DES for payment to their providers. Providers are required to report service data through the AFF web-portal only for client services that are funded by DES. For those client services that are funded through DBHS, enrollment and service encounter data are provided by DBHS. These data are derived from the DBHS Client Information System (CIS). Using the service domain and levels structure and service definitions, specified in the Arizona Department of Health Services’ Covered Behavioral Health Services Guide and Client Information System File Layouts and Specifications Manual12, a common services taxonomy is used to organize the services funded both by DBHS and DES. Three additional data sets used for this evaluation include: the ADES CHILDS information system which provides child welfare information; the ADES JAS/AZTEC information system providing employment services information; and data from the Temporary Assistance for Needy Families (TANF) information systems. The third source of data used for the analysis of the AFF program is the nine DES providers’ agency management personnel and program supervisors. These individuals were interviewed by telephone in September 2009 to gain perspectives on programmatic successes, changes in program implementation, updates on collaborative partnerships, perceived barriers and facilitators to program implementation, changes in contextual issues, and other events that may have influenced service delivery. CHAPTER 3 ARIZONA FAMILIES F.I.R.S.T. CLIENTS AND SERVICES RECEIVED During the SFY 2009 reporting period, a total of 4,845 individuals were served by the Arizona Families F.I.R.S.T. program, representing a 15% decrease over the previous year (5,722 clients) but an 8% increase over SFY 2007. This figure includes clients who were referred, assessed, and received treatment in SFY 2009 (n = 2,936) and clients who were referred and assessed in SFY 2008 and continued to receive services in SFY 2009 (n = 1,909). 12 For more information on how these services are derived, consult http://www.azdhs.gov/bhs/bhs_gde.pdf and http://www.azdhs.gov/bhs/cis.pdf Center for Applied Behavioral Health Policy y Arizona State University 10 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 3 SFY 2009 Referrals and Client Participation Reported Unique Individuals Referred to AFF n = 3,944 Reported Referral Outreach n = 3,616 Reported Individuals Accepting Services13 n = 2,473 Reported Individuals Assessed n = 3,147 2,790 new referrals + 357 referred in SFY 2008 but assessed in SFY 2009 SFY 2009 New AFF Clients14 n = 2,936 SFY 2008 Continuing AFF Clients n = 1,909 + Total AFF Clients, SFY 2009 N = 4,845 DES only Funded Clients n = 1,305 (26.9%) 933 372 clients clients continuing closed to receive from services services DES & RBHA jointly Funded Clients n = 1,685 (34.8%) 624 clients closed in both systems 516 closed by DES, continuing to receive services from RBHA 158 clients closed by RBHA, continuing to receive services from DES 387 clients continuing to receive services from both systems RBHA only Funded Clients n = 1,855 (38.3%) 952 clients 903 clients closed from continuing services to receive services 13 The term “accepting services” is defined as an individual who has been referred to the AFF program, who has been reported through the web-based portal as expressing a willingness to receive AFF services. The number of individuals noted as accepting services (2,473) is less than the number of new referrals assessed (2,790) because no report of accepting services was detected for 317 individuals even though these individuals were subsequently reported as being assessed. 14 Two hundred – eleven (211) individuals assessed in SFY 2009 did not have any services recorded after the assessment date; consequently, they are not included in the count of Total AFF Clients, SFY 2009. Center for Applied Behavioral Health Policy y Arizona State University 11 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report 3.1 Referrals to the AFF Program A total of 3,954 referrals (representing 3,944 unduplicated individuals15) were received by DES providers during SFY 2009, averaging 988 referrals per quarter. Nearly all referrals to the AFF program (99%) were provided by CPS caseworkers, a trend that has been consistent since the inception of the program. Only five referrals came from the Jobs program during the reporting period ending June 30, 2009. There were nine referrals for which the referral source was from another department within DES (n = 4) or unspecified (n = 5) by the DES provider. Overall, the total number of unique individuals referred to the AFF program for SFY 2009 declined by 15.9% compared to SFY 2008, continuing a declining trend in referrals, but exacerbated during the current reporting period. This decline was accelerated during the second half of the state fiscal year (January through June) during which time a 40% decline, relative to the same period in SFY 2008, was observed. These reductions in referrals occurred concomitant with the mid-year budget reductions previously described. Exhibit 4 Statewide AFF Referrals Jul‐Sep 2008 Oct‐Dec 2008 Jan‐Mar 2009 Apr‐Jun 2009 Total Referrals Unique Clients 3.2 SFY 2008 1260 1047 1260 1233 4800 4691 SFY 2009 1256 1208 863 627 3954 3944 % change ‐0.3% 15.4% ‐31.5% ‐49.1% ‐17.6% ‐15.9% Client Outreach and Engagement Nearly all clients referred to the AFF program (91%) received at least one or more recorded outreach efforts on or after the referral date. Outreach efforts occurred in a timely manner, averaging 2.1 days, slightly longer than the 1.8 average days recorded in SFY 2008. It should be noted that a few outreach dates (n = 20) submitted by providers were 30 days earlier than the referral date. DES providers reported that in these instances they had been informally notified by CPS of an impending referral and contacted the client immediately rather than waiting for the formal referral. 15 Each referral is valid for a six-month period. If an individual does not engage in services within six months of the initial referral, a new referral may be sent to the AFF provider. Center for Applied Behavioral Health Policy y Arizona State University 12 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 5 Disposition of Cases Referred to the AFF Program SFY 2008 SFY 2009 n % n % 4800 100% 3954 100% # of referrals # unduplicated referrals16 4691 97.7% 3944 99% # outreached 4365 93.1% 3616 91% 3639 77.6% 2473 62% # of referred clients accepting services17 # referred to RBHA 836 17.8% 1497 36% # of referred clients refusing services 49 1% 30 1% Among referred clients, 2,473 (62%) accepted AFF services, down from the 78% acceptance rate observed in SFY 2008. The average number of days from referral to acceptance was 10.73 days (SD = 28.1) slightly longer than the average time reported for service acceptance in SFY 2008 (mean = 7.8 days; SD = 37.5). Slightly more than one-third of referred individuals were referred to the local RBHA for treatment services because they were Medicaid (Title XIX) eligible. The observed rate of RBHA referral was 79% higher than the rate of RBHA referrals observed in SFY 2008 (17% of referrals). On average, clients were referred to the RBHAs within three weeks of their AFF program referral. Exhibit 6 Duration from AFF Referral to Selected Events (Number of Days; N = 3954) First Outreach Contact 3616 (91.4%) Client Accepting AFF Services 2473 (62.5%) Referral Sent to RBHA 1497 (37.9%) Client Refusing AFF Services 30 (0.7%) Median # Days 1.00 2.00 11.00 5.00 Mean # of Days 2.04 10.73 16.98 17.17 SD 6.6 28.07 19.83 31.93 n 16 The term “referrals” is defined as the receipt of an AFF referral form from DES by a provider. The referral identifies the name of an individual referred for AFF services. 17 The term “accepting services” is defined as a referred individual indicating their willingness to accept AFF services upon outreach by a DES provider. Center for Applied Behavioral Health Policy y Arizona State University 13 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 7 Disposition of Cases Referred to the AFF Program18 Total Individuals Referred 3,954 100% Outreach Attempted 3,616 91% Outreach Attempted Case Closed 300 8% Outreach Attempted In process 177 4% Outreach Completed Refused Services 6 <1% Outreach Attempted Referred to RBHA 443 11% No Outreach Attempted 338 9% Outreach Completed Accepted Referral to Services 2,373 60% No Outreach Attempted Accepted Referral to Services 100 3% Total Accepted Referrals to Services And Assessed 2473 62% No Outreach Information 206 5% Case Closed 32 1% Note: 987 (25%) referred to RBHA Outreach Completed No Acceptance Documented 317 8% New AFF Clients Assessed 2790 71% 18 3,954 referrals were received, representing 3,944 unique clients. At least one and not more than 10 clients had been referred more than once to the AFF program during the fiscal year. Center for Applied Behavioral Health Policy y Arizona State University 14 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report 3.3 DES Provider Assessments and DBHS Enrollments A total of 3,14719 individuals (representing 80% of individuals referred to the AFF program) were assessed during SFY 2009. Assessments were conducted by a contracted DES provider and/or a DBHS/RBHA contracted provider, depending on the referred individual’s eligibility status for RBHA services. Assessment data were compiled from two sources: DES provider data and DBHS enrollment data. As depicted in Exhibit 8, 36% of individuals assessed have records from a DES assessment and a DBHS enrollment, 34% were unique assessments provided by DES providers, and 30% were unique assessments reported from DBHS enrollment data. Total Assessments RBHA only DES & RBHA DES only Exhibit 8 Assessments Statewide SFY 2008 SFY 2009 n % n % 4381 100% 3147 100.0% 2003 45.7% 954 30.3% 1154 26.3% 1134 36.0% 1224 27.9% 1059 33.7% As these data reflect, variations are evidenced when compared to SFY 2008. Most noticeably, these data reflect a significant decline in the proportion of individuals assessed by the RBHA only, concomitant with sharp increases in those clients with both DES provider and RBHA assessments (up 10% ) and those cases assessed by DES providers only (up 6%). Among clients assessed in 2009, the average number of days from referral to assessment was 36 days (SD = 41.7) with half of the clients assessed within 23 days. 3.4 Substance Use Among Clients at Time of DES Assessment or RBHA Assessment Individuals that are assessed by a DES or RBHA provider complete a self-report of their substance use patterns during the immediately preceding 30-day period. Exhibit 9 provides a summary of the substances used by these clients at the time of their initial assessment. About 85% of these clients reported they had used alcohol or one or more illicit substances in the 30 days immediately prior to their assessment. Alcohol (46%), methamphetamine (40%), and marijuana (40%) continue to be the more commonly reported substances. Polysubstance use continues to be the norm with most clients. 19 Note: This figure includes individuals that had been referred to the AFF program in SFY 2008, but not assessed until SFY 2009, along with clients who were referred and assessed during SFY 2009. Center for Applied Behavioral Health Policy y Arizona State University 15 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 9 Substances Used by AFF Clients 30 Days Prior to Enrollment Total Assessed Clients: 3,147 n 2,665 Clients Reporting Use Alcohol 1441 Methamphetamine 1266 Marijuana 1260 Cocaine/crack 460 Other Narcotics 157 Heroin/Morphine 91 Other drugs 82 Hallucinogens 49 Benzodiazepines 44 Other sedatives 27 Other Stimulants 13 Inhalants 7 % 84.5% 45.8% 40.2% 40.0% 14.6% 5.0% 2.9% 2.6% 1.5% 1.4% 0.8% 0.4% 0.2% Exhibit 10 Multiple Substance Use Among AFF Clients whose Clients also used…. Primary Substance Use is…… Alcohol 48% also use Marijuana (n = 1441) 39% also use Methamphetamine 26% also use other illegal substances 56% also use alcohol Marijuana (n=1260) 44% also use methamphetamine 24% also use other illegal substances Methamphetamine 44% also use marijuana (n= 1266) 46% also use alcohol 18% also use other illegal substances Center for Applied Behavioral Health Policy y Arizona State University 16 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report 3.5 Service Access by Service Domain Nearly all clients that were served in the AFF program during the past year received services within the support services domain and the treatment domain. Support services encompass such things as case management and transportation assistance. Treatment services include, for example, counseling (individual, group, family) and outpatient services. Approximately 72% of clients received services within the medical domain, while 19% or fewer of all AFF clients received services within the Rehabilitation, Crisis Intervention, Inpatient, Residential, or Behavioral Health Day Program domains (see Appendix A for service patterns by DES district). Exhibit 11: Statewide Patterns of Service Access Service Domain Treatment Services Rehabilitation Services Medical Services Support Services Crisis Intervention Services Inpatient Services Residential Services Behavioral Health Day Programs 3.6 DES only funded services n = 3156 # % 2499 79.2 201 6.4 2511 79.6 2824 89.5 1 0 0 0 32 1 0 0 DBHS only funded services n = 3660 # % 2942 80.1 762 20.8 1378 37.7 3501 95.7 439 12.0 73 2.0 283 7.7 208 5.7 DES & DBHS funded services n = 4845 # % 4058 83.8 936 19.3 3465 71.5 4580 94.5 439 9.1 73 1.5 303 6.3 208 4.3 Service Closure and Service Duration During SFY 2009, 3,183 client cases (representing 66% of all clients served) were closed20 by either the DES provider and/or the RBHA provider. 20 Although 3,183 clients were closed in SFY 2009, only 1,662 cases had usable data to calculate a client’s length of stay in the program. Center for Applied Behavioral Health Policy y Arizona State University 17 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 12 Summary of AFF Case Closures Length of stay (LOS) is computed by counting the number of calendar days from the date of a client assessment to the date of case closure. LOS has become an increasingly important indicator of treatment success and correlates with long term sobriety.21 Overall, clients receiving services only from DES providers spent nearly five months in the AFF program (mean = 143.0 days), compared with 11 months (mean = 330.4 days) for those receiving services exclusively through the RBHA providers. Not surprisingly, continuing clients experienced longer LOS (mean = 357.0 days) compared to new clients in 2009 (mean = 111.4 days). 3.7 Summary of Interviews with AFF Provider Agency Management and Program Supervisors Semi-structured interviews involving a total of 12 AFF provider agency management personnel and program supervisors (respondents), representing all 9 AFF provider agencies were conducted during the month of September 2009. These audio taped telephone interviews sought to solicit information from AFF provider agency management personnel and program supervisors regarding their perceptions of the relative strengths, areas for improvement, and impacts that the recent budget crises and corresponding funding reductions have had upon the availability, timeliness, and accessibility of substance abuse treatment services. As in previous years, these individuals were asked to identify barriers and compensatory strategies their agencies have experienced with regard to providing timely outreach and 21 United Nations-Office on Drugs and Crime. (2002). Contemporary Drug Abuse Treatment: A Review of the Evidence Base (Electronic Version) Retrieved from www.unodc.org/pdf/report_2002-11-30_1.pdf Center for Applied Behavioral Health Policy y Arizona State University 18 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report engagement, completion of client assessments, treatment plans, and client treatment engagement. Additionally, these individuals were questioned about unmet client treatment needs, changes in other services in their communities, and the quality of communication and collaboration with CPS case workers, the RBHAs, and other agencies in their communities. Finally, information was solicited regarding distinguishing program characteristics, areas of weakness and improvement, training needs, and recommendations to DES. These telephone surveys involved informed consent, following a protocol approved by the Arizona State University’s Institutional Review Board. The comments summarized below are frequently specific to individual providers, or specific DES districts and do not necessarily reflect upon all providers and/or Districts. Strengths and Qualities of AFF Programs across Arizona Each of the individuals interviewed identified particular strengths and unique aspects of their programs locally. Exhibit 13 highlights the more commonly identified and/or unique strengths of their programs. Exhibit 13 Identified Program Strengths • Community Resources and Community • Partnership with Drug Court Collaboration in the Face of Resource Reductions • Program Staff • Home‐based services, sober living facilities, and thrift store • Program motivation to keep clients engaged • Smaller caseloads Of particular note was the consistency with which respondents identified the collaboration and resource sharing among and between community based agencies. Particularly in light of the budget reductions, the degree of collaboration noted was significant and spoke to the shared sense of camaraderie in the face of tribulation that many community based organizations are experiencing during this budget crisis. Weaknesses and Areas of Improvement of the AFF Programs These individuals identified a variety of areas for improvement and strengthening of their local program. Ranging from data base enhancements to more consistent procedures for closing cases, these noted areas for program improvement tended to focus on operational and infrastructure needs along with clinical enhancements. Training and information for staff on substance abuse and addictive disorders, methods for detecting and treating prescription drug abuse, and evidence-based substance abuse family-treatment models (such as Community Reinforcement & Family Therapy, Brief Strategic Family Therapy, and Multi-Dimensional Family Therapy) were also identified as areas for program improvement and program strengthening. Center for Applied Behavioral Health Policy y Arizona State University 19 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report • • • • Exhibit 14 Identified Program Weaknesses/Areas for Growth Improve communications between CPS [case • Data base enhancements and better billing, workers] and DBHS [RBHA provider staff], documentation, and reporting procedures providing clarity of roles and responsibilities Strengthening case closure procedures • Staff training with specific attention to intergenerational substance use, prescription drug abuse, and evidence‐based family treatment models Life skills/living skills training • More employment support services Incorporating evidence‐based family treatment • Broader base of peer support specialist services strategies and family support groups While the issue of improved communications with CPS case workers and DBHS/RBHA provider staff was identified as a program need, it is important to note that all respondents indicated that communication with CPS staff has improved significantly this year. Respondents noted that these improved communications occurred in spite of staff cutbacks within CPS and in a few instances, office closures. Likewise, most respondents spoke positively with regard to their relationships with their RBHAs, although not with the same overwhelmingly positive tone. One respondent noted that the relationship with the RBHA in their community has gotten worse, noting added bureaucratic and paperwork demands from the RBHA. Significant Program Changes in Treatment Services and Community Collaborative Partnerships The individuals interviewed identified a variety of resources or services that have been reduced during the final four months of the fiscal year. The most significant change cited by all respondents was the cut in program funds and the subsequent measures taken to stabilize their programs. In addition to identification of reductions in a wide range of supports and supportive services, many of those interviewed noted the reduction of transportation assistance (bus vouchers, cab fare, etc.) for their clients and higher CPS caseloads and/or reduced responsiveness from CPS case managers due to budgetary cutbacks at CPS. Exhibit 15 Service Impacts Resulting from Budget Cutbacks • Loss of transportation assistance cited repeatedly • Reductions in CPS staff and CPS office consolidations • Reductions in services in general; specific examples including food boxes, parenting classes, parent aides for supervised visitation, utility assistance, etc. Center for Applied Behavioral Health Policy y Arizona State University 20 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Barriers and Improvements Respondents were asked to discuss specific barriers and strategies they had found to be effective in resolving these barriers in three key performance areas of outreach, assessment and treatment planning, and engagement in treatment services. Exhibit 16 Barriers and Improvements in Key Performance Areas Outreach Barriers • Delays in receiving client information from the RBHA when clients transfer Assessment and Treatment Plan Barriers • Limited certified staff or loss of certified staff • Clients not coming back to complete assessments they started • The lack of transportation, especially in the rural communities, but now coupled with the downturn in the economy has reduced the availability of lower skilled positions, typically taken by AFF clients. As transportation resources have dried up and unemployment has risen, these families find themselves without the means to get to AFF services, nor the means to pay for vehicle fuel and/or cab/bus fare Engagement in Treatment Services Barriers • CPS closing cases “…when CPS is not involved, the families sometimes won’t stay engaged because they no longer have CPS to motivate them.” • Lack of Flexibility from DES contract requirement: “There is no flexibility for individuals who are no longer in need of high intensity services. Even though a client is progressing along with treatment and they don’t need all three hours of service each week, the only way we get paid is if the client received three hours of service per week. It is both an unfair burden for the agency and for the client. The lack of flexibility of the requirements for three hours of treatment services is a barrier for clients who are [fully engaged] in treatment.” Outreach Improvement Strategies • Attend TDMs, CFTs and court hearings whenever possible in order to make contact and engage clients • Continue to engage clients while paperwork is catching up with them and/or while they are in jail • Making repeated visits to clients’ residence to connect with them in person to get them to engage in services Center for Applied Behavioral Health Policy y Arizona State University 21 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Engagement in Treatment Services Improvement Strategies • Adaptation of the Recovery Team Treatment Model • In order to keep clients engaged when there is limited IOP and SOP group capacity due to budget reductions, one DES provider is enrolling clients in education groups: “We will get clients into education groups to engage them. They are learning things in a non‐threatening environment. Clients have stated that the education group has helped to prepare them for treatment and has helped them to understand the impact of substance abuse in the family.” Meeting Client Needs The provider agency management personnel and program supervisors indicated that treatment specific services, including inpatient and outpatient treatment services continue to be readily available without too much difficulty. Supportive services, however, such as employment support services, housing, and transportation are becoming much more problematic to access. Respondents stated that while residential treatment is available, new barriers and challenges have emerged most notably related to the cost and obtaining authorization for payment. Clients with acute or unstable medical problems may not be admitted because of the guidelines of residential care facilities. In the accompanying table are direct quotes from program coordinators regarding AFF client needs. Exhibit 17 Provider Agency Management Personnel and Program Supervisor Comments “We have clients who are doing very well in treatment and they just can’t get a stable home or they can’t find stable employment, and that is what is holding them back. They are stressed, they get triggered and they might relapse. We refer them to county and city housing, but the lists are so long. As far as employment, we print out lists of job openings almost weekly and provide them to our clients in our front lobby. We help them fill out applications – anything to help them get their foot in the door. If a client wants us to help them with resume preparation, we definitely will help them.” District IV, Yuma “The lack of employment opportunities, the lack of affordable housing, and the lack of childcare needed for the time period that they are engaged in services and treatment activities are real issues and barriers to treatment. Our clients are having issues with keeping their phones in service.” District III, Flagstaff “We don’t get a lot of requests or demands; however, we help them however we can and we are flexible. We have, for the most part, eliminated support services from our menu of services we offer clients because of our budget.” District II, CPSA “There is no domestic violence shelter in Apache County. If a client is in crisis, I have no place to put them.” District III, Winslow “The biggest gap really comes down to vocational training‐whether it is pre‐vocational training, job coaching, or job placement.” District VI, SEABHS “Clients need assistance with utilities and other essentials, like food boxes and household supplies.” District IV, WestCare Center for Applied Behavioral Health Policy y Arizona State University 22 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report “They need transportation. Employment is very difficult, if not impossible, if there is no public transportation. How can you keep a job if you have to walk five miles in the heat of the desert to a job? There were people living in their cars this summer.” District V, Horizon “Clients desire to have better communication with CPS. At times they have difficulty making contact with the CPS worker. Sometimes the process is not explained to them in a way they can understand, especially if the parents have any type of developmental issue or learning problem which makes it difficult for them to understand. We try to explain the process to them in a way they can understand.” District III, Prescott CHAPTER 4 AFF PROGRAM OUTCOMES This chapter highlights the outcomes experienced by families that have participated in the AFF program. Outcome information is presented on the following key dimensions articulated in the enabling legislation establishing the AFF program: child safety, family stability and permanency, self-sufficiency as reflected in employment, and recovery from alcohol and drug problems. 4.1 Child Safety: Recurrence of Child Maltreatment Of the total of 4,845 clients in the AFF program, 4,398 (91%) had at least one report of suspected child maltreatment prior to entering AFF22. Among the 4,398 clients with a report at intake, 1,107 (23%) clients had reports that had been substantiated23; 3,233 (67%) clients had reports that were unsubstantiated, while 58 (1%) clients had reports whose status was pending. During the reporting period, 75% of substantiated cases consisted of neglect, 7% were physical abuse, and 1% sexual abuse. Similarly, last year the vast majority of substantiated maltreatment cases were also for neglect (84%), and the remainder for physical (12%) or sexual abuse (3%). These findings are consistent with other studies that showed substance abusing caregivers tend to be linked with neglect referrals rather than with sexual or physical abuse referrals.24 22 While there were no reports received from DES for the remaining 447 clients, this is due probably to data matching issues, such as incomplete SSN, CHILDS person ID, etc. 23 A substantiated finding is one in which the facts of a report provide a reasonable ground, i.e., some credible evidence, to believe that abuse or neglect occurred (Arizona Department of Economic Security, Division of Children, Youth and Families. Children’s Services Manual. Retrieved from www.azdes.gov/dcyf/cmdps /cps/Policy/ServiceManual.htm on February 3, 2009). 24 Sun, A., Shillington, A.M., Hohman, M., & Jones, L. (2001). Caregiver AOD Use, Case Substantiation, and AOD Treatment: Studies Based on Two Southwestern Counties. Child Welfare, 80(2), 151-177. Center for Applied Behavioral Health Policy y Arizona State University 23 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 18 Summary of Pre and Post Assessment Report Findings Pre‐Assessment Finding Post Assessment Finding Totals Substantiated Unsubstantiated Pending/Other # %2 # %2 # %1 # %2 Substantiated 1107 22.9% 5 0.5% 24 2.2% 0 0.0% Unsubstantiated 3233 66.7% 19 0.6% 99 3.1% 3 0.1% Pending/Other 58 1.2% 0 0.0% 3 5.2% 1 1.7% No Report21 447 9.2% 1 0.2% 17 3.8% 0 0.0% Total 4845 100% 25 0.5% 143 3.0% 4 0.1% 1 No Report # %2 1078 97.4% 3112 96.3% 54 93.1% 429 96.0% 4673 96.4% Percentages total vertically; 2Percentages total horizontally Exhibit 18 also provides information on child maltreatment recurrence statewide. Of the families with a report (substantiated or unsubstantiated) at pre-assessment, only 7% had a recurrence. Using the more conservative definition of recurrence used by NCANDS (subsequent substantiated reports following an initial substantiated report), a recurrence rate of 0.5%, lower than last year’s 2.4%, was observed. For informational purposes, the federal standard for absence of maltreatment recurrence within six months is 94.6% (allowing, therefore, recurrence of 5.4%). Thus, for SFY 2009 among AFF families, recurrence was significantly lower (better) than this national standard.25 4.2 Permanency Achieved by Children of Parents in AFF A total of 2,443 children whose parents (1,268) were AFF clients in SFY 2009 were in CPS outof-home placement at some point during the reporting period. As depicted in Exhibit 19, 60% (1,464) of these children were still in out of home placements at the end of the reporting period.26 By comparison, in SFY 2008, 54% of children of parents in AFF were still in care at year’s end. Exhibit 19 Permanency Achieved by Children of Parents in AFF Total Children 2443 100.0% Still in Care 1464 59.9% Other 31 1.3% Achieved Permanency 948 38.8% Reunification Guardianship Adoption Relatives Emancipated 1 794 64 27 39 24 83.8%1 6.8%1 2.8%1 4.1%1 2.5%1 Percentage is based on those achieving permanency 25 U.S. Department of Health and Human Services, Administration on Children, Youth and Families. Child Maltreatment 2006 (Washington, D.C.: U.S. Government Printing Office, 2008). 26 Included in this group are children who are participating in trial visits with relatives, guardians, or potential adoptive families. Center for Applied Behavioral Health Policy y Arizona State University 24 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report More than one-third of the total number of children in care at any point during the year (39%) achieved permanency during SFY 2009. Of those who were discharged from care and achieved permanency, the vast majority (84%) were reunified with their families, up slightly from 83% in SFY 2008. Others found permanent homes with relatives (n=39, 4%), through adoption (n=27, 3%), emancipation (n=24, 3%) or guardianship (n=64, 7%). Among the children who achieved permanency the median number of days27 in out-of-home care for children subsequently living with relatives was 11 days, followed by 153 days for children reunified with birth families, 354 days for children where guardianship was arranged, and 776 for children who were adopted. Exhibit 20 Days in Out of Home Care Among Children Achieving Permanency N = 948 n Median Average Reunification 794 153 216.2 Guardianship 64 354 374.6 Relatives 39 11 76.1 Adoption 27 776 835.2 Emancipated 24 560.5 654.4 4.3 Recovery from Substance Abuse Drug Screens. Drug screen results were analyzed only for those clients whose drug screens were entered directly into the AFF web-based data entry portal or uploaded by the DES provider into the data portal. Clients who were also enrolled with a RBHA provider may have been drug tested as well; however, the data from DBHS is less complete and does not provide the result of the drug screen. Consequently, this section of the report only discusses the results of the drug screen data available from the AFF web-based data entry portal. Drug screen results were reported in the AFF web-based data entry portal for 2,492 clients, more than half (51.4%) of all AFF clients, and slightly higher than the percentage reported last year (49%). Among clients whose drug screens were reported in the AFF web-based data entry portal, the average number of screens reported was 5 drug screens in SFY 2009 (higher than the 4.6 drug screens in SFY 2008), while more than half of the clients had reports of three or more drug screenings. The maximum number of drug screens reported for any client for the year was70. 27 The mid-point wherein half the children spent less time in care and half spent more time in care. Center for Applied Behavioral Health Policy y Arizona State University 25 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Exhibit 21 Statewide Summary of AFF Client Drug Screens Reported in AFF Web Portal Avg drug screens n % per month (SD) Total Clients 4845 100% Drug Screens Reported 2492 51.4% 1.24 (1.68) in Web Portal No Drug Screens 2353 48.6% Reported in Web Portal DES providers are required by contract to conduct or collect a minimum of two (2) drug screenings per client per month. Among clients with screens reported through the AFF webbased data entry portal, the average was 1.68 screens per 30 days (less than the 2.09 drug screens per month observed in SFY 2008). For all drug screens that were reported through the AFF web-based portal, 83% were negative, reflecting no drug use. This rate is down from the 90% rate of negative drug screens reported in SFY 2008. Among those clients that were tested in SFY 2009 and reported through the AFF web-based data entry portal, 67% had all negative drug screens indicating no drug use, while the remaining 33% with drug screens had at least one positive drug screen. Self-Report. Previous annual reports contained information on self-report substance use at completion of services. These data reflect only a partial sample of all clients, i.e., those DBHS clients exiting services from the local RBHA. Issues about the integrity of the self-report data at closure were raised in last year’s annual report. Since that time, the Arizona Office of the Auditor General identified a variety of problems in the ADHS reporting system28 that calls into question the usefulness and integrity of the self-report substance use data at program closure. Consequently, we have removed those data and discussion from this year’s report. 4.4 Employment Outcomes For Jobs-Referred AFF Clients As stated in the enabling legislation for the AFF program, services are provided to recipients of Temporary Assistance for Needy Families (TANF) whose substance abuse is a significant barrier to maintaining or obtaining employment. These individuals are referred to the DES providers through the Department’s Jobs program. Five individuals referred by the Jobs program were AFF clients during 2009 (three new clients in 2009 and two continuing clients from 2008). Among these five clients, all were unemployed at the time of their intake assessment. Three clients exited the program in 2009 and one of these clients was reported as employed at the time of their closure. 28 Arizona Office of the Auditor General (July 2009). Report No 09-07: Performance Audit, Department of Health Services, Division of Behavioral Health Services – Substance Abuse Treatment Programs. Phoenix: Author. Center for Applied Behavioral Health Policy y Arizona State University 26 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report CHAPTER 5 SUMMARY AND CONCLUSIONS This report summarizes the key processes and outcomes of the Arizona Families F.I.R.S.T. program, now in its eighth year of operation. The continued commitment of the legislature to critically examine the processes and outcomes of this highly innovative program has afforded the opportunity to study the development and operations of a program unique in its scope and focus. The utilization of information gathered from a variety of sources, including administrative data, focus groups, key informant interviews, and service utilization records provide diverse perspectives to address the performance of the AFF program in relation to each of the five goals articulated by the legislature. Increases in Timeliness, Availability and Accessibility of Services In SFY 2009, 4,845 individuals were served by the AFF program statewide. These individuals and their families continue to be characterized by mothers, slightly more than half of whom reported they were single and had never been married. Slightly more than one fourth of these clients identify themselves to be Latino. Nearly three quarters of the clients were unemployed and slightly less than half report their highest educational level to be a high school diploma or equivalent. All but five of these families were known to CPS. Nearly 85% of these individuals self-reported at the time of their assessment that they abused alcohol or other illicit substances, with alcohol, marijuana, and methamphetamine continuing to be the more commonly reported substances. For these families, the AFF program continues to provide services in a manner consistent with which the program was designed. During SFY 2009, these families received outreach, assessment, and treatment engagement services in a timely manner; most families were contacted by a local DES provider within two days of their referral with these same families accepting AFF services in less than two weeks of their referral. Those families that are engaged in treatment services typically find themselves receiving services from their local DES provider and/or a RBHA contracted treatment provider in their community, depending upon their program eligibility. This year witnessed a sharp increase in the rate at which these families were referred to a RBHA, up nearly 80% relative to SFY 2008. The referrals to, and joint service delivery with, the RBHAs reflect the integrated and complementary nature of the AFF program design. Interviews conducted with the DES provider agency management personnel and program supervisors indicate the high degree of inter-agency collaboration that occurs locally among and between DES providers, the RBHAs, and other state and local agencies. Notably this year, DES provider agency management personnel and program supervisors reported on significantly improved communications and coordination with CPS field offices and CPS program managers. As such, the AFF program continues to exhibit a pattern of early and timely engagement into substance abuse treatment, which has been a hallmark of this program. Center for Applied Behavioral Health Policy y Arizona State University 27 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Recovery from Alcohol and Drug Problems Data collected this year are consistent with that reported in past years: the overwhelming majority of parents entering the AFF program self-report patterns of substance use with alcohol, marijuana, and methamphetamine predominating. The results of drug screens or urine testing conducted with AFF clients to detect continued drug use and reported through the AFF webbased data entry portal indicate that two thirds (67%) of the AFF clients tested clean (no detection of continued substance use) on all occasions that they were tested, a rate consistent with that observed in past years. Among the rest of these clients, the patterns of drug testing results are uneven, suggesting periodic relapse among these individuals as they proceed on a path toward recovery and sobriety. While these results are encouraging, it should be noted that drug screen results submitted through the AFF web-based data entry portal were available for only 51% of the participating clients, up from the 49% of clients that were reported tested in SFY 2008. Child Safety and Reduction of Child Abuse and Neglect Among those families served in the AFF program, the rates of recurrence of child maltreatment following AFF program enrollment continues, as in past years, to be exceedingly low. Nearly all families (91%) served by the AFF program had a substantiated or unsubstantiated report of child maltreatment before enrolling in the program and nearly all (97%) had no subsequent report filed during this reporting period. Among those families with a substantiated report at intake, only .5% and 2.2% had a subsequent filing of a substantiated or unsubstantiated report, respectively. As such, these data suggest that the AFF program continues its tradition of enhancing child safety and avoiding the recurrence in child maltreatment as a result of parental substance abuse treatment. Permanency for Children Through Reunification An increase in the rate of children still in out of home care at the end of this reporting period was evidenced for 60% of all children placed into out of home care, as compared to 54% of all out of home placed children as reported for SFY 2008. This rate of continuing out of home placement is still well below the 75% rate that was observed in SFY 2007. Concomitant with the increase in continuing out of home placement was a corresponding decrease in the proportion of youth achieving permanency, observed among 39% of AFF client children and down from the 45% observed in SFY 2008. This drop, like the corresponding increase in continuing out of home placement is relatively small and statistically insignificant. For those youth that did achieve permanency, the overwhelming majority (84%) were reunified with their family, up slightly from the 83% rate observed in SFY 2008. As such, these data suggest that the families and children involved in the AFF program continue to experience rates of permanency planning and reunification that meet or exceed existing program performance patterns and/or national or state trends. Center for Applied Behavioral Health Policy y Arizona State University 28 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Achievement of Self-Sufficiency through Employment Employment activity is reported only for those clients referred to the AFF program who were already enrolled in the Jobs program. Among those five clients who were referred to the AFF program and completed AFF services this year (n=3), one client was reported employed at the time of AFF program completion. As such, due to the low number of AFF clients referred by the Jobs program, no determination can be made regarding the AFF program’s performance on this goal. Center for Applied Behavioral Health Policy y Arizona State University 29 Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report APPENDIX A DES Districts Services Treatment Services (A) Family Counseling Individual Counseling Group Counseling Assess., Eval. & and Screening Other Treatment by Profess Intensive Outpatient Services Outpatient Services Rehabilitation Services (B) Skills Training and Development BH Prevention/Promotion Psycho educational Medical Services (C) Medication Laboratory Medical Management Pharmacy Support Services (D) Case Management Personal Care Home Care Training Family Self-Help/Peer Unskilled Respite Care Supported Housing Sign Language Flex Fund Transportation Child Care After Care Other Support I # 1591 7 1488 0 1231 0 0 0 197 113 39 114 606 74 138 495 485 1851 1816 23 42 184 0 0 0 0 751 0 0 0 Source: DBHS Encounter Data II % # 690 0 493 1 579 0 0 0 274 126 0.4% 93.5% 0.0% 77.4% 0.0% 0.0% 0.0% 57.4% 19.8% 57.9% 12.2% 22.8% 81.7% 80.0% 98.1% 1.2% 2.3% 9.9% 0.0% 0.0% 0.0% 0.0% 40.6% 0.0% 0.0% 0.0% 196 28 408 47 106 324 343 809 778 16 4 252 1 0 0 0 197 0 0 0 III % 0.0% 71.4% 0.1% 83.9% 0.0% 0.0% 0.0% 46.0% 71.5% 10.2% 11.5% 26.0% 79.4% 84.1% 96.2% 2.0% 0.5% 31.1% 0.1% 0.0% 0.0% 0.0% 24.4% 0.0% 0.0% 0.0% # 254 0 201 1 179 0 0 0 100 69 51 6 157 5 20 130 130 341 339 11 5 15 0 0 0 0 144 0 0 0 IV % 0.0% 79.1% 0.4% 70.5% 0.0% 0.0% 0.0% 69.0% 51.0% 6.0% 3.2% 12.7% 82.8% 82.8% 99.4% 3.2% 1.5% 4.4% 0.0% 0.0% 0.0% 0.0% 42.2% 0.0% 0.0% 0.0% Center for Applied Behavioral Health Policy y Arizona State University # 139 0 79 0 110 0 0 0 53 48 11 5 64 4 23 53 51 175 171 3 7 31 1 0 0 0 51 0 0 0 V % 0.0% 56.8% 0.0% 79.1% 0.0% 0.0% 0.0% 90.6% 20.8% 9.4% 6.3% 35.9% 82.8% 79.7% 97.7% 1.7% 4.0% 17.7% 0.6% 0.0% 0.0% 0.0% 29.1% 0.0% 0.0% 0.0% # 107 0 100 0 79 0 0 0 38 18 23 12 61 2 19 42 46 139 137 8 3 16 2 0 0 0 65 0 0 0 30 VI % 0.0% 93.5% 0.0% 73.8% 0.0% 0.0% 0.0% 47.4% 60.5% 31.6% 3.3% 6 31.1% 68.9% 75.4% 98.6% 5.8% 2.2% 11.5% 1.4% 0.0% 0.0% 0.0% 46.8% 0.0% 0.0% 0.0% # 161 0 134 0 132 0 0 0 100 83 39 82 0 9 60 65 186 184 5 12 63 3 0 0 0 91 0 0 0 Statewide % 0.0% 83.2% 0.0% 82.0% 0.0% 0.0% 0.0% 83.0% 39.0% 6.0% 0.0% 11.0% 73.2% 79.3% 98.9% 2.7% 6.5% 33.9% 1.6% 0.0% 0.0% 0.0% 48.9% 0.0% 0.0% 0.0% # 2942 7 2495 2 2310 0 0 0 762 457 359 171 1378 132 315 1104 1120 3501 3425 66 73 561 7 0 0 0 1299 0 0 0 % 0.2% 84.8% 0.1% 78.5% 0.0% 0.0% 0.0% 60.0% 47.1% 22.4% 9.6% 22.9% 80.1% 81.3% 97.8% 1.9% 2.1% 16.0% 0.2% 0.0% 0.0% 0.0% 37.1% 0.0% 0.0% 0.0% Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Source: DBHS Encounter Data DES Districts Services Crisis Intervention Services (E) Crisis Crisis Mobile Crisis Stabilization Inpatient Services (F) Inpatient Professional Residential Services (G) Residential Level II Residential Level III Child Residential w/Parent Behavioral Health Day Programs (H) Supervised BH Treatment Therapeutic BH Services Community Psych. I # 235 1 111 177 22 22 128 128 0 0 177 7 171 0 II % 0.4% 47.2% 75.3% 100.0% 100.0% 0.0% 0.0% # 132 0 10 126 22 22 102 102 0 0 18 4.0% 96.6% 0.0%0 18 0 III % 0.0% 7.6% 95.5% 100.0% 100.0% 0.0% 0.0% 0 0.0% 100.0% 0.0% # 15 0 11 4 14 14 23 23 0 11 0 11 0 IV % # 13 0.0% 73.3% 26.7%0 11 100.0% 100.0% 0.0% 0.0% 0 0.0% 100.0% 0.0% 4 2 2 6 6 0 0 0 0 V % 0.0% 84.6% 30.8% 100.0% 100.0% 0.0% 0.0% # 16 0 14 5 3 3 6 6 0 0 % 0.0% 87.5% 31.3% 100.0%5 100.0% 0.0% 0.0% 0 0.0% 0.0% 0.0%0 0 0 0.0% 0.0% 0.0% 0 1 1 0 Center for Applied Behavioral Health Policy y Arizona State University VI 31 # 28 0 24 10 10 18 18 0 2 0 Statewide % 100.0% 0.0% 0.0% # 439 1 181 321 73 73 283 283 0 0 100.0% 0.0% 0.0% 0.0% 50.0% 50.0% 208 7 201 1 3.4% 96.6% 0.5% 0.0% 85.7% 17.9% 100.0% % 0.2% 41.2% 73.1% 100.0% Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Source: Services In AFF Webportal DES Districts Services Treatment Services (A) Family Counseling Individual Counseling Group Counseling Assess., Eval. & and Screening Other Treatment by Profess Intensive Outpatient Services Outpatient Services Rehabilitation Services (B) Skills Training and Development BH Prevention/Promotion Psycho educational Medical Services (C) Medication Laboratory Medical Management Pharmacy Support Services (D) Case Management Personal Care Home Care Training Family Self-Help/Peer Unskilled Respite Care Supported Housing Sign Language Flex Fund Transportation Child Care After Care Other Support I # 1969 13 1647 1035 1675 0 175 656 132 5 127 0 1908 0 1897 0 87 2331 2328 0 0 0 0 0 0 0 7 0 81 182 II % 0.7% 83.6% 52.6% 85.1% 0.0% 8.9% 33.3%0 3.8% 96.2% 0.0% # 136 0 0 0 135 0 68 0 0 0 0 109 0.0% 0 99.4% 109 0.0% 0 4.6% 0 60 99.9% 60 0.0% 0 0.0% 0 0.0% 0.0% 0 0.0%0 0 0.0% 0.0% 0 0.3%0 0 0.0% 0 3.5% 0 7.8% 0 III % 0.0% 0.0% 0.0% 99.3% 0.0% 0.0% 50.0%0 0.0% 0.0% 0.0% # 120 3 0 0 106 21 26 16 6 11 0 189 0 0.0% 100.0% 0.0% 189 0 0.0% 0 186 100.0% 163 0.0% 0 0.0% 3 0 0.0% 0.0% 0 0.0% 16 0.0% 0 0.0% 0.0% 37 0.0%6 1 0.0% 28 0.0% 57 Center for Applied Behavioral Health Policy y Arizona State University IV % # 139 0 2.5% 0.0% 0.0% 6 88.3%7 117 0.0% 17.5% 21.7%0 34 0 44 37.5% 2 68.8% 42 0.0% 170 0.0%0 0 100.0% 0.0% 170 0 0.0% 0 155 87.6% 0.0% 0 95 1.6% 0.0% 0 0.0%14 0 8.6% 0.0% 0 3.2%2 0 19.9% 0.5% 6 15.1%3 30.6% 128 28 V % 0.0% 5.0% 4.3% 84.2% 0.0% 0.0% 24.5% 0 4.5% 95.5% 0.0%1 # 40 8 12 16 32 0 6 6 5 0 49 0 0.0% 100.0% 0.0% 49 0 0 0.0% 13 61.3% 0 0.0% 9 9.0% 0 0.0% 0.0%0 0 1.3% 0 0.0% 0 0.0% 0 1.9% 3.9% 0 18.1%2 82.6% 2 1 32 VI % # 95 20.0% 30.0% 0 40.0% 0 80.0% 0 0.0% 86 0.0% 15.0%0 25 0 3 16.7% 83.3% 0 0.0% 3 86 0.0%0 100.0% 0.0%086 0.0% 3 79 2 69.2% 22 0.0% 0.0% 0.0%0 0.0%0 0.0%0 0.0%0 0.0%0 15.4%0 0 0.0%0 7.7% 12 15.4%0 67 Statewide % # 2499 24 1666 1057 2151 0 0.0% 0.0% 0.0% 90.5% 0.0% 0.0% 26.3% 196 201 815 0.0% 14 100.0% 188 0.0% 0 2511 0.0% 0 100.0% 2500 3.5% 3 2.3% 89 2824 27.8% 2677 0.0% 0 0.0% 17 0.0% 0 0.0% 0 0.0% 18 0.0% 0 0.0% 6 0.0% 49 0.0% 7 15.2% 150 84.8% 436 % 1.0% 66.7% 42.3% 86.1% 0.0% 7.8% 32.6% 7.0% 93.5% 0.0% 0.0% 99.6% 0.1% 3.5% 94.8% 0.0% 0.6% 0.0% 0.0% 0.6% 0.0% 0.2% 1.7% 0.2% 5.3% 15.4% Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Source: Services In AFF Webportal DES Districts Services Crisis Intervention Services (E) Crisis Crisis Mobile Crisis Stabilization Inpatient Services (F) Inpatient Professional Residential Services (G) Residential Level II Residential Level III Child Residential w/Parent Behavioral Health Day Programs (H) Supervised BH Treatment Therapeutic BH Services Community Psych. I # 1 0 0 1 0 0 21 21 0 0 0 0 0 0 II % 0.0% 0.0% 100.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 0 0 0 0 0 0 7 7 0 0 0 0 0 0 III % 0.0% 0.0% 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 0 0 0 0 0 0 2 2 0 0 0 0 0 0 Center for Applied Behavioral Health Policy y Arizona State University IV % 0.0% 0.0% 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 0 0 0 0 0 0 0 0 0 0 0 0 0 0 V % 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 0 0 0 0 0 0 0 0 0 0 0 0 0 0 33 VI % 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 0 0 0 0 0 0 2 2 0 0 0 0 0 0 % 0.0% 0.0% 0.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 1 0 0 1 0 0 32 32 0 0 0 0 0 0 Statewide % 0.0% 0.0% 100.0% 0.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Source: Services In AFF Webportal and DBHS Encounter Data DES Districts Services Treatment Services (A) Family Counseling Individual Counseling Group Counseling Assess., Eval. & and Screening Other Treatment by Profess Intensive Outpatient Services Outpatient Services Rehabilitation Services (B) Skills Training and Development BH Prevention/Promotion Psycho educational Medical Services (C) Medication Laboratory Medical Management Pharmacy Support Services (D) Case Management Personal Care Home Care Training Family Self-Help/Peer Unskilled Respite Care Supported Housing Sign Language Flex Fund Transportation Child Care After Care Other Support I II # % # 2444 792 0 0.8% 20 2198 42.3%493 1 103589.9% 2086 85.4% 683 0.0% 0 0 7.2% 175 0 68 26.8% 656 325 274 115 35.4% 126 166 51.1% 196 114 35.1% 28 2235 508 47 74 2000 3.3% 22.1%212324 495 89.5% 22.9% 343 511 2775 840 2751 0.8% 810 16 23 99.1% 4 1.5% 42 6.6% 184 0.0%252 1 0 0 0.0% 0 0.0% 0 0 0.0% 0 27.3%0 197 757 0 0.0% 0 0 2.9% 81 182 6.6% 0 III % 0.0% 62.2% 0.1% 86.2% 0.0% 0.0% 0 8.6% 0.0% 0.0% 0.0% 9.3% 41.7% 63.8% 67.5% # 303 3 201 1 229 21 26 108 74 60 6 286 5 200 130 130 387 96.4% 1.9% 380 11 8 0.5% 30.0% 0.1% 15 0 16 0.0% 0 0.0% 6 0.0% 23.5% 166 1 0.0% 28 0.0% 0.0% 57 Center for Applied Behavioral Health Policy y Arizona State University IV % 1.0% 66.3% 0.3%0 75.6% 0.0% 6.9% 0 8.6% 68.5% 55.6% 5.6% # 206 84 6 173 0 34 86 49 51 5 205 1.7% 69.9% 182 45.5%4 45.5%53 51 238 98.2% 2.8% 219 3 21 2.1% 31 3.9% 1 0.0% 4.1% 0 0.0% 2 0 1.6% 53 42.9% 6 0.3% 28 7.2% 14.7% 128 V % 0.0% 40.8% 2.9% 84.0% 0.0% 0.0% 16.5% # 125 8 102 16 96 0 0 6 42 57.0% 59.3%18 27 5.8% 12 95 2.0% 2 88.8% 63 25.9% 42 24.9% 46 145 92.0% 142 8 1.3% 3 8.8% 16 13.0% 2 0.4% 0 0.8% 0.0% 0 0.0% 22.3%0 67 0 2.5% 1 11.8% 53.8% 34 2 VI % # 188 0 134 0 6.4% 81.6% 12.8% 76.8% 0.0% 159 0 0.0% 4.8% 0 101 25 42.9% 83 64.3% 41 28.6% 6 136 2.1% 0 66.3% 44.2%88 48.4%62 67 195 97.9% 190 5 5.5% 12 2.1% 11.0% 1.4% 63 3 0 0.0% 0 0.0% 0.0% 46.2% 0.0%091 0.7% 0 1.4%12 67 Statewide % 0.0% 71.3% 31 0.0% 84.6% 0.0% 0.0% 13.3% # 4058 3212 1059 3426 0 196 815 936 465 % 0.8% 79.2% 26.1% 84.4% 0.0% 4.8% 20.1% 82.2% 40.6% 5.9% 541 3465 171 0.0% 132 64.7% 2745 45.6% 1106 49.3% 1148 4580 97.4% 4492 2.6% 90 6.2% 66 32.3% 1.5% 561 7 0.0% 0 0.0% 18 6 0.0% 46.7% 1331 7 0.0% 6.2% 49.7% 57.8% 18.3% 98.1% 1.4% 2.0% 12.2% 0.2% 0.4% 0.0% 0.1% 29.1% 0.2% 3.3% 34.4%150 436 9.5% 3.8% 79.2% 31.9% 33.1% Arizona Families F.I.R.S.T. Program 2009 Annual Evaluation Report Source: Services In AFF Webportal and DBHS Encounter Data DES Districts Services Crisis Intervention Services (E) Crisis Crisis Mobile **Crisis Stabilization 31 Inpatient Services (F) Inpatient Professional Residential Services (G) Residential Level II Residential Level III **Child Residential w/Parent 32 Behavioral Health Day Programs (H) Supervised BH Treatment Therapeutic BH Services Community Psych. 31 32 I # 235 1 111 178 22 22 144 144 0 0 177 7 171 0 II % 0.4% 47.2% 75.7% 100.0% 100.0% 0.0% 0.0% 4.0% 96.6% 0.0% # 132 0 10 126 22 22 105 105 0 0 18 0 18 0 III % 0.0% 7.6% 95.5% 100.0% 100.0% 0.0% 0.0% 0.0% 100.0% 0.0% # 15 0 11 4 14 14 23 23 0 0 11 0 11 0 IV % 0.0% 73.3% 26.7% 100.0% 100.0% 0.0% 0.0% 0.0% 100.0% 0.0% # 13 0 11 4 2 2 6 6 0 0 0 0 0 0 V % 0.0% 84.6% 30.8% 100.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 16 0 14 5 3 3 6 6 0 0 0 0 0 0 Can include medical detoxification. Child can be placed with a parent in a Level II facility. Center for Applied Behavioral Health Policy y Arizona State University 35 VI % 0.0% 87.5% 31.3% 100.0% 100.0% 0.0% 0.0% 0.0% 0.0% 0.0% # 28 0 24 5 10 10 19 19 0 0 2 0 1 1 Statewide % 0.0% 85.7% 17.9% 100.0% 100.0% 0.0% 0.0% 0.0% 50.0% 50.0% # 439 1 181 322 73 73 303 303 0 0 208 7 201 1 % 0.2% 41.2% 73.3% 100.0% 100.0% 0.0% 0.0% 3.4% 96.6% 0.5%