2013 Annual Consumer Survey Report December 2013 Office of Performance Improvement Arizona Department of Health Services Division of Behavioral Health Services Bureau of Quality and Integration 2013 Consumer Survey Report TABLE OF CONTENTS EXECUTIVE SUMMARY…………………………………………………..………………………….4 INTRODUCTION……………………………………………………………...............................6 SURVEY SAMPLE AND RESPONSE RATES...……………………………………….………6 METHODOLOGY AND SURVEY ADMINISTRATION………………….…………………...7 DATA ANALYSIS………………….…………………………………………………….………8 STATEWIDE SURVEY DATA LIMITATIONS………………………………………………...9 ADULT SURVEY ANALYSIS AND RESULTS..........................................................................9 RBHA SPECIFIC ADULT INITIATIVES……………………………………………………...14 YSS-F SURVEY ANALYSIS AND RESULTS...………………………………………………15 RBHA SPECIFIC YSS-F INITIATIVES………………………………………………....……..20 STATEWIDE IMPROVEMENT OPPORTUNITIES…………………………………………..21 ADHS/DBHS INITIATIVES…………………………………………………….……...............21 CONCLUSION……………………………………………………………………….……...…..24 ATTACHMENTS Attachment A: 2013 Tribal Behavioral Health Authority Health Survey Results Attachment B: 2013 Statewide Consumer Survey Protocol Attachment C: 2013 Raw Data Attachment D: 2013 Adult Demographics Attachment E: 2013 Adult Line Item Attachment F: 2013 YSS-F Demographics Attachment G: 2013 YSS-F Line Item Attachment H: List of Acronyms 2 2013 Consumer Survey Report APPENDICES YEAR 2013 ADULT CONSUMER SURVEY (ENGLISH AND SPANISH) YEAR 2013 YOUTH CONSUMER SURVEY FOR FAMILIES (ENGLISH AND SPANISH) 3 2013 Consumer Survey Report I. Executive Summary The Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS), Tribal/Regional Behavioral Health Authorities (T/RBHAs) and contracted service providers jointly conduct statewide consumer satisfaction surveys each fiscal year. Each year, two surveys are administered based on the Substance Abuse and Mental Health Services Administration‟s (SAMSHA‟s) Mental Health Statistics Improvement Program (MHSIP) consumer surveys: The Adult Consumer Survey and The Youth Services Survey for Families (YSS-F). The surveys request independent feedback from Title XIX/XXI adults and families of youth receiving services through Arizona‟s publicly funded behavioral health system. The surveys measure consumers‟ perceptions of behavioral health services in relation to the following domains: General Satisfaction Access to Services Service Quality/Appropriateness Participation in Treatment Outcomes Cultural Sensitivity Improved Functioning Social Connectedness Within ADHS/DBHS, Consumer Survey results are:  Reviewed by the ADHS/DBHS Quality Management (QM) Committee;  Used by the QM Committee as a critical source of information to drive system improvements through RBHA contracts and QM Plans;  Incorporated into the Annual Performance Improvement Reports to provide a comprehensive update on performance across contractual performance measures  Used for certain Performance Incentive System calculations  Tracked and trended by Network development to determine network capacity by measuring member perception of behavioral health services in relation to the established survey domains.  Analyzed by the office of Information Management and the Office of Performance improvement to identify and improve on performance gabs in the various survey domains. Survey outcomes are also:  Disseminated to ADHS/DBHS stakeholders and consumers via the ADHS/DBHS website and Performance Dashboard;  Submitted to these national organizations as part of federal grant requirements for use in national reporting and evaluation: o The National Association of State Mental Health Program Directors‟ (NASMHPD) o National Research Institute (NRI) o Western States Decision Support Group (WSDSG) o Substance Abuse and Mental Health Services Administration‟s (SAMHSA) Mental Health Statistics Improvement Program (MHSIP) 4 2013 Consumer Survey Report The fiscal year 2013 survey was conducted from April through May 2013. Adult and Youth survey results are summarized as follows. Adult Consumer Survey results: Statewide: Service Accessibility, General satisfaction and Service Quality and Appropriateness domains showed a statistically significant change from 2012 results. Outcomes domain scores dropped by two percentage points. Improved Functioning and Social Connectedness domains showed slight increases in 2013 but represented no statistically significant changes from the 2012 survey results. RBHA Specific: Magellan (GSA 6) showed a statistically significant change in the General Satisfaction and the Service Quality domains. Cenpatico in GSA 2 showed a decrease in two domains: Participation in treatment planning by one percentage points and Social Connectedness by two percentage points CPSA (GSA 5) had a percentage drop in scores for the Outcomes, Improve Functioning and Social Connectedness domains YSS-F Consumer Survey results: Statewide: Service Accessibility showed a statistically significant decrease in 2013 when compared to the 2012 survey results (P<0.03). Improved Functioning domain decreased by four percentage point. Participation in Treatment, increased by one percentage points. RBHA Specific: Cenpatico in GSA 3 demonstrated a significant decrease of six percentage points in the improved Functioning domain from 2012 results. Cenpatico‟s GSA 4 demonstrated a significant twelve percent increase for the Outcomes domain with a statistically significant (P<0.005). Cenpatico‟s GSA 4 showed percentage increases in five of its six domains. CPSA (GSA 5) experienced a five percent increase in Cultural Sensitivity and Participation in Treatment planning domains. Regional Behavioral Health Authorities (RBHA) and Tribal Behavioral Health Authorities (TBHA) results from the 2013 Consumer Survey are presented separately to preserve the integrity of findings as two different sampling methodologies were used to get the survey sample. Refer to Section III-Survey Response Rates for details about the differences in the RBHA and TBHA surveys. 5 2013 Consumer Survey Report II. Introduction The Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS) and the Regional Behavioral Health Authorities (RBHAs) in collaboration with their contracted providers administered the statewide consumer survey during April and May of 2012. Within ADHS/DBHS, Consumer Survey results are:  Reviewed by the ADHS/DBHS Quality Management (QM) Committee;  Used by the QM Committee as a critical information source to drive system improvements through RBHA contracts and QM Plans;  Incorporated into the Annual Performance Improvement Reports to provide a comprehensive update on performance across contractual performance measures  Used for certain Performance Incentive System calculations.  Tracked and trended by the office of Network Development to determine network capacity by measuring member perception of behavioral health services in relation to the established survey domains and,  Analyzed by the office of Information Management and the Office of Performance improvement to identify and improve on performance gabs in the various survey domains.  Used by the Office of Information Management to conduct correlation analyses between survey domains and performance indicators. Survey outcomes are also:  Disseminated to ADHS/DBHS stakeholders and consumers via the ADHS/DBHS website and Performance Dashboard;  Submitted to these national organizations as part of federal grant requirements for use in national reporting and evaluation: o The National Association of State Mental Health Program Directors‟ (NASMHPD) o National Research Institute (NRI) o Western States Decision Support Group (WSDSG) o Substance Abuse and Mental Health Services Administration‟s (SAMHSA) Mental Health Statistics Improvement Program (MHSIP) III. Survey Sample and the Response Rates A total of 5,800 RBHA-enrolled consumers were included for the MHSIP Consumer Survey random sample pool; 4,001 consumers were contacted and asked to complete a survey. The response rate is calculated by dividing the number of surveys returned by the number of enrollees asked to participate. Statewide response rates based on surveys offered were 82% for adults and 82% for youth, as seen in TABLE-1. Please see Attachment B for a complete discussion of the survey protocol. The Tribal Behavioral Health Authorities (TBHA) scores are not included in the statewide results due to the TBHA convenience sampling methodology which solicits responses at the convenience of the recipients rather than from a random sample of TBHA enrolled recipients. Therefore the results are reported separately in Attachment A 2013Tribal Behavioral Health Authority Health Survey Results. 6 2013 Consumer Survey Report TABLE-1: Survey Response Rate Based on Surveys Offered for Each RBHA/GSA Consumer Survey 2013 Surveys Offered (a) RBHA/GSA GSA 1 - NARBHA GSA 2 - Cenpatico GSA 3 - Cenpatico GSA 4 - Cenpatico GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Overall 938 519 636 667 693 548 4,001 Surveys Completed (b) Adult Youth Overall 437 298 360 296 365 254 2,010 501 221 276 371 328 294 1,991 800 421 519 542 539 462 3,283 Adult 358 247 290 242 304 210 1,651 Youth 442 174 229 300 235 252 1,632 Response Rate (b/a) Overal l 85% 81% 82% 81% 78% 84% 82% Adult Youth 82% 83% 81% 82% 83% 83% 82% 88% 79% 83% 81% 72% 86% 82% IV. Methodology and Survey Administration Two survey populations (sampling frame) were identified for use in the Adult Consumer Survey and the YSS-F: Adults - Title XIX/XXI behavioral health recipients who are 18 years of age or older, and are enrolled in any of the adult programs: Serious Mental Illness (SMI), Substance Abuse or General Mental Health (Non-SMI). Youth - Title XIX/XXI behavioral health recipients under age 18 and enrolled in the Child/Adolescent program. A total of 43,263 adult and 34, 032 youth RBHA-enrolled Title XIX/XXI consumers were eligible to participate in the 2013 survey. Please refer to Attachment B (2013 Consumer Survey Protocol) for details on sampling frame development, inclusion/exclusion criteria, survey instruments, and survey administration guidelines. The ADHS/DBHS utilized a Likert Scale model to have respondents rate the domain questions of the Consumer Survey from strongly agree (numeric value of 5) to strongly disagree (numeric value of 1), and an option for Not Applicable (numeric value 0). Consumer perception was determined as positive for those domains where the question responses equaled 3.5 or above. TABLE-2 and TABLE-3 show the number and percentage of sample surveys that were not included in the domain score. This is presented by RBHA/GSA and domain. Surveys are excluded from a domain‟s score when more than one-third of the questions have a response that cannot be determined. This occurs most often when no response is marked. It also occurs when more than one response is marked. See Attachment B for a full discussion of the survey protocol. 7 2013 Consumer Survey Report TABLE-2: Adult Number and Percent of Surveys Excluded Due to Non-Responses by Domain and RBHA /GSA RBHA GSA 1 NARBHA GSA 2 Cenpatico GSA 3 Cenpatico GSA 4 Cenpatico GSA 5 - CPSA GSA 6 Magellan Statewide RBHA General Satisfacti on Service Access Participation in Treatment Planning Service Quality & Appropriatenes s Outcomes Improved Functionin g Social Connectedn ess 2 (0.6%) 3 (0.8%) 9 (2.5%) 8 (2.2%) 14 (3.9%) 8 (2.2%) 6 (1.7%) 3 (1.2%) 5 (2.0%) 16 (6.5%) 9 (3.6%) 15 (6.1%) 12 (4.9%) 3 (1.2%) 0 1 (0.3%) 7 (2.4%) 3 (1.0%) 13 (4.5%) 8 (2.8%) 9 (3.1%) 1 (0.4%) 1 (0.3%) 0 2 (0.7%) 4 (1.7%) 20 (6.6%) 3 (1.2%) 6 (2.0%) 3 (1.2%) 21 (6.9%) 5 (2.1%) 21 (6.9%) 8 (3.7%) 9 (1.2%) 27 (12.3%) 13 (5.9%) 5 (2.1%) 23 (7.6%) 23 (10.5%) 18 (8.2%) 19 (8.7%) 15 (0.9%) 20 (1.2%) 83 (5.0%) 93 (5.6%) 70 (4.2%) 63 (3.8%) 42 (2.5%) TABLE-3: Youth Number and Percent of Surveys Excluded Due to Non-Responses by Domain and RBHA/GSA RBHA GSA 1 NARBHA GSA 2 Cenpatico GSA 3 – Cenpatico GSA 4 – Cenpatico GSA 5 CPSA GSA 6 Magellan Statewide RBHA General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 4 (0.9%) 7 (1.6%) 9 (2.1%) 26 (6.0%) 3 (0.7%) 3 (0.7%) 7 (1.6%) 1 (0.6%) 2 (1.1%) 1 (0.6%) 6 (3.4%) 1 (0.6%) 1 (0.6%) 5 (2.9%) 1 (0.4%) 4 (1.7%) 3 (1.3%) 5 (2.2%) 4 (1.7%) 3 (1.3%) 2 (0.9%) 2 (0.7%) 3 (1.0%) 5 (1.7%) 9 (3.0%) 8 (2.7%) 7 (2.3%) 4 (1.3%) 3 (1.3%) 10 (4.3%) 2 (0.9%) 12 (5.1%) 8 (3.4%) 8 (3.4%) 9 (3.8%) 6 (2.3%) 13 (5.1%) 9 (3.5%) 20 (7.8%) 10 (3.9%) 9 (3.5%) 9 (3.5%) 17 (1.0%) 39 (2.4%) 29 (1.8%) 78 (4.8%) 34 (2.1%) 31 (1.9%) 36 (2.2%) V. Data Analysis ADHS/DBHS provided each RBHA with an Access database to compile survey responses. All completed surveys were entered and the database was submitted to ADHS/DBHS. Each RBHA analyzed its respective survey data using an SPSS script designed and provided by ADHS/DBHS to ensure consistency in data analysis. Statewide survey data is analyzed as follows: 8 2013 Consumer Survey Report By Domain; By Domain Line Item; Sub-group Analysis; and Comparison to Past Survey Performance. The statewide survey results are weighted to adjust for the different population sizes of each Geographic Service Area (GSA). The adjusted rate eliminates confounding due to variations in the population sizes of the GSAs sampled. Please refer to Attachment C, 2013 Raw Data, for further specifics of the weighting methodology and weights applied to each GSA. VI. Statewide Survey Data Limitations Two different sampling methodologies were used to obtain the samples for the 2013 Consumer Survey: one for the Regional Behavioral Health Authorities (RBHA) and another for the Tribal Behavioral Health Authorities (TBHA). The TBHA scores were not included in the statewide results due to the TBHA convenience sampling methodology which solicits responses at the convenience of the recipients and providers rather than from a random sample of TBHA enrolled recipients. VII. Adult Survey Analysis and Results Demographics A total of 1,651 completed adult surveys were analyzed. The breakdown of the respondents is done by gender, age, ethnicity, race, length of enrollment in services, length of enrollment and behavioral health category. Please see Attachment D, 2013 Adult Demographics for complete demographics information. Domain Line Item Analysis A specific set of questions was developed to gather scores for each domain. Line items are specific questions pertaining to each survey domain (Attachment E, 2013 Adult Line Item); analysis of the respondents‟ answers to a question under a domain, either positive or negative, affects the overall domain score. Table-4 demonstrates the results of the 2013 Adult Survey and compares them to previous surveys. The t-test was used to compare survey results for 2012 and 2013 and determine whether there were any statistically significant differences between the mean of the samples of each domain. A P-value of 0.05 with 95% confidence interval is used to determined statistical significance. A score is therefore considered to be of statistical significance when it is less than or equals to 0.05. The initials: “ns” (Not Significant) is used to denote changes in the results between the current and previous year that are not statistically significant. The statewide analyses are based on weighted scores. 9 2013 Consumer Survey Report TABLE-4 Statewide RBHA Adult Consumer Survey Results by Domain for 2007 – 2013 2008 % (Pvalue) 82 (ns) 73 (ns) 89 (0.01) 84 (ns) 68 (ns) 68 (ns) 68 (ns) 2007 % Domain General Satisfaction 86 Service Access 77 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 79 Improved Functioning 66 Social Connectedness 65 Color 88 74 2009 % (Pvalue) 84 (ns) 78 (ns) 82 (0.03) 89 (ns) 70 (ns) 68 (ns) 68 (ns) 2010 % (Pvalue) 86 (ns) 81 (ns) 92 (0.01) 90 (ns) 75 (ns) 71 (ns) 79 (0.02) 2011 % (Pvalue) 86 (ns) 84 (ns) 87 (ns) 88 (ns) 72 (ns) 69 (ns) 76 (ns) 2012 % (Pvalue) 85 (ns) 83 (ns) 90 (0.01) 87 (ns) 74 ns) 69 (ns) 76 (ns) 2013 % Pvalue) 90) (0.01 87 (0.06) 91 (ns) 91 (0.00) 72 (ns) 71 (ns) 77 (ns) Key statistically significant increase statistically significant decrease Figure 1-The 2013 Statewide Adult Survey Results: Rate Change Analyses Social Connectedness Improved Functioning Outcomes 2013 Service Quality & Appropriateness 2012 Participation in Treatment Planning Service Access General Satisfaction 0 20 40 60 80 100 General Satisfaction: Three questions were asked in the survey to ascertain General Satisfaction (Attachment E). The overall score for this domain is 90%, representing an average score based on three questions. This domain increased by five percent from the 2012 survey results (See fig 1). This also led to a statistically significant change. 10 2013 Consumer Survey Report Service Access: Six questions in the survey gathered information regarding Access to Services (Attachment E). The average score for this domain is 87%, indicating a four percent increase from 2012. (See fig 1), indicating a statistically significant change. Participation in Treatment Planning: Two survey questions were used to determine score for the Participation in Treatment Planning (Attachment E). The average score for this domain is 91%, indicating a slight one percent increase from the 2012 results (See fig 1). . Service Quality and Appropriateness: Nine questions in the survey focused on Service Quality and Appropriateness (Attachment E). The average score for this domain is 91%. This domain showed a four percent increase (See fig 1), which is of statistical significant. Outcomes: Eight questions were asked to obtain consumers‟ perception on Outcomes (Attachment E). The overall score for this domain is 72%, indicating a decreased by two percent from the 2012 results (See fig 1). . Improved Functioning: Five questions indicate respondents‟ impressions of Improved Functioning (Attachment E). The overall score for this domain is 71%, indicating a two percent increases in this domain. (See fig 1). Social Connectedness: Four questions were asked in the survey to ascertain Social Connectedness (Attachment E). This domain scored 77%, showing a one percent increase from the 2012 survey score. (See fig 1). RBHA and GSA specific Adult survey findings are presented in tables 5 through 10 (page 11 and 12); the Tables include percentages and P-values for each domain by RBHA and GSA ranging from 2007 through 2013. 11 2013 Consumer Survey Report TABLE-5: GSA 1 - NARBHA Adult Consumer Survey Results by Domain for 2007 - 20123 Domain 2007 % General Satisfaction 84 Service Access 73 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 78 Improved Functioning 63 -Social Connectedness 61 87 74 2008 % (Pvalue) 85 (ns) 80 (ns) 92 (0.00) 88 (ns) 71 (ns) 68 (ns) 65 (ns) 2009 % (Pvalue) 84 (ns) 79 (ns) 80 (0.00) 88 (ns) 64 (ns) 63 (ns) 68 (ns) 2010 % (Pvalue) 88 (ns) 83 (ns) 92 (0.00) 90 (ns) 72 (ns) 71 (ns) 78 (0.03) 2011 % (Pvalue) 90 (ns) 85 (ns) 91 (ns) 93 (ns) 76 (ns) 74 (ns) 80 (ns) 2012 % (Pvalue) 86 (ns) 86 (ns) 93 (ns) 92 (ns) 72 (ns) 69 (ns) 78 (ns) 2013 % Pvalue) 89 (ns) 85 (ns) 93 (ns) 94 (ns) 71 (ns) 71 (ns) 80 (ns) TABLE-6: GSA 2 – Cenpatico Adult Consumer Survey Results by Domain for 2007 - 2013 Domain 2007 % General Satisfaction 87 Service Access 80 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 79 Improved Functioning 73 Social Connectedness 74 12 87 78 2008 % (Pvalue) 84 (ns) 82 (ns) 86 (ns) 88 (ns) 71 (ns) 75 (ns) 75 (ns) 2009 % (Pvalue) 86 (ns) 81 (ns) 82 (ns) 89 (ns) 71 (ns) 67 (ns) 71 (ns) 2010 % (Pvalue) 91 (ns) 90 (0.02) 93 (0.00) 92 (ns) 81 (0.03) 75 (ns) 83 (0.01) 2011 % (Pvalue) 89 (ns) 88 (ns) 91 (ns) 90 (ns) 82 (ns) 77 (ns) 83 (ns) 2012 % (Pvalue) 88 (ns) 83 (ns) 92 (ns) 91 (ns) 69 (0.00) 70 (0.03) 80 (ns) 2013 % Pvalue) 91 (ns) 88 (ns) 91 (ns) 91 (ns) 73 (ns) 71 (ns) 78 (ns) 2013 Consumer Survey Report TABLE-7: GSA 3 - Cenpatico Adult Consumer Survey Results by Domain for 2007 - 2013 2007 % Domain General Satisfaction 90 Service Access 83 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 82 Improved Functioning 72 Social Connectedness 71 Color Key statistically significant increase statistically significant decrease 13 89 79 2008 % (Pvalue) 82 (0.01) 77 (ns) 87 (ns) 86 (ns) 68 (0.01) 73 (ns) 72 (ns) 2009 % (Pvalue) 83 (ns) 79 (ns) 79 (0.02) 88 (ns) 71 (ns) 71 (ns) 71 (ns) 2010 % (Pvalue) 86 (ns) 84 (ns) 88 (0.03) 88 (ns) 77 (ns) 72 (ns) 80 (0.05) 2011 % (Pvalue) 86 (ns) 86 (ns) 91 (ns) 88 (ns) 75 (ns) 73 (ns) 81 (ns) 2012 % (Pvalue) 89 (ns) 91 (ns) 94 (ns) 93 (ns) 81 (ns) 78 (ns) 88 (ns) 2013 % Pvalue) 88 (ns) 88 (ns) 93 (ns) 92 (ns) 79 (ns) 75 (ns) 79 (0.005) 2013 Consumer Survey Report TABLE-8: GSA 4 - Cenpatico Adult Consumer Survey Results by Domain for 2007 - 2013 Domain 2007 % General Satisfaction 87 Service Access 86 Participation in Treatment Planning Service Quality & Appropriateness 81 Outcomes 78 Improved Functioning 76 Social Connectedness 73 91 2008 % (P-value) 87 (ns) 83 (ns) 92 (0.01) 90 (ns) 73 (ns) 67 (0.04) 67 (ns) 2009 % (Pvalue) 86 (ns) 83 (ns) 84 (0.00) 89 (ns) 70 (ns) 64 (ns) 66 (ns) 2010 % (Pvalue) 87 (ns) 87 (ns) 92 (0.03) 90 (ns) 75 (ns) 73 (ns) 77 (0.02) 2011 % (Pvalue) 91 (ns) 89 (ns) 95 (ns) 93 (ns) 76 (ns) 70 (ns) 79 (ns) 2012 % (Pvalue) 93 (ns) 93 (ns) 94 (ns) 95 (ns) 82 (ns) 76 (ns) 81 (ns) 2013 % Pvalue) 91 (ns) 93 (ns) 92 (ns) 95 (ns) 76 (ns) 70 (ns) 82 (ns) TABLE-9: GSA 5 - CPSA Adult Consumer Survey Results by Domain for 2007 – 2013 Domain 2007 % General Satisfaction 85 Service Access 78 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 78 Improved Functioning 64 Social Connectedness 64 14 89 70 2008 % (Pvalue) 83 (ns) 67 (0.01) 91 (0.00) 80 (0.00) 66 (ns) 64 (ns) 62 (ns) 2009 % (Pvalue) 78 (ns) 69 (ns) 78 (0.00) 87 (ns) 68 (ns) 65 (ns) 65 (ns) 2010 % (Pvalue) 87 (0.03) 79 (0.03) 92 (0.03) 90 (ns) 69 (ns) 68 (ns) 78 (0.01) 2011 % (Pvalue) 89 (ns) 85 (ns) 92 (ns) 91 (ns) 75 (ns) 72 (ns) 76 (ns) 2012 % (Pvalue) 87 (ns) 78 (0.02) 89 (ns) 85 (0.02) 75 (ns) 70 (ns) 75 (ns) 2013 % Pvalue) 87 (ns) 79 (ns) 89 (ns) 87 (ns) 71 (ns) 68 (ns) 74 (ns) 2013 Consumer Survey Report TABLE-10: GSA 6 – Value Options in 2007 and Magellan 2008 - 2013 Adult Consumer Survey Results by Domain 2007 % Domain General Satisfaction 87 Service Access 83 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 83 Improved Functioning 67 Social Connectedness 67 Color Key statistically significant increase statistically significant decrease 15 90 77 2008 % (Pvalue) 80 (0.04) 71 (0.00) 87 (ns) 83 (0.02) 68 (0.03) 68 (ns) 69 (ns) 2009 % (Pvalue) 86 (ns) 82 (0.02) 85 (ns) 90 (ns) 73 (ns) 71 (ns) 69 (ns) 2010 % (Pvalue) 85 (ns) 79 (ns) 92 (0.05) 90 (ns) 77 (ns) 72 (ns) 79 (0.03) 2011 % (Pvalue) 83 (ns) 83 (ns) 82 (0.00) 84 (0.05) 68 (0.03) 65 (ns) 74 (ns) 2012 % (Pvalue) 83 (ns) 85 (ns) 89 (0.04) 84 (ns) 72 (ns) 65 (ns) 74 (ns) 2013 % Pvalue) 92 (0.003) 89 (ns) 91 (ns) 91 (0.021) 71 (ns) 72 (ns) 77 (ns) 2013 Consumer Survey Report Summary of the 2013 Adult Survey Results: Statewide:  The domains with the highest scores in 2013 for the adult population in the 2013 survey results are: General Satisfaction, Service Access and Service Quality and appropriateness.  General Satisfaction domain improved by five percent representing a statistically significant score with a P-value of 0.01 when compared to the 2012 results.  Service Quality and Appropriateness increased by four percent from the 2012 score, indicating a statistical significance with a P-value of 0.21 when compared to the 2012 results. Service Access also showed an increase of four percent from the 2012 results. This was however not statistically significant. Outcomes domain showed a slight decrease by two percent.    Social Connectedness and Improved Functioning domains both showed a slight increase by one and two percent respectively from the 2012 survey results. Figure 2 (A) -Trending Annual Survey Results by Domains FY 2012-2013 Survey score in percent 96.1 91.1 General Satisfaction 86.1 Service Access 81.1 Participation in Treatment Planning 76.1 71.1 2007 Service Quality & Appropriateness 2008 2009 2010 2011 2012 2013 Year Domains General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness 16 2007 86% 77% 2008 82% 73% 2009 84% 78% 2010 86% 81% 2011 86% 84% 2012 85% 83% 2013 90% 87% 79% 89% 82% 92% 87% 90% 91% 88% 84% 89% 90% 88% 87% 91% 2013 Consumer Survey Report 81.6 Figure 2 (B) -Trending Annual Survey Results by Domains FY2012-2013 79.6 Survey Scores in Percent 77.6 75.6 73.6 71.6 69.6 Outcomes 67.6 Improved Functioning 65.6 63.6 2007 Social Connectedness 2008 2009 2010 2011 2012 2013 Years Domains 2007 2008 2009 2010 2011 2012 2013 Outcomes 74% 68% 70% 75% 72% 74% 72% Improved Functioning 66% 68% 68% 71% 69% 69% 71% Social Connectedness 65% 68% 68% 79% 76% 76% 77% RBHA Specific: 17  Magellan (GSA 6) showed a statistically significant increase in two domains: General Satisfaction (P<0.003), and Service Quality (P<0.02), which may be attributed to Magellan‟s initiatives during FY2012.  Cenpatico (GSA 3) showed statistically significant change in the Social Connectedness domain (P<0.005) 2013 Consumer Survey Report VIII. RBHA Specific Adult Initiatives Cenpatico and CPSA submitted to ADHS/DBHS some initiatives for the adult population to address deficiencies identified in their Consumer Survey report: Cenpatico recognized the statistically significant decreases in Outcomes and Functioning domains for GSA 2 in the 2012 survey and reported actions planned to address these areas.  Cenpatico developed a Performance Improvement Plan (PIP) in December 2012 that included the collection of a quarterly satisfaction survey to measure its consumer satisfaction with providers in each GSA.  Cenpatico will maintain this practice, and plans to enhance the survey in January 2013 with particular attention to these domains.  Feedback from the surveys will be shared with Cenpatico‟s QM Committee and its provider network each quarter.  After two quarters of collecting results from the new 2013 survey, Cenpatico will assess each of its geographic areas for the need of targeted interventions where the need for improvement is indicated. ADHS/DBHS requires Cenpatico to regularly report updates on these initiatives to ensure there is a positive impact and performance improvement. ADHS/DBHS will meet and discuss with Cenpatico about trends in survey results. CPSA acknowledged the need for improvements specifically in the Service Access and Service Quality & Appropriateness domains. Listed below are steps CPSA is implementing to improve and meet the needs of consumers.        18 CPSA continues to maintain frequent communication with members to ensure coordination of, and access to, behavioral health services among Comprehensive Service Providers and other community resources. CPSA maintains an Audio Visual Library on the website that includes a comprehensive overview of member-centered educational materials intended to educate and empower members. CPSA will continue to improve its website to be more inclusive of information and training opportunities for members and families. CPSA shares these results with all Comprehensive Service Providers (CSP) and expects implementation of improvements in their systems to meet and exceed the needs of members. CPSA QM staff continues to audit comprehensive Service Provider medical charts and provide real-time feedback to Comprehensive Service Provider staff on key MHSIP indicators. CPSA Compliance Committee is working on issues around system-wide sub-contract tracking systems and any specific compliance issues that the CPSA Comprehensive Service Provider Team cannot resolve. CPSA continues to involve peers in the support of members through the Recovery Support Initiative and the development of Recovery Support Centers established within the CSPs. CPSA implemented a pilot program utilizing the Outcomes Rating Scale and the Session Rating Scale in order to elicit ongoing member feedback to improve treatment or service outcomes. 2013 Consumer Survey Report IX. YSS-F Survey Analysis and Results Demographics A total of 1,991 completed youth surveys were analyzed. The breakout of the respondents is reported by gender, age group, ethnicity, race, length of enrollment in services, and involvement in a Child and Family Team (CFT). Please see Attachment F2013 YSS-F Demographics for complete demographics information. Domain Line Item Analysis Line items are specific questions pertaining to each survey domain. Analysis of answers to each domain specific line item indicates the aspects of service delivery the respondents report as either positively or negatively affecting the overall domain score (Attachment G, 2013 YSS-F Line Item). Table-11 demonstrates the results of the 2013 YSS-F Survey and compares them to the 2012 survey results. The t-test analysis was used to compare survey results for 2012 and 2013 to determine if there were any statistically significant differences between each domain‟s average scores. A P-value 0.05 with 95% confidence interval significance was used to determine statistical significance: if the p-value is less than or equal to 0.05, the result will be considered statistically significant and if it is more than 0.05, it will be considered not statistically significant. The abbreviation “ns” is used for “not significant” to denote no statistically significant change from the previous year. The statewide analyses are based on weighted scores. TABLE-11: Statewide Youth Consumer YSS-F Survey Results by Domain for 2007 – 201 3 2007 % Domain General Satisfaction 78 Service Access 78 Participation in Treatment Planning Cultural Sensitivity 90 Outcomes 58 Improved Functioning 61 Social Connectedness 82 Color 19 Key statistically significant increase statistically significant decrease 92 2008 % (P-value) 77 (ns) 73 (ns) 87 (ns) 90 (ns) 60 (ns) 58 (ns) 79 (ns) 2009 % (Pvalue) 80 (ns) 76 (ns) 89 (ns) 90 (ns) 68 (ns) 66 (ns) 83 (ns) 2010 % (Pvalue) 85 (ns) 83 (ns) 92 (ns) 96 (0.05) 69 (ns) 71 (ns) 88 (ns) 2011 % (Pvalue) 82 (ns) 86 (ns) 94 (ns) 96 (ns) 65 (ns) 67 (ns) 89 (ns) 2012 % (Pvalue) 82 (ns) 86 (ns) 92 (0.02) 96 (ns) 62 (0.00) 68 (ns) 86 (0.03) 2013 % (P-value 82 (ns) 84 (0.03) 93 (ns) 95 (ns) 61 (ns) 64 (ns) 87 (ns) 2013 Consumer Survey Report The 2013 statewide YSS-F Survey Results: Figure 3-Statewide YSS-F Survey Results : Rate Change Analysis by Domains: 2012-2013 Social Connectedness Improved Functioning Outcomes 2013 Cultural Sensitivity 2012 Participation in Treatment Planning Service Access General satisfaction 0 Domains General satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 20 40 60 80 2012 82% 86% 92% 96% 62% 68% 86% 100 2013 82% 84% 93% 95% 61% 64% 87% General Satisfaction: Six questions were asked in the survey to ascertain General Satisfaction (Attachment G). The overall score for this domain is 82%. There was no change in this domain between the 2012 and 2013 survey results. Service Access: Two questions in the survey gathered information regarding Access to Services (Attachment G). The score for this domain is 84%, indicating a 2% drop from the 2012 survey results. Participation in Treatment Planning: The survey comprised three questions to indicate Participation in Treatment Planning (see Attachment G). The score for this domain is 93%, up one percent from 2012 results. The increase was however not statistically significant. Cultural Sensitivity: Four questions in the survey focused on Cultural Sensitivity (Attachment G). The average score for this domain is 95%, indicating a decrease by 1% from the 2012 results. 20 2013 Consumer Survey Report Outcomes: Eight questions were asked to get consumers‟ perception on Outcomes (Attachment G). The average score for this domain is 61% which is a decline by one percent from 2012. Improved Functioning: Six questions indicate respondents‟ impressions of Improved Functioning (Attachment G). The average score for this domain is 64%, demonstrating a four percent decrease from 2012m results Social Connectedness: Four questions were asked in the survey to ascertain Social Connectedness (see Attachment G). This domain scored 87%, indicating one percent increase from 2012. RBHA and GSA specific YSS- F survey findings are presented in tables 12 through 17 (page 16 and 17); they include percentages and P-values for each domain by RBHA and GSA ranging from 2007 through 2013. 21 2013 Consumer Survey Report TABLE-12: GSA 1 - NARBHA Youth Consumer YSS-F Survey Results by Domain for 2007 – 2013 Domain 2007 % General Satisfaction 76 Service Access 77 Participation in Treatment Planning Cultural Sensitivity 91 Outcomes 51 Improved Functioning 55 Social Connectedness 80 93 2008 % (Pvalue) 84 (ns) 81 (ns) 88 (ns) 93 (ns) 72 (0.00) 71 (0.00) 87 (ns) 2009 % (Pvalue) 83 (ns) 79 (ns) 92 (ns) 93 (ns) 67 (ns) 66 (ns) 87 (ns) 2010 % (Pvalue) 90 (ns) 90 (0.01) 93 (ns) 96 (ns) 70 (ns) 73 (ns) 91 (ns) 2011 % (Pvalue) 90 (ns) 94 (ns) 94 (ns) 97 (ns) 69 (ns) 71 (ns) 90 (ns) 2012 % (Pvalue) 85 (ns) 90 (ns) 93 (ns) 98 (ns) 65 (ns) 68 (ns) 92 (ns) 2013 % (Pvalue) 85 (ns) 93 (ns) 96 (ns) 97 (ns) 67 (ns) 71 (ns) 90 (ns) TABLE-13: GSA 2 - Cenpatico Youth Consumer YSS-F Survey Results by Domain for 2007 – 2013 Domain 2007 % General Satisfaction 89 Service Access 86 Participation in Treatment Planning Cultural Sensitivity 88 Outcomes 76 Improved Functioning 78 Social Connectedness 89 22 96 2008 % (Pvalue) 76 (0.00) 81 (ns) 86 (ns) 94 (ns) 70 (ns) 69 (0.03) 86 (ns) 2009 % (Pvalue) 81 (ns) 82 (ns) 94 (0.02) 93 (ns) 62 (ns) 61 (ns) 85 (ns) 2010 % (Pvalue) 82 (ns) 90 (0.04) 93 (ns) 99 (0.02) 69 (ns) 70 (ns) 89 (ns) 2011 % (Pvalue) 91 (0.02) 95 (ns) 96 (ns) 99 (ns) 71 (ns) 73 (ns) 97 (0.01) 2012 % (Pvalue) 89 (ns) 93 (ns) 93 (ns) 97 (ns) 75 (ns) 76 (ns) 94 (ns) 2013 % (Pvalue) 88 (ns) 91 (ns) 93 (ns) 95 (ns) 74 (ns) 76 (ns) 91 (ns) 2013 Consumer Survey Report TABLE-14: GSA 3 - Cenpatico Youth Consumer YSS-F Survey Results by Domain for 2007 – 2013 2007 % Domain General Satisfaction 77 Service Access 74 Participation in Treatment Planning Cultural Sensitivity 88 Outcomes 65 Improved Functioning 65 Social Connectedness 77 Color Key statistically significant increase statistically significant decrease 23 90 2008 % (Pvalue) 74 (ns) 74 (ns) 80 (0.01) 89 (ns) 65 (ns) 62 (ns) 77 (ns) 2009 % (Pvalue) 75 (ns) 76 (ns) 88 (0.05) 89 (ns) 66 (ns) 65 (ns) 78 (ns) 2010 % (Pvalue) 87 (0.01) 90 (0.00) 90 (ns) 94 (ns) 77 (0.02) 79 (0.00) 87 (0.03) 2011 % (Pvalue) 86 (ns) 90 (ns) 89 (ns) 96 (ns) 73 (ns) 74 (ns) 87 (ns) 2012 % (Pvalue) 88 (ns) 93 (ns) 96 (0.00) 97 (ns) 71 (ns) 74 (ns) 91 (ns) 2013 % (Pvalue) 82 (ns) 92 (ns) 93 (ns) 96 (ns) 67 (ns) 68 (ns) 89 (ns) 2013 Consumer Survey Report TABLE-15: GSA 4 - Cenpatico Youth Consumer YSS-F Survey Results by Domain for 2007 – 2013 Domain 2007 % General Satisfaction 78 Service Access 86 Participation in Treatment Planning Cultural Sensitivity 92 Outcomes 66 Improved Functioning 68 Social Connectedness 82 91 2008 % (Pvalue) 78 (ns) 80 (ns) 90 (ns) 92 (ns) 61 (ns) 62 (ns) 87 (ns) 2009 % (Pvalue) 85 (ns) 91 (0.01) 98 (0.01) 96 (ns) 66 (ns) 65 (ns) 87 (ns) 2010 % (Pvalue) 80 (ns) 87 (ns) 91 (0.00) 95 (ns) 66 (ns) 69 (ns) 88 (ns) 2011 % (Pvalue) 84 (ns) 94 (0.04) 94 (ns) 97 (ns) 70 (ns) 71 (ns) 91 (ns) 2012 % (Pvalue) 82 (ns) 92 (ns) 92 (ns) 96 (ns) 63 (ns) 67 (ns) 88 (ns) 2013 % (Pvalue) 88 (0.04) 91 (ns) 93 (ns) 95 (ns) 75 (0.007) 76 0.01 92 (ns) TABLE-16: GSA 5 - CPSA Youth Consumer YSS-F Survey Results by Domain for 2007 – 201 3 Domain 2007 % General Satisfaction 81 Service Access 76 Participation in Treatment Planning Cultural Sensitivity 89 Outcomes 65 Improved Functioning 66 Social Connectedness 86 24 90 2008 % (Pvalue) 79 (ns) 73 (ns) 86 (ns) 90 (ns) 63 (ns) 62 (ns) 82 (ns) 2009 % (Pvalue) 82 (ns) 74 (ns) 86 (ns) 86 (ns) 72 (ns) 72 (0.04) 84 (ns) 2010 % (Pvalue) 81 (ns) 84 (0.02) 88 (ns) 94 (0.02) 62 (0.03) 66 (ns) 84 (ns) 2011 % (Pvalue) 88 (ns) 87 (ns) 93 (ns) 97 (ns) 68 (ns) 71 (ns) 90 (ns) 2012 % (Pvalue) 84 (ns) 81 (ns) 89 (ns) 91 (0.01) 62 (0.01) 66 (ns) 86 (ns) 2013 % (Pvalue) 85 (ns) 83 (ns) 94 (ns) 96 (ns) 60 (ns) 64 (ns) 85 (ns) 2013 Consumer Survey Report TABLE-17: GSA 6 - Value Options in 2007 and Magellan in 2008 – 201 3 Youth Consumer YSS-F Survey Results by Domain 2007 % Domain General Satisfaction 77 Service Access 75 Participation in Treatment Planning Cultural Sensitivity 86 Outcomes 62 Improved Functioning 63 Social Connectedness 79 Color Key statistically significant increase statistically significant decrease 25 93 2008 % (Pvalue) 75 (ns) 70 (ns) 87 (ns) 90 (ns) 56 (ns) 53 (0.04) 75 (ns) 2009 % (Pvalue) 77 (ns) 73 (ns) 88 (ns) 91 (ns) 68 (0.02) 64 (0.03) 82 (ns) 2010 % (Pvalue) 86 (0.03) 80 (ns) 93 (ns) 98 (0.01) 71 (ns) 72 (ns) 88 (ns) 2011 % (Pvalue) 78 (0.02) 82 (ns) 94 (ns) 96 (ns) 61 (0.03) 63 (0.05) 88 (ns) 2012 % (Pvalue) 80 (ns) 87 (ns) 93 (ns) 97 (ns) 60 (ns) 60 (ns) 84 (ns) 2013 % (Pvalue) 80 (ns) 81 (ns) 92 (ns) 95 (ns) 58 (ns) 61 (ns) 86 (ns) 2013 Consumer Survey Report Summary of the 2012 YSS-F Survey Results: Statewide:  Significant declines were noted in the 2013 statewide youth survey results compared to 2012.  Service Access and the improved functioning domains showed a percentage decrease from the 2012 results.  Social Connectedness domain increased by two percentage points. Additionally,  Improved Functioning domain increased in score by one percentage point from 2012.  General Satisfaction domain showed no change in percentage points between the 2012 and 2013 Results in percentage Figure 5-Statewide Youth Consumer YSS-F Survey Trends by Domains 2007-2013 99.2 94.2 89.2 84.2 79.2 74.2 69.2 64.2 59.2 54.2 2007 General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning 2008 2009 2010 Years 2011 2012 2013 Social Connectedness RBHA Specific: 26  Cenpatico (GSA 2) showed significant increase in the Service Access and Outcomes domains but experience slight percentage decreases in Participation and Treatment and Social Connectedness domains  CPSA (GSA 5) experienced percentage decreases in three domains: Improved Functioning, Social Connectedness and Outcomes 2013 Consumer Survey Report X. RBHA Specific YSS-F Initiatives CPSA recognizes the declines in certain domains of the Youth Consumer survey and has developed plans and activities to improve performance in the Outcomes domain of the youth survey. CPSA plans to utilize performance improvement projects as a strategic to improve scores in the youth consumer survey. Specifically, Comprehensive Service Providers have addressed some of the following in their PIPs: • In order to guide service planning and service provision in a way that aligns with the MHSIP, CSPs are choosing to continue to incorporate MHSIP outcome indicators into forms used by clinical services, that will be utilized at the beginning of every appointment to address where members feel they are in treatment relative to outcomes attributable to treatment. Staff representatives from QM, Clinical services and Customer Service Committees have been meeting to create these forms as well as a process for staff to discuss these eight indicators with the members. This is an opportunity for staff to address what may or may not be working in treatment, allowing the opportunity for members to address outcomes with their clinical team on a consistent basis. This PIP initiative was a huge focus for the Adult providers in 2012 and once again appeared to make a significant impact based on the Outcome domain scores. Some of the youth providers are choosing to address the Outcomes Indicator language more specifically in the CFT agenda, by implementing this language into the CFT Agenda forms, and calling on CFT coaches to monitor the effectiveness of this language within the CFT. The effectiveness of these tools will be closely evaluated in the 2013 Corrective Action Plans across the CPSA Children‟s system. 27 • CSPs are choosing to continue to conduct internal biannual audits to assess compliance. Any identified trends in this audit structure would lead to QM interventions and implementation of new strategies based on the results of these audits. • Continued incorporation of MHSIP Outcomes domain language in assessment tools will allow for a better assessment of progress, or lack thereof, by the members of the clinical team and guardians/parents/caregivers. • CSPs have continued to focus on the accuracy and effectiveness of maintaining highly functioning Electronic Medical Records in order to better track members and their progress while coordinating care with other agencies that may be addressing member care as well. • CSPs have also created work groups comprised of Clinical and QM staff, who are addressing how to incorporate, identify, communicate and document outcomes with the Child and Family Team. Many CSPs report that though this is a current occurrence the focus on documenting these actions in a way that clearly defines outcomes must be addressed. • CSPs are creating additional Service Planning Training in hopes of creating more meaningful measures and goals with members. This strategy should directly impact the 2013 Consumer Survey Report • 28 perception of Outcomes with members. CSPs have implemented the Customer Service/Cultural Liaison positions to focus on increasing customer satisfaction. • CSP‟s have implemented consumer advisory boards to encourage the consumer voice in efforts to provide the highest quality services to consumers. • CSPs are also providing training by RSS staff at New Employee Orientation and at CM sites. The training will focus on recovery and strengths-based language. MHSIP outcomes language will be discussed as examples for clinicians to use when engaging members in the discussion of progress. 2013 Consumer Survey Report Magellan identified domains that did not perform well during the 2013 survey and has put in place some activities to improve the scores. To improve the scores in the Youth Outcome, Improved Functioning and Service Access domains for example,  In October 2013, analysis began to determine the reasons recipients do not feel their services are positively impacting their outcomes began through the existing committee structure as well as a Child Provider Meeting and Workgroup. Two areas were targeted for improvement: 1.) limited feedback delivered to individuals and families regarding the progress towards goals and 2.) staff members do not consistently focus interactions / services based on the person's goals, preferences and needs. Performance Improvement Plans will be implemented to address these areas. The Clinic Advisory Committees will also be asked to solicit feedback from recipients. The Customer Service Committee will review information and PIP updates on a routine basis and put forward further recommendations. NARBHA recognizes the opportunity to use the information from the 2013 MHSIP Survey to make system Improvements, and will participate in statewide initiatives with ADHS/DBHS to continue to improve scores for each domain. Special attention will be placed around the Outcomes and Improved Functioning domains, which continue to receive the lowest scores from both Adult and Youth surveys.  As part of their strategy, NARBHA survey outcomes are reported to the Quality Management Committee, the NARBHA Leadership Council, ADHS/DBHS, Responsible Agencies, the NARBHA Board of Directors and all NARBHA staff. Performance Improvement and Quality Management will provide continuous technical assistance throughout the survey process, assisting Responsible Agencies to identify opportunities to improve the survey process, response rates and results. Targeted technical assistance is provided when there is a substantial decrease in overall results from the prior year XI. Statewide Improvement Opportunities ADHS/DBHS recognizes the opportunity to use the information from the 2013 survey results to make system improvements, and will implement statewide initiatives with the RBHAs to continue to improve scores for each domain. Special attention will be placed around the Outcomes and Improved Functioning domains, which continue to receive the lowest scores from both Adults and Children. DBHS will also carry out a performance improvement project to improve on the number of surveys responses that are returned. A comprehensive plan to reengineer the consumer survey distribution process includes a review of the time of the year when the surveys are distributed, increase consumer education on the relevance of the survey to systems improvement, and increase monitoring of the use of survey results by all ADHS/DBHS contractors to drive systems improvement. In addition to general statewide initiatives, RBHA-specific activities are undertaken as part of standard ADHS/DBHS processes, as appropriate. Survey results are shared with the RBHAs, and they are required to respond with analysis of any problem areas, along with plans for 29 2013 Consumer Survey Report improvement. ADHS/DBHS reviews their conclusions and plans, and monitors implementation and progress. XII. ADHS/DBHS Initiatives Below are ADHS/DBHS initiatives for both the Children and Adult systems of care that address Outcomes, Improved Functioning, Service Access and Social Connectedness: Adult Initiatives:  The 2013-2014 ADHS/DBHS Adult System of Care (ASOC)-Strategic Plan (SP) establishes goals and objectives for statewide network development priorities in part through a review of State and GSA network analysis, review of data sets, community input, and the identification of best practices efforts that are designed to assist individuals moving through recovery and increasing the quality of life outcomes. The analyses of the ASOC-strategic plan and the 2012 and 2013 Adult and Youth Consumer Survey identified similar priorities. For example: o An objective in the ASOC-SP is to increase the employment rate for persons with Serious Mental Illness (SMI), General Mental Health (GMH) and Substance Abuse (SA). The Outcomes domain of the Consumer Survey addresses a person‟s ability to: 1) control his/her life, 2) operate in school, and work environment, and 3) do better in social situations; by achieving the improvement in employment rate in ASOC-SP, ADHS/DBHS expected an increase in the Outcomes domain. However, based on the 2013 survey results, there was a decrease in the Outcome domain by two percentage points. ADHS/DBHS will perform a root cause analysis of the drop in scores in the outcome domain and develop a plan with its contractors to improve scores. o The ASOC-SP includes an objective to increase the percentage of peers and family members satisfied with their access to services which was identified as an area needing improvement in the Consumer Survey Report. Additionally, ASOC-SP includes strategies to conduct statewide discussion on the awareness and community needs around Trauma Informed Care. System Initiatives: 30  ADHS/DBHS, along with family members and recipient input, developed the Behavioral Health System Performance Framework and Dashboard as a means to evaluate the effectiveness of the system using standardized measures, such as family and recipient identified outcomes, the National Outcomes Measures (NOMs) and ADHS/DBHS/BQMO performance metrics.  ADHS/DBHS continues to routinely update its Behavioral Health System Performance Framework and Dashboard, providing transparency, accountability, and increased consumer education regarding ADHS/DBHS performance measures.  ADHS/DBHS as part of the survey re-engineering process has initiated a plan through the office of Performance Improvement and the Office of Information Management to 2013 Consumer Survey Report carryout correlation analyses between the survey results and other appropriate performance measures to identify and utilized any indicators that associate survey results to performance measures.  ADHS/DBHS has also increase information sharing of survey results and initiatives between departments of the ADHS/DBHS. The Office of Information Management, the Office of Performance Improvement and the Office of Costumer Service have expanded information sharing of survey results. This is in addition to existing collaboration between the office of information management and the Adult System of Care in the use of survey results to drive systems improvement. Children’s Initiatives: 31  ADHS/DBHS continues to use the System of Care Practice Review (SOCPR) process developed by the University of South Florida to assess Child and Family Team (CFT) practice for youth and families with complex needs. These intensive case reviews include outcomes-focused questions to ensure CFTs are identifying and tracking outcomes as well as evaluating the effectiveness of provided services and improved functioning. Consumer Survey results are shared with all divisions at DBHS, including the Children‟s System of Care so the areas needing improvement can be addressed. 180 SOCPR reviews were conducted last year with another 180 scheduled for FY13. There were also more than one thousand Brief Practice Reviews (BPR‟s) conducted via telephonic interviews with caregivers. The reviews are conducted at the individual provider level and are used to develop annual performance improvement plans. The plans are approved by DBHS and monitored by the RBHA with regular updates provided to DBHS.  For FY10, ADHS/DBHS earmarked approximately five million dollars of additional funding to the RBHAs to increase the number of case managers with reduced caseloads. During the course of FY10, there was an increase of approximately 104 case managers statewide serving children with complex needs. In total this focused effort has developed approximately 469 case managers statewide as of the end of the FY11 fiscal year. ADHS/DBHS continues to monitor the availability of case managers with reduced caseloads for children with complex needs through the 2013 Fiscal year.  The continued emphasis on in-home direct support services, as established by the Meet Me Where I Am (MMWIA) Campaign, has been a valuable resource to children and families with the most complex needs. These services put direct support workers in homes, schools and communities at the times and for as many hours as necessary to meet the family‟s needs, leading to improved functioning and better outcomes.  ADHS/DBHS continues to monitor and promote development of generalist direct support programming through the use of a SE Modifier to track encounter activity and through regular program updates.  ADHS/DBHS continues to promote family and youth involvement in the Children‟s Behavioral Health System. In partnership with family-run organizations, efforts are focused on building and sustaining an infrastructure and agency culture to support and involve youth and family members at all levels of the system. During FY13 a 2013 Consumer Survey Report particular emphasis has been placed on supporting the certification process for peer mentors in order to more formally integrate those services into the developing system of care. Additionally the newly revised Youth Involvement Practice Protocol will continue to strengthen the youth and family involvement in the system of care. 32  The annual Children‟s System of Care (CSOC) Strategic Plan (SP) provides a vehicle for the Division and RBHAs to address key goals around CFT Practice Model monitoring through the Practice Review process, and development of youth and family involvement and leadership, Generalist Direct Support services and High Needs Case Management services to work with children and families with the most complex needs.  The strategic activities prioritized in CSOC-SP continue to promote family driven care as well as youth and family involvement in local and statewide committees and system development activities. Focus is placed on specific populations such as children ages birth to 5, transition aged youth, and youth with substance use disorders in order to more clearly define their needs and help providers develop expertise in working with these youth. 2013 Consumer Survey Report XIII. Conclusion The Annual Consumer Surveys are a valuable source of information, providing direct feedback from members about their experience with behavioral health services. The results identify areas of satisfaction as well as opportunities for improvement. The Participation in Treatment Planning and Service Quality & Appropriateness domains have had the most favorable statewide results in the Adult Survey for each of the last three years, while Outcomes, Improved Functioning, and Social Connectedness have consistently shown the most room for improvement. The Children‟s Survey has reflected statewide satisfaction levels of over 90 percent in the Participation in Treatment Planning and Cultural Sensitivity domains for the last three years; the results for the Outcomes and Improved Functioning domains have been the lowest statewide scores of all domains for that population for every year of the Survey. This year‟s 2013 adult statewide Survey results show a slight drop in the Outcomes domain while General Satisfaction and Service Quality and Appropriateness domains demonstrated significant increases by five and four percentage points respectively. The General Satisfaction domain however demonstrated the largest improvement in rate change. The highest scoring domain for the adult population in 2013 is Participation in Treatment Planning and the lowest is the Improved Functioning domain. In the 2013 adult survey at the RBHA/GSA level, Magellan (GSA 6) showed a significant increase in Participation in Treatment Planning domain by seven percentage points, leading to a statistically significant change (P<0.04). This may be attributed to Magellan‟s initiatives during FY2012. Cenpatico (GSA 4) showed six percent decrease in the Outcome domain. Declines were noted in the 2013 statewide youth survey. Four domains: Service Access, Cultural Sensitivity, Outcomes and improved Functioning showed slight decreases from the 2012 survey results. At the RBHA/GSA level on the children‟s 2013 survey, Cenpatico (GSA 3) showed a significant increase in Participation in Treatment Planning domain by seven percentage points (P<0.00). CPSA (GSA 5) experienced statistically significant decreases in two domains: Cultural Sensitivity and Outcomes domains fell six percentage points each from 2011 (P<0.01). Initiatives have been developed by ADHS/DBHS to address areas of concern reflected in the Survey results, specifically the Outcomes, Improved Functioning, Service Access, and Social Connectedness domains. Both the Adult and Children‟s System of Care Strategic Plans are informed by Survey results. Standardized measures are shared with DBHS stakeholders through the Behavioral Health System Performance Framework and Dashboard. In addition, there are RBHA-specific activities. Cenpatico is enhancing an existing Performance Improvement Project and its quarterly satisfaction survey to better identify the need for targeted interventions. CPSA maintains frequent contact with members and has implemented feedback through rating scales to enhance treatment and service outcomes. ADHS/DBHS will monitor and work closely with these RBHAs to improve member satisfaction. 33 2013 Consumer Survey Report As per the ADHS/DBHS Contract with the RBHAs, satisfaction with service Outcomes on the Annual Consumer Survey is a performance incentive; a score of 70% or greater on the Outcomes domain of the Consumer Survey makes up 25% of the incentive award. Adult and Youth survey results are evaluated independently, each contributing to half of the award. ADHS/DBHS believes that assisting members in the identification and tracking of treatment outcomes is imperative to improving behavioral health recipients‟ overall perception of the quality of services they receive. 34 2013 Consumer Survey Report ATTACHMENT-A 35 2013 Consumer Survey Report 2013 Tribal Behavioral Health Survey Results Arizona Tribal Behavioral Health Authorities (TBHAs) Gila River Indian Community, Pascua Yaqui Centered Spirit Program, Navajo Nation and White Mountain Apache Tribe participated in the 2013 Consumers Survey. The Tribal BHAs have shown an overall slight increase in participation from 433 participants in 2012 to 775 in 2013. All Tribal BHAs conduct a convenience sampling of their enrolled members. The number of surveys completed by tribal participants varied this year, White Mountain, Navajo Nation and Pascua Yaqui increased while Gila River decreased. TRIBAL Participation in Survey: 2012-2013 500 Number of Surveys completed 400 300 200 100 2012 2013 0 Gila River Navajo Nation Pascua Yaqui White Mountain Apache TRIBES Number of Surveys Completed TBHA Gila River Adult Survey 26 YSS-F 19 Total 45 Navajo Nation 137 46 183 Pascua Yaqui 317 130 447 White Mountain Apache Statewide TBHA 80 560 20 215 100 775 Methodology and Survey Administration A convenience sampling methodology was used (rather than a random sampling); therefore, all persons enrolled with the TBHAs were eligible for participation in the survey. 36 2013 Consumer Survey Report Data Analysis TBHAs results were submitted directly to ADHS/DBHS and entered into the database; analysis was conducted by ADHS/DBHS staff in the Office of Information Management. Summary of Results: General Satisfaction domain demonstrates the highest scores for adults receiving services through Tribal BHAs in 2013 compared to 2012. Social Connectedness domain showed a decrease in rate of satisfaction amongst TBHA adults. Navajo Nation scored 100% in the Cultural Sensitivity domain and White Mountain showed a great decrease from 100% tom 67% from 2012 to 2013 in the Youth Improved Functioning domain. Outcomes and Improved Functioning domains continue to remain the lowest rates of satisfaction reported from families of youth receiving services through the Tribal BHAs. 37 2013 Consumer Survey Report Summary of 2010 to 2013 Tribal Behavioral Health Authorities Results for Adults General Satisfaction TBHA 2011 Gila River Navajo Nation Pascua Yaqui White Mtn Apache Participation in Treatment Planning Service Access Service Quality & Appropriateness Improved Functioning Outcomes 2013 2011 2012 Social Connectedness 2012 2013 2011 2012 2013 2011 2012 2013 2011 2012 2013 2011 2012 2013 92% 89% 96% 85% 81% 88% 92% 85% 92% 96% 83% 88% 84% 68% 78% 84% 70% 88% 85% 2011 85% 2012 88% 2013 91% 85% 82% 80% 85% 83% 89% 88% 88% 88% 87% 88% 82% 75% 72% 83% 80% 72% 86% 87% 77% 94% 94% 92% 86% 92% 90% 90% 89% 92% 93% 92% 91% 82% 82% 81% 78% 84% 79% 85% 91% 85% 93% 93% 94% 96% 87% 73% 86% 87% 74% 86% 88% 70% 76% 87% 72% 86% 80% 74% 90% 93% 88% TBHA scores are not included in statewide numbers due to TBHA convenience sampling methodology. Summary of 2010 to 2013 Tribal Behavioral Health Authorities Results for Youth General Satisfaction TBHA Gila River Navajo Nation Pascua Yaqui White Mtn Apache Service Access Participation in Treatment Planning Outcomes 2011 82% 2012 86% 2013 90% 2011 86% 2012 96% 2013 90% 2011 96% 2012 91% 2013 78% 2011 100% 2012 100% 84% 94% 98% 88% 96% 87% 88% 91% 91% 96% 93% 90% 72% 100 % 92% 89% 95% 92% 90% 100% 100% 76% 90% 100% 81% 90% 100% 86% Social Connectedness 2011 64% 2012 68% 2013 74% 2011 64% 2012 68% 2013 74% 2011 96% 2012 96% 2013 95% 94% 2013 95% 100 % 73% 94% 93% 75% 94% 93% 91% 94% 98% 100% 97% 95% 71% 93% 80% 75% 93% 80% 95% 95% 92% 100% 100% 95% 85% 100% 67% 85% 100% 67% 95% 100% 91% TBHA scores are not included in statewide numbers due to TBHA convenience sampling methodology. 38 Improved Functioning Cultural Sensitivity 2013 Consumer Survey Report ATTACHMENT-B 39 2013 Consumer Survey Report STATEWIDE CONSUMER SURVEY PROTOCOL 2013 Arizona Department of Health Services Division of Behavioral Health Services 40 2013 Consumer Survey Report Executive Summary The protocol for the 2013 consumer survey is based on the methodological and administrative guidelines followed in previous years. Two consumer surveys will be administered again in 2013; one for adults and one for the parents/guardians of children receiving behavioral health services. The survey will be available to consumers in two languages: English and Spanish. Survey data will be entered into an SPSS database by the Regional Behavioral Health Authorities (RBHAs) and forwarded to the Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS). Prior to implementation of the 2013 survey, ADHS/DBHS will provide each Geographic Service Area (GSA) with the CIS (Client Information System) Client IDs for 500 adult consumers and 400 to 500 youth consumers who the RBHAs/providers will encourage to participate in the survey. Each CIS Client ID will be linked to a Survey ID reference number to allow for tracking of which consumers completed a survey. The survey will be administered during April and May 2013. As in previous years, the primary administration route will be to distribute the survey at the provider sites. As the consumer checks in for their appointment, the survey questionnaire will be handed to them. The consumer will be requested to complete the questionnaire prior to their appointment, but will be allowed to finalize the survey after the appointment if needed, or be provided with an addressed, stamped envelope to mail the survey in to the RBHA if they did not have time to complete it in the office. A drop box will be provided at each provider site for consumers to drop off their completed surveys. In cases where consumers who are selected for the sample have home appointments (as opposed to clinic appointments) during the survey timeframe, the questionnaire will be completed at home and mailed using the addressed, stamped envelope to be provided with the survey. If the consumer does not have a clinic or home appointment scheduled during the survey administration period or no-shows for a scheduled appointment, a non-clinical staff at the RBHA may conduct the survey by phone. Providers will never have access to completed surveys or individual survey results. ADHS has the statewide oversight responsibility for implementation and analysis of the survey data. The RBHAs will be responsible for ensuring that providers strictly adhere to the protocol. The providers are primarily responsible for the survey administration. Survey Instruments The two MHSIP survey instruments are the Adult Consumer Survey and the Youth Services Survey for Families (YSS-F). The adult survey will be administered to adult consumers of behavioral health services and the YSS-F will be administered to parents/guardians of children receiving behavioral health services. The MHSIP Adult Consumer Survey measures seven domains: (1) Service Accessibility; (2) Service Quality or Appropriateness (which includes one item concerning cultural sensitivity); (3) 41 2013 Consumer Survey Report Consumer Participation in Treatment Planning; (4) Outcomes; (5) General Satisfaction; (6) Improved Functioning; and (7) Social Connectedness. In addition, the questionnaire includes a module of questions to determine the impact of services received on the recipient‟s involvement with the criminal justice system. All questions are scored using a Likert Scale of 1 through 5 as follows: 5=Strongly Agree, 4=Agree, 3=Neutral, 2=Disagree, and 1=Strongly Disagree. A Not Applicable option is also available if the question does not apply. In addition to the state-added question related to cultural sensitivity, two state-added questions introduced in the 2011 Adult survey tool to gather information specific to National CLAS Standards (Culturally and Linguistically Appropriate Services) will be retained again this year. The three state-added questions in the Adult survey tool are: Q37 – In creating my service plan my cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) Q38 – Services were provided in a language I understood. Q39 – In creating my service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) The MHSIP YSS-F focuses on the following seven domain areas: (1) Service Accessibility; (2) Participation in Treatment Planning; (3) Cultural Sensitivity; (4) Satisfaction with Services; (5) Outcomes; (6) Social Connectedness; and (7) Improved Functioning. Additional questions solicit information about the youth‟s criminal justice contact and school attendance. Questions are scored with a five-point Likert Scale where 5=Strongly Agree, 4=Agree, 3=Neutral, 2=Disagree, and 1=Strongly Disagree. A Not Applicable option is also available if the question does not apply. As with the Adult tool, in addition to the state-added question related to cultural sensitivity, two state-added questions introduced in the 2011 YSS-F tool to gather information specific to National CLAS Standards (Culturally and Linguistically Appropriate Services) will be retained again this year. The three state-added questions in the YSS-F tool are: Q29 – In creating my child‟s service plan our cultural preferences were included (For example: values, traditions, beliefs, race, language, etc.) Q30 – Services were provided in a language my child understood. Q31 – In creating my child‟s service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) The RBHA and/or provider may pre-print the following information on the survey tool prior to distribution to client:  RBHA Name  Provider Name  Survey Id  Provider Facility ID  Behavioral Health Category  Entitlement Status (Title XIX or XXI)  Distribution Method (Home, Clinic, or Phone) 42 2013 Consumer Survey Report  Client enrollment with DDD Both survey tools have four main sections: (1) demographic section, (2) MHSIP survey questions, (3) state-added questions, and (4) open-ended qualitative section. The demographic section provides descriptive information about the consumer‟s age, gender, race, ethnicity, and relationship of the person completing the survey to the service recipient. The second section of the survey contains the MHSIP standardized questions. They serve as benchmark tools to compare consumer perception of behavioral health systems across the nation. The third section consists of state-added questions to elicit consumer input regarding the inclusion of cultural preferences in the consumer‟s treatment planning and to provide information relative to the National CLAS Standards. The fourth and final section of the survey contains open-ended questions to solicit consumer comments. Two questions are asked of consumers – focusing on identifying what has been most helpful with their services, and what the consumer believes would improve services. The section entitled other comments is intended to provide consumers with an additional area on the survey to provide open-ended feedback on any issue. Confidentiality The front page of the survey questionnaire addresses confidentiality of the responses. Survey results are aggregated and not presented at an individual consumer level. The providers will never have access to completed surveys or individual survey results. Thematic analysis is conducted on written comments. Non-mandatory (Voluntary) The survey questionnaire likewise informs the respondent that participation in the survey is voluntary; every individual can choose to participate or not. It also notifies the respondent that non-participation will not affect the services they are currently or will receive in the future. Participating in the survey is one way the consumer may help improve the system they use. Languages The survey will be available to consumers in English and Spanish languages. Each survey form will be printed in English on one side and Spanish on the reverse side. For consumers with limited English proficiency and speak a language other than Spanish, the RBHA and/or the provider administering the survey will extend their best effort to translate the survey in the consumer‟s preferred language by utilizing the Language Line or other translation/interpretation services officially utilized by the RBHA or their provider. The extent of assistance provided in language translation should not attempt to define what the question means. 43 2013 Consumer Survey Report Sampling Design Sample Frame The sample frame refers to the population eligible to take the survey. This is the pool from which ADHS/DBHS will randomly select the sampled population. ADHS/DBHS will create an adult and a youth sample from the CIS database for each GSA. The adult clients may be enrolled in any program: General Mental Health, Substance Abuse, or Seriously Mentally Ill. Clients turning age 18 between October 1, 2012 and the end of the survey period, May 24, 2013, will be filtered out of the sample frame. The sample frame will be composed of: (a) Client must have an open Episode of Care (EOC) in FY2013; (b) Client must be Title XIX or Title XXI eligible in FY2013; (c) Client must have received a mental health service other than inpatient, transportation, laboratory and/or radiology services, or crisis; (d) The service must have occurred in the previous 6 months. Drawing of sample ADHS/DBHS will create a random sample and provide each GSA with a list of 500 adult and 400 to 500 youth clients to be surveyed. Administering survey ADHS/DBHS will provide the RBHAs with a list of randomly selected clients. The RBHA will duplicate the Survey ID number on the survey tool to be offered that specific consumer. To protect the confidentiality of the consumer, none of the parties privy to this information (DBHS, RBHA, Provider) will link the survey responses to consumers. ADHS/DBHS will provide the RBHAs with a database to store the survey results. The RBHA will disseminate the surveys; track which consumers have completed the survey; track why surveys were not completed; enter the survey data into the provided database; and submit to ADHS/DBHS: the completed paper surveys, the database containing all survey results, the database detailing which consumers completed a survey and for every consumer that did not complete a survey, the reason the survey was not completed. The RBHAs will enter data into the two ADHS/DBHS provided databases but the RBHAs will not alter the databases in any way. The provider agency is responsible for identifying the specific provider location or site from which the consumer is presently receiving services. Survey Methodology 44 2013 Consumer Survey Report Distribution Method The primary distribution method is a non-clinical staff at the provider office (i.e., clinic) handing the survey questionnaire to the consumer. As the consumer checks in for their appointment, s/he will be provided with a copy of the survey questionnaire to complete. If the consumer agrees to participate, s/he will be requested to complete the survey prior to his/her appointment. If the consumer is unable to complete the questionnaire, s/he will be allowed to finish it on site after the appointment or be provided with an addressed, stamped envelope to mail the survey in if they did not have time to complete it in the office. A drop box will be provided on site for completed surveys. Additionally, a specific area at the provider office will be designated for completing the survey. If the individual randomly selected has a scheduled appointment at home during the survey window, the provider staff will bring the survey questionnaire at the appointment date. If the consumer agrees to participate, s/he will be advised to complete the survey after the staff leaves and to mail the completed questionnaire to the RBHA using the pre-addressed, stamped envelope provided with the survey. If the individual does not have an appointment during the survey window, a non-clinical staff at the RBHA may conduct the survey over the phone. As an alternative, a non-clinical staff at the provider site may contact the consumer by phone to ask for their participation in the survey, offering a return envelope for the completed survey to be mailed. All return envelopes provided as a means for the consumer to submit their completed survey must be addressed to the RBHA. A check box in the questionnaire will be used to track the distribution method. The adult survey will be administered to the adult consumer. If the individual requests assistance, a guardian may complete the questionnaire on the consumer‟s behalf. The YSS-F will be administered to the parent/guardian of the child receiving services. If the parent or guardian is not at the appointment, then the survey will not be provided. List of Survey Clients ADHS/DBHS will provide each RBHA with a list of survey consumers to track those who complete a survey or the reason they did not complete a survey. The list of survey consumers will contain fields for:           45 RBHA ID Contractor ID Survey ID AHCCCS ID Last Name First Name Date of Birth Sex Survey Offered Date Survey Completed Date 2013 Consumer Survey Report    Reason Not Completed Provider ID Client contact phone number The RBHAs will submit their lists of survey clients to ADHS/DBHS bi-weekly throughout the survey administration period to keep ADHS/DBHS informed regarding the status of response rates and reasons for non-participation of sampled consumers. Due dates for submission of biweekly lists are included in the Timeline Attachment to this Protocol. Since this document contains protected health information, it will be treated as a confidential document. ADHS/DBHS Post-administration Follow-up During the survey administration period, RBHAs will attempt to achieve participation of all consumers on its list of survey clients. Every effort will be made to separate service delivery from sample collection. At the end of the administration period, RBHAs will provide ADHS/DBHS with the completed list of survey consumers indicating which consumers completed the survey and which did not, with the reasons and a contact phone number for those who did not participate. At that point, the RBHA will delete the list of survey consumers. RBHAs will submit to ADHS/DBHS the completed paper surveys, retaining copies for reference, and databases containing survey results by July 26, 2013. Tribal RBHAs participate in the Annual Consumer Survey by way of a convenience sampling of their enrolled consumers. These entities will be provided a master survey tool for each population that can be copied for distribution. Tribal RBHAs will submit to ADHS/DBHS the completed paper surveys, retaining copies for reference, by July 26, 2013. ADHS/DBHS will enter the survey data collected at the tribal RBHAs into an SPSS database and provide results to each Tribal BHAS in Excel spreadsheets. Survey Administration Survey Timeframe The survey will be administered for a period of eight weeks: April 1 through May 24, 2013. Roles and Responsibilities ADHS/DBHS is responsible for the statewide oversight of the survey administration to ensure consistent implementation of the survey protocol. The protocol, client sample, survey instruments, and survey results database will be created by ADHS/DBHS. ADHS/DBHS will 46 2013 Consumer Survey Report run the analysis to determine RBHA survey results and provide the results to each RBHA from which the RBHAs will write their reports. ADHS/DBHS will provide technical assistance throughout the survey process. Periodic monitoring, training, timelines, and use of checklist will be utilized to guide the T/RBHAs on critical points in the process. The T/RBHAs have the primary responsibility for ensuring that the protocol is precisely followed within their geographic regions. Direct oversight and assistance will be provided by the T/RBHAs to their providers. The T/RBHAs will ensure that the providers are appropriately trained and prepared to administer the survey. The RBHAs will enter data into the two ADHS/DBHS provided databases but the RBHAs will not alter the databases in any way including creating tables, changing the names of tables or fields, or creating a different value coding scheme. Each provider agency is primarily responsible for each of its sites in which the survey is to be administered. Each site will maintain all necessary materials for survey administration. At each site, a drop box and a designated area will be provided for consumers to complete the survey. Providers will also be responsible for the day-to-day operations – including having the survey tools, materials for completing the survey (pens, pencils, clipboards), envelopes for return of the survey if needed, assigned resources for administration and collection of data for the survey. Non-Randomly Selected Consumers (Walk-in Requests) ADHS/DBHS does not require the RBHAs to extend survey participation efforts beyond the randomly selected populations but at the individual discretion of each RBHA/GSA, the RBHA may furnish its provider agencies with survey forms to give consumers who express a desire to participate in the survey but whose names do not appear on the lists of survey consumers (i.e., the consumer was not randomly selected). However, to maintain the scientific rigor of the protocol, the survey questionnaire completed by this group of respondents will be tracked separately by the RBHA. Names of individuals who belong to this group will not be added to the survey client list. This group will be tracked in some other ways as described in the succeeding paragraph. If the RBHA elects to collect responses from non-selected consumers, the same protocol will be followed for this group of respondents but the staff administering the survey will ensure that these “voluntary” surveys are kept separate from those on the lists of survey clients (randomly selected consumers). Several control measures will be used. First, consumers that are not randomly selected to participate in the survey (i.e., walk-in respondents) will be given a copy of the survey questionnaire with the pre-filled section of the questionnaire not completed. Second, the survey questionnaire that will be used for non-randomly selected consumers will be colorcoded. (Note: Surveys for the random sample will be printed on white paper.) RBHAs will enter the responses on the voluntary surveys into the same database as the results of the selected respondents, using the correct indicator, “C” for control (selected) respondents and “V” for voluntary (non-selected) respondents. 47 2013 Consumer Survey Report Pre-Survey Activities Notification to Consumers about the Survey To encourage greater participation, efforts will be made to inform consumers in advance about the survey. T/RBHAs and providers will be encouraged to utilize all or a combination of any of the following media: flyers, posters, website announcements, or other promotional materials. A staff member or members at each provider site will be assigned to work on the survey. In addition to daily survey administrative duties, a component of the staff member‟s role will include assisting consumers with the survey if necessary. Assistance may include: reading the survey to individuals unable to read, explaining the Likert scale used for scoring answers, emphasizing confidentiality of the survey, or ensuring consumers that participation in the survey is voluntary. Staff will be allowed to provide administrative assistance to the consumer, or provide encouragement to participate in the survey process. However, staff will not be able to explain the meaning of particular questions or provide interpretations on what particular questions mean. Data Management and Reporting Scoring Protocol The scoring protocol that will be utilized for evaluating the domain areas within the survey are as follows: 1. Recode ratings of „not applicable‟ as missing values 2. For each survey, exclude domains where more than one-third of the domain questions are missing 3. Calculate the mean of the items for each respondent 4. Calculate the percent of scores that are greater than or equal to 3.5 through 5 Technical assistance ADHS will provide technical assistance to the RBHAs as needed. Response Rate Calculation: The rate will be calculated for each population for each GSA using the formula: Response rate = A / B Where: A= Total number of surveys returned B= Total number of clients in sample that were contacted and asked to participate in the survey 48 2013 Consumer Survey Report Weighting methodology The statewide data will be weighted by GSA client population to compensate for the stratified sample collection. Weights will be applied to the survey data prior to any statewide data analysis. Dissemination of Findings The following methods have been identified to circulate findings: Reporting of survey results in management meetings – Executive Management, Quality Management, Human Rights Committees, Behavioral Health Planning Council, other consumer advocate groups, and other interest groups that may be identified  Dissemination of information by providers to their local communities  Having copies of the survey available at the provider sites  Publishing results of survey on ADHS, RBHA and provider websites  49 2013 Consumer Survey Report ATTACHMENT-C Raw Data 50 2013 Consumer Survey Report Arizona Department of Health Services Division of Behavioral Health Services 2012 Annual Consumer Survey Raw Data Assumptions for interpreting data: 1. Denominator – Number of responses to specific domain or question on completed/analyzed surveys. 2. Numerator – Number of positive responses to specific domain or question on completed/analyzed surveys. 3. RBHA – Specific numbers and percentages are based on actual valid survey returns. 4. Statewide RBHA numbers and percentages are based on weighted scores. 5. Tribal BHA scores are not included in statewide numbers/percentages due to Tribal BHA convenience sampling methodology. 6. Weighting: As a result, if weighting were not applied for the statewide performance calculation, a GSA with a relatively smaller population would be over represented and a GSA with a larger population would be under represented. The weight applied to each GSA is determined by the number of clients enrolled in the GSA. Weighting is utilized to correct the sample design when describing statewide data. Each GSA is provided a similar number of surveys to have completed but the GSAs have very different population sizes. The table below details the specific weighting factors used: RBHA Total Surveys RBHA Population RBHA Surveys Total Population Equation Weight 7 26 15 22 2 32 7 26 15 22 32 2 1,660 1,660 1,660 1,660 1,660 1,660 1,642 1,642 1,642 1,642 1,642 1,642 24,166 8,846 6,186 1,757 1,065 1,243 19,542 7,139 3,851 1,696 930 874 219 304 358 242 247 290 260 235 444 300 229 174 43,263 43,263 43,263 43,263 43,263 43,263 34,032 34,032 34,032 34,032 34,032 34,032 (1660*24166)/(219*43263) (1660*8846)/(304*43263) (1660*6186)/(358*43263) (1660*1757)/(242*43263) (1660*1065)/(247*43263) (1660*1243)/(290*43263) (1642*19542)/(260*34032) (1642*7139)/(235*34032) (1642*3851)/(444*34032) (1642*1696)/(300*34032) (1642*930)/(229*34032) (1642*874)/(174*34032) 4.23401 1.11652 0.66301 0.27858 0.16544 0.16446 3.62645 1.46573 0.41848 0.27277 0.19594 0.24235 51 2013 Consumer Survey Report 2013 MHSIP Consumer Survey Numbers Used in Calculation of Performance ADULT SURVEY RESULTS BY T/RBHA & MHSIP DOMAIN Domain: General Satisfaction T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 356 244 290 241 303 211 1,623 25 136 314 Numerator 316 221 255 220 263 194 1,464 24 112 288 Positive Response Percentage 89% 91% 88% 91% 87% 92% 90% 96% 82% 92% 80 75 94% Denominator 355 242 289 242 302 210 1,617 25 136 316 Numerator 302 214 255 225 239 187 1,399 22 113 285 Positive Response Percentage 85% 88% 88% 93% 79% 89% 87% 88% 83% 90% 77 56 73% Numerator Positive Response Domain: Service Access T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Participation in Treatment Planning T/RBHA 52 Denominator 2013 Consumer Survey Report 349 231 283 238 284 192 1,512 26 131 304 326 210 263 220 254 174 1,376 24 115 280 Percentage 93% 91% 93% 92% 89% 91% 91% 92% 88% 92% 65 48 74% Denominator 350 238 287 239 298 206 1,590 25 137 314 Numerator 329 217 264 226 258 188 1,444 22 121 285 Positive Response Percentage 94% 91% 92% 95% 87% 91% 91% 88% 88% 91% 74 52 70% Denominator 344 232 277 237 281 196 1,522 23 136 306 75 Numerator 245 169 218 181 199 140 1,092 18 98 247 54 Positive Response Percentage 71% 73% 79% 76% 71% 71% 72% 78% 72% 81% 72% GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Service Quality & Appropriateness T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Domain: Outcomes T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain 53 2013 Consumer Survey Report Apache Domain: Improved Functioning T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 350 235 282 239 283 201 1,551 24 137 304 Numerator 247 167 210 168 191 144 1,096 21 98 239 Positive Response Percentage 71% 71% 75% 70% 68% 72% 71% 88% 72% 79% 78 58 74% Denominator 352 244 281 237 283 200 1,549 24 135 308 Numerator 281 191 221 194 209 154 1,194 21 104 262 Positive Response Percentage 80% 78% 79% 82% 74% 77% 77% 88% 77% 85% 77 68 88% Domain: Social Connectedness T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache STATEWIDE RESULTS FOR EACH QUESTION IN DOMAIN General Satisfaction: Q1 Q2 Q3 Service Access Q4 54 Denominator 1,618 1,599 1,600 Numerator 1,474 1,408 1,444 Positive Response Percentage 91% 88% 90% 1,606 1,332 83% 2013 Consumer Survey Report Q5 Q6 Q7 Q8 Q9 1,388 1,196 1,417 1,344 1,184 87% 77% 88% 85% 77% Participation in Treatment Planning Q11 1,570 Q17 1,561 1,404 1,285 89% 82% Service Quality & Appropriateness Q10 1,570 Q12 1,581 Q13 1,598 Q14 1,569 Q15 1,562 Q16 1,589 Q18 1,549 Q19 1,549 Q20 1,535 1,384 1,326 1,417 1,362 1,287 1,456 1,309 1,304 1,285 88% 84% 89% 87% 82% 92% 85% 84% 84% Outcomes Q21 Q22 Q23 Q24 Q25 Q26 Q27 Q28 1,572 1,577 1,543 1,506 1,545 1,234 1,434 1,558 1,220 1,196 1,109 1,101 1,039 731 971 1,028 78% 76% 72% 73% 67% 59% 68% 66% Improved Functioning Q28 Q29 Q30 Q31 Q32 1,558 1,552 1,549 1,547 1,545 1,028 1,126 1,150 1,080 1,039 66% 73% 74% 70% 67% Social Connectedness Q33 Q34 Q35 Q36 1,547 1,552 1,536 1,559 1,203 1,233 1,003 1,258 78% 79% 65% 81% 55 1,599 1,553 1,605 1,589 1,543 2013 Consumer Survey Report 2013 MHSIP Consumer Survey Numbers Used in Calculation of Performance YOUTH SERVICES SURVEY FOR FAMILIES RESULTS BY T/RBHA & MHSIP DOMAIN Domain: General Satisfaction T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 431 173 228 298 232 251 1,599 19 44 128 21 Numerator 368 153 186 261 197 201 1,316 17 43 115 16 Positive Response Percentage 85% 88% 82% 88% 85% 80% 82% 90% 98% 90% 76% Denominator 428 172 225 297 225 244 1,561 19 44 128 21 Numerator 398 156 207 270 187 197 1,307 17 42 114 17 Positive Response Percentage 93% 91% 92% 91% 83% 81% 84% 90% 96% 89% 81% Numerator 409 161 Positive Response Percentage 96% 93% Domain: Service Access T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Participation in Treatment Planning T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 56 Denominator 426 173 2013 Consumer Survey Report GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache 226 295 233 248 1,586 18 44 125 21 209 275 218 228 1,473 14 40 112 18 93% 93% 94% 92% 93% 78% 91% 90% 86% Denominator 409 168 224 291 223 237 1,521 19 42 127 21 Numerator 397 160 214 277 213 225 1,451 18 42 121 20 Positive Response Percentage 97% 95% 96% 95% 96% 95% 95% 95% 100% 95% 95% Numerator 288 128 150 219 135 144 961 14 41 102 14 Positive Response Percentage 67% 74% 67% 75% 60% 58% 61% 74% 93% 80% 67% Numerator 305 Positive Response Percentage 71% Domain: Cultural Sensitivity T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Domain: Outcomes T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Denominator 432 173 225 292 227 247 1,575 19 44 127 21 Domain: Improved Functioning T/RBHA GSA 1 - NARBHA 57 Denominator 432 2013 Consumer Survey Report GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mtn 173 226 293 227 248 1,579 19 44 127 21 132 154 222 146 152 1,016 14 41 102 14 76% 68% 76% 64% 61% 64% 74% 93% 80% 67% Numerator 383 153 202 273 191 214 1,367 18 43 119 19 Positive Response Percentage 90% 91% 89% 92% 85% 86% 87% 95% 98% 92% 91% Domain: Social Connectedness T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA GSA 6 - Magellan Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mtn Denominator 428 169 227 296 226 248 1,576 19 44 129 21 STATEWIDE RESULTS FOR EACH QUESTION IN DOMAIN General Satisfaction: Q1 Q4 Q5 Q7 Q10 Q11 Denominator 1,593 1,588 1,542 1,579 1,574 1,570 Service Access Q8 1,589 Q9 1,584 Participation in Treatment Planning Q2 1,567 Q3 1,569 Q6 58 1,558 Numerator 1,353 1,295 1,230 1,275 1,255 1,161 Positive Response Percentage 85% 82% 80% 81% 80% 74% 1,234 1,302 78% 82% 1,413 1,449 90% 92% 94% 1,466 2013 Consumer Survey Report Cultural Sensitivity Q12 Q13 Q14 Q15 1,596 1,503 1,575 1,505 1,500 1,363 1,512 1,353 94% 91% 96% 90% Outcomes Q16 Q17 Q18 Q19 Q20 Q21 Q22 1,577 1,566 1,570 1,575 1,583 1,574 1,555 1,025 994 1,046 981 924 881 959 65% 64% 67% 62% 58% 56% 62% Improved Functioning Q16 Q17 Q18 Q19 Q20 Q22 1,577 1,566 1,570 1,575 1,583 1,555 1,025 994 1,046 981 924 959 65% 64% 67% 62% 58% 62% Social Connectedness Q23 Q24 Q25 Q26 1,585 1,579 1,573 1,563 1,345 1,378 1,275 1,294 85% 87% 81% 83% CLAS (New in 2011) Q30 Q31 1,526 804 1,478 640 97% 80% 59 2013 Consumer Survey Report ATTACHMENT-D 60 2013 Consumer Survey Report 2013 Adult Demographics 2013 Adult Consumer Survey, Percent of Positive Response by Domain and Subgroup General Satisfaction Subgroup N Service Access Service Quality and Appropriateness Participation in Treatment Planning Outcomes Improved Functioning Social Connectedness % N % N % N % N % N % N % 604 816 93.1% 88.3% 578 783 89.3% 85.2% 566 829 90.7% 90.6% 562 786 92.6% 90.2% 477 584 78.1% 67.4% 470 598 75.4% 67.8% 489 671 79.0% 76.3% 93 164 522 570 59 10 89.4% 80.0% 88.6% 94.1% 98.3% 100% 91 172 511 514 58 10 88.3% 86.0% 85.7% 85.2% 95.1% 100% 74 186 518 559 56 10 76.3% 92.1% 88.2% 93.9% 98.2% 100% 86 170 495 524 53 10 88.7% 90.9% 88.6% 93.1% 96.4% 100% 69 155 391 391 46 9 74.2% 77.5% 69.3% 70.5% 79.3% 90.0% 62 148 400 395 47 9 64.6% 74.0% 71.7% 67.2% 79.7% 90.0% 86 150 434 434 54 8 87.8% 77.3% 76.4% 75.0% 88.5% 80.0% 1,017 104 13 90.0% 89.7% 100% 975 95 13 86.4% 82.6% 100% 1,015 99 13 91.6% 86.1% 100% 993 96 13 91.1% 87.3% 100% 759 59 12 71.3% 57.8% 92.3% 771 62 11 70.7% 58.5% 84.6% 835 73 11 76.6% 65.8% 84.6% 27 93.1% 21 29 96.7% 27 93.1% 20 69.0% 13 44.8% 22 73.3% 7 45 100% 88.2% 7 41 100% 80.4% 7 46 100% 90.2% 7 44 100% 95.7% 6 42 85.7% 85.7% 5 45 71.4% 97.8% 6 37 85.7% 74.0% 403 893 91.6% 89.4% 386 849 88.9% 85.4% 377 895 88.5% 91.8% 343 872 88.9% 91.3% 314 658 76.6% 70.4% 286 680 69.4% 70.5% 308 743 78.0% 76.3% 40 45 119 159 1,049 78.4% 80.4% 91.5% 90.9% 90.7% 41 48 119 146 998 78.8% 87.3% 92.2% 83.4% 86.6% 39 51 117 158 1,033 76.5% 92.7% 94.4% 90.3% 91.3% 42 43 102 148 1,008 82.4% 78.2% 91.9% 86.5% 93.5% 37 31 98 110 782 78.7% 67.4% 76.6% 67.9% 71.4% 36 40 77 118 783 76.6% 78.4% 64.2% 70.2% 70.2% 38 45 95 121 855 88.4% 84.9% 79.8% 72.5% 76.1% 894 567 90.0% 90.6% 845 550 85.3% 88.6% 890 550 90.7% 91.1% 861 511 92.1% 89.0% 668 421 70.5% 73.7% 684 408 71.4% 69.3% 711 481 74.2% 81.9% Gender Male Female Age Group 18-21 22-30 31-45 46-65 66-75 76+ Race White only African American only Asian only Am Indian/Al Native only Nat Hawaiian/Pacific Islander only Multiple Race Ethnicity Hispanic or Latino Not Hispanic or Latino Length of Services 0-6 months 7-11 months 1-2 years 3-5 years 5 years + Program SMI Non-SMI 61 72.4% 2013 Consumer Survey Report ATTACHMENT-E 62 2013 Consumer Survey Report 2013 Adult Line Item 2013 Adult Consumer Survey, Statewide Percent of Positive Response by Line Item (Line specific numbers are based on actual valid survey returns. Domain percentages are based on weighted scores.) Survey Item General Satisfaction: 1,464 90.2% 1. I like the services that I received here. 2. If I had other choices, I would still get services from this agency. 3. I would recommend this agency to a friend or family member. Service Access: 1,399 86.5% 4. The location of services was convenient (parking, public transportation, distance, etc.) 5 Staff were willing to see me as often as I felt it was necessary.. 6. Staff returned my call in 24 hours. 7. Services were available at times that were good for me. 8. I was able to get all the services I thought I needed. 9. I was able to see a psychiatrist when I wanted to. Participation in Treatment Planning 1,376 91.0% 11. I felt comfortable asking questions about my treatment and medication. 17. I, not staff, decided my treatment goals. Service Quality and Appropriateness 1,444 90.8% 10. Staff here believe that I can grow, change and recover. 12. I feel free to complain. 13. I was given information about my rights. 14. Staff encouraged me to take responsibility for how I live my life. 15. Staff helped me to understand what side effects to watch out for. 16 Staff respected my wishes about who is and who is not to be given information about my treatment.. 18. Staff were sensitive to my cultural background (For example: values, traditions, beliefs, race, language, etc.) 19. Staff helped me obtain the information I needed so that I could take charge of managing my illness. 20. I was encouraged to use consumer-run programs (support groups, drop-in centers, crisis phone line, etc.) Outcomes 1,092 71.7% 21. I deal more effectively with daily problems. 22. I am better able to control my life. 23. I am better able to deal with crisis. 24 I am getting along better with my family. 25. I do better in social situations. 26. I do better in school and/or work 27. My housing situation has improved 28. My symptoms are not bothering me as much. Improved Functioning 28. My symptoms are not bothering me as much. 29. I do things that are more meaningful to me. 30. I am better able to take care of my needs. 31. I am better able to handle things when they go wrong. 32 I am better able to do things that I want to do. Social Connectedness 33. I am happy with the friendships I have. 34. I have people with whom I can do enjoyable things. 35. I feel I belong in my community. 36. In a crisis, I would have the support I need from family or friends. 63 1,096 1,194 Number Percent 1,474 1,408 1,444 91.1% 88.0% 90.3% 1,332 1,388 1,196 1,417 1,344 1,184 83.0% 86.8% 77.0% 88.3% 84.6% 76.7% 1,404 1,285 89.4% 82.3% 1,384 1,326 1,417 1,362 1,287 88.2% 83.8% 88.6% 86.8% 82.4% 1,456 91.6% 1,309 84.5% 1,304 84.2% 1,285 83.7% 1,220 1,196 1,109 1,101 1,039 731 971 1,028 77.6% 75.8% 71.8% 73.1% 67.2% 59.2% 67.7% 66.0% 70.6% 1,028 1,126 1,150 1,080 1,039 66.0% 72.6% 74.3% 69.8% 67.3% 1,203 1,233 1,003 1,258 77.8% 79.4% 65.3% 80.7% 77.1% 2013 Consumer Survey Report ATTACHMENT-F 64 2013 Consumer Survey Report 2013 YSS-F Demographics 2013 YSS-F, Percent of Positive Response by Domain and Subgroup Subgroup Gender Male Female Age Group 0-4 5-12 13-17 Race White only African American only Asian only Am Indian/Al Native only Nat Hawaiian/Pacific Islander only Multiple Race Ethnicity Hispanic or Latino Not Hispanic or Latino Length of Services 0-6 months 7-11 months 1-2 years 3-5 years 5 years + CFT Yes No 65 General Satisfaction Service Access Cultural Competency Participation in Treatment Planning Outcomes Improved Functioning Social Connectedness N % N % N % N % N % N % N % 929 383 82.7% 81.5% 917 386 83.7% 83.9% 1,027 420 95.7% 94.4% 1043 424 93.2% 92.0% 690 266 62.2% 57.7% 730 284 65.6% 61.6% 969 393 87.3% 85.4% 1 148 1,168 100% 84.1% 82.2% 1 148 1,158 100% 87.6% 83.3% 1 165 1,285 100% 95.4% 95.3% 1 167 1304 100% 96.0% 92.5% 1 121 839 50.0% 69.1% 60.0% 1 131 883 50.0% 74.4% 63.0% 1 151 1,215 100% 86.3% 86.8% 877 114 5 40 80.2% 89.8% 100% 85.1% 882 110 5 39 82.3% 87.3% 100% 88.6% 975 118 5 45 95.2% 96.7% 100% 97.8% 1005 124 5 41 92.6% 98.4% 100% 87.2% 626 79 1 36 58.1% 63.2% 20.0% 76.6% 669 85 1 36 61.8% 67.5% 20.0% 76.6% 939 104 5 45 86.9% 83.9% 100% 95.7% 9 60 100% 80.0% 9 60 100% 85.7% 9 67 100% 89.3% 7 59 87.5% 84.3% 7 45 77.8% 64.3% 7 48 77.8% 68.6% 8 58 100% 77.3% 470 774 87.2% 80.0% 445 783 84.8% 82.6% 513 856 95.9% 95.0% 508 884 95.0% 92.3% 331 566 62.3% 59.2% 342 610 64.4% 63.5% 466 824 87.9% 86.6% 74 94 141 259 705 79.6% 87.0% 75.8% 83.0% 83.3% 74 97 152 260 682 80.4% 93.3% 82.6% 84.4% 82.7% 90 98 169 285 763 98.9% 96.1% 93.9% 96.0% 95.4% 90 105 176 277 785 97.8% 93.8% 94.6% 91.7% 93.3% 45 58 113 180 541 48.4% 56.3% 61.1% 57.7% 64.5% 53 64 117 197 562 57.0% 62.1% 62.9% 63.1% 66.7% 71 91 144 264 758 83.5% 82.7% 78.3% 85.4% 90.0% 967 286 85.3% 77.1% 932 302 84.3% 82.1% 1,045 327 96.0% 95.1% 1,051 343 94.2% 91.2% 683 221 61.4% 59.2% 719 237 64.5% 63.5% 979 309 88.0% 83.7% 2013 Consumer Survey Report ATTACHMENT-G 66 2013 Consumer Survey Report 2013 YSS-F Line Item 2013 YSSF, Statewide Percent of Positive Response by Line Item (Line specific numbers are based on actual valid survey returns. Domain percentages are based on weighted scores.) Survey Item General Satisfaction: 1. Overall, I am satisfied with the services my child received. 4. The people helping my child stuck with us no matter what. 5. I felt my child had someone to talk to when he/she was troubled. 7. The services my child and/or family received were right for us. 10. My family got the help we wanted for my child. 11. My family got as much help as we needed for my child. Service Access 8. The location of services was convenient for us. 9. Services were available at times that were convenient for us. Participation in Treatment Planning 2. I helped to choose my child‟s services. 3. I helped to choose my child‟s treatment goals. 6. I participated in my child‟s treatment. Cultural Sensitivity 12 Staff treated me with respect. 13. Staff respected my family‟s religious/spiritual beliefs. 14. Staff spoke with me in a way that I understood. 15. Staff were sensitive to my cultural/ethnic background. Outcomes 16 My child is better at handling daily life. 17. My child gets along better with family members. 18. My child gets along better with friends and other people. 19. My child is doing better in school and/or work. 20. My child is better able to cope when things go wrong. 21. I am satisfied with our family life right now. 22. My child is better able to do things he or she wants to do. Improved Functioning 16 My child is better at handling daily life. 17. My child gets along better with family members. 18. My child gets along better with friends and other people. 19. My child is doing better in school and/or work. 20. My child is better able to cope when things go wrong. 22. My child is better able to do things he or she wants to do. Social Connectedness 23. I know people who will listen and understand me when I need to talk. 24. I have people that I am comfortable talking with about my child‟s problems. 25. In a crisis, I would have the support I need from family or friends. 26. I have people with whom I can do enjoyable things. 67 Number Percent 1,316 1,353 1,295 1,230 1,275 1,255 1,161 1,307 1,234 1,302 1,473 1,413 1,449 1,466 1,451 1,500 1,363 1,512 1,353 961 1,025 994 1,046 981 924 881 959 1,016 1,025 994 1,046 981 924 959 1,367 1,345 1,378 1,275 1,294 82.3% 84.9% 81.6% 79.8% 80.7% 79.7% 73.9% 83.8% 77.7% 82.2% 92.9% 90.2% 92.3% 94.1% 95.3% 94.0% 90.6% 96.0% 89.9% 61.0% 65.0% 63.5% 66.6% 62.3% 58.4% 56.0% 61.6% 64.3% 65.0% 63.5% 66.6% 62.3% 58.4% 61.6% 86.8% 84.9% 87.3% 81.0% 82.8% 2013 Consumer Survey Report ATTACHMENT-H 68 2013 Consumer Survey Report List of Acronyms 2013 Consumer Survey Report Acronyms Description ADHS/DBHS ASOC CBHS CFT CPSA CSOC CSOCPR CSR GMH GSA MHSIP MMWIA NARBHA NASMHPD NOMs NRI QM RBHA RSS SA SAMSHA SMI T/RBHA WSDSG YSS-F Arizona Department of Health Services/Division of Behavioral Health Services Adult System of Care Cenpatico Behavioral Health Services Child and Family Team Community Partnership of Southern Arizona Children‟s System of Care Child System of Care Practice Review Consumer Survey Report General Mental Health Geographical Service Area Mental Health Statistics Improvement Program Meet Me Where I Am Northern Arizona Regional Behavioral Health Authority National Association of State Mental Health Program Directors National Outcomes Measures National Research Institute Quality Management Regional Behavioral Health Authority Recovery Support Specialist Substance Abuse Substance Abuse and Mental Health Services Administration Serious Mental Illness Tribal/Regional Behavioral Health Authority Western States Decision Support Group Youth Services Survey for Families 69 2013 Consumer Survey Report APPENDIX-1 (See attached) Year 2013 Adult Consumer Survey (English and Spanish) 70 2013 Consumer Survey Report THIS SECTION MUST BE COMPLETED BY T/RBHA OR PROVIDER! Name of Service Agency: ____________________________________________________________________ T/RBHA NAME: _________________________ SMI  Program/Fund Source: Client’s Entitlement Status: Survey completed at: Facility ID: ______________ Non-SMI (GMH or SA)  TXIX  TXXI  Survey ID: ________________________ Home  Clinic  By Phone  Client enrolled in DDD: Yes _____ No _____ YEAR 2013 ADULT CONSUMER SURVEY Please help us improve our program by completing this survey about the services you have received in the last six months. We are interested in your honest opinion. All responses will be treated as confidential. Any personal information will be excluded in the presentation of the survey results. Your current and/or future services will not be affected if you decide not to participate in this survey. If you have already taken this survey during the months of April or May 2011, you do not need to complete it again. After you have completed the survey, please fold and drop it in the survey box before you leave the office today. Thank you. Use Pen or Pencil Please fill in the bubbles completely with your answers to the following questions: Information about the Person Receiving Services: Age: Sex:  Male  Female Ethnicity:  Hispanic or Latino  Not Hispanic or Latino Please check all applicable race categories: Race:  White  Black/African American  Native Hawaiian/Pacific Islander  Asian  American Indian/Alaska Native How long have you been receiving mental health and/or substance abuse services? (from any provider)  0 - 6 months  7 - 11 months  1 - 2 years  3 - 5 years  more than 5 years About the Person Completing This Survey: I am the person directly receiving services.  Yes  No If not, please check your relation to the person who is directly receiving services:  Parent/Guardian  Friend  Relative (Uncle, Aunt, Cousin, Grandparent, etc.) Please continue to answer questions on the next page. PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 71  Other Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable 2013 Consumer Survey Report 1. I like the services that I received here.       2. If I had other choices, I would still get services from this agency.       3. I would recommend this agency to a friend or family member. 4. The location of services was convenient (parking, public transportation, distance, etc.) 5. Staff were willing to see me as often as I felt it was necessary.                   6. Staff returned my call in 24 hours.       7. Services were available at times that were good for me.       8. I was able to get all the services I thought I needed.       9. I was able to see a psychiatrist when I wanted to.       10. Staff here believe that I can grow, change and recover.       11. I felt comfortable asking questions about my treatment and medication.       12. I feel free to complain.       13. I was given information about my rights.       14. Staff encouraged me to take responsibility for how I live my life.       15. Staff helped me to understand what side effects I should watch out for. 16. Staff respected my wishes about who is and who is not to be given information about my treatment. 17. I, not staff, decided my treatment goals. 18. Staff were sensitive to my cultural background. (For example: values, traditions, beliefs, race, language, etc.) 19. Staff helped me obtain the information I needed so that I could take charge of managing my illness. 20. I was encouraged to use consumer-run programs (support groups, drop-in centers, crisis phone line, etc.)                                     21. I deal more effectively with daily problems.       22. I am better able to control my life.       23. I am better able to deal with crisis.       24. I am getting along better with my family.       25. I do better in social situations.       26. I do better in school and/or work.       27. My housing situation has improved.       28. My symptoms are not bothering me as much.       29. I do things that are more meaningful to me.       30. I am better able to take care of my needs.       31. I am better able to handle things when they go wrong.       32. I am better able to do things that I want to do.       In order to provide the best possible behavioral health services, we need to know what you think about the services you received DURING THE LAST 6 MONTHS, the people who provided it, and the results. As a direct result of the services I received: Please continue to answer questions on the next page. 72 Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable 2013 Consumer Survey Report PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 33. I am happy with the friendships I have.       34. I have people with whom I can do enjoyable things.       35. I feel I belong in my community.       36. In a crisis, I would have the support I need from family or friends.                         For questions 33-36 please answer for relationships with persons other than your mental health provider(s) State-added Questions: 37. In creating my service plan my cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) 38. Services were provided in a language I understood. 39. In creating my service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) If you do not participate in tribal customs, please mark Not Applicable. Please answer the following questions to let us know how you are doing. 40. How long have you received mental health services from this provider?  Less than a year (less than 12 months) (Continue to Question 41)  1 year or more (at least 12 months) (Skip to Question 44) If you answered “Less than a year (less than 12 months)”, please complete questions 41 - 43. 41. Were you arrested since you began to receive mental health services? 42. Were you arrested during the 12 months prior to that? 43. Since you began to receive mental health services, have your encounters with police…  Yes  No  Yes  No  been reduced (for example, I have not been arrested, hassled by police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (I had no police encounters this year or last year) If you answered “1 year or more (at least 12 months)”, please complete questions 44 - 46. 44. Were you arrested during the last 12 months?  Yes 45. Were you arrested during the 12 months prior to that? 46. Over the last year, have your encounters with police…  No  Yes  No  been reduced (for example, I have not been arrested, hassled by police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (I had no police encounters this year or last year) Please continue to answer questions on the next page. 73 2013 Consumer Survey Report Please feel free to use the space provided below to comment on any of your answers. Also, if there are areas which were not covered by this questionnaire which you feel should have been, please write them in the comments section. Thank you for your time and cooperation in completing this questionnaire. What have been some of the most helpful things about the services you received over the last 6 months? ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ What would improve the services that you receive here? ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Please list any other comments you may want to share: ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Thank you for completing this questionnaire. 74 2013 Consumer Survey Report ¡¡ESTA SECCIÓN DEBE SER COMPLETADA POR LA RBHA O EL PROVEEDOR!! Nombre de Agencia de Servicio ____________________________________________________ NOMBRE de LA RBHA: ___________________________ Fuente de Programa/Fondo: SMI  No - SMI (GMH o SA)  TXIX  El Estado de Elegibilidad del Cliente: Encuesta tomada en: Casa  ID de la agencia o proveedor: ___________ TXXI  Clínica  Por teléfono  ID de la encuesta: _________________ Cliente inscrito en DDD: Si_____ No_____ ENCUESTA DE SERVICIOS PARA ADULTOS QUE RECIBIERON SERVICIOS DURANTE EL AÑO 2011 Por favor ayúdenos a mejorar nuestro programa completando ésta revisión sobre los servicios que usted ha recibido en los últimos seis meses. Estamos interesados en su opinión honesta. Todas las respuestas serán tratadas como confidenciales. Cualquier información personal será excluida en la presentación de los resultados ésta encuesta. Sus servicios actuales y/o futuros no se verán afectados si usted decide no participar en ésta encuesta. Si usted ha tomado ya ésta encuesta en los meses de abril o mayo de 2011, no tiene que completarla otra vez. Después de que usted haya finalizado, por favor doble la hoja y deposítela en la caja de encuestas, antes de usted salga de la oficina hoy. Gracias. Use pluma o lápiz Por favor rellene las burbujas completamente con sus respuestas a las siguientes preguntas: Información acerca de la persona que recibe los servicios: Edad: _______ Sexo:  Masculino  Femenino Pertenencia étnica:  Hispano o Latino  No Hispano o Latino Por favor compruebe todas las categorías de raza aplicables Raza:  Blanco  Negro/Africano Americano  Asiático  Nativo Americano / Nativo de Alaska  Nativo de Hawai/Isleño del Pacífico ¿Por cuánto tiempo ha estado recibiendo servicios de salud mental y/o de abuso de sustancias? (de cualquier proveedor )  0 - 6 meses  7 - 11 meses  1 - 2 años  3 - 5 años  más de 5 años Sobre la persona que completa ésta encuesta: Soy la persona que directamente recibe servicios: 75  Sí  No 2013 Consumer Survey Report Si no, por favor marque la relación con la persona que recibe directamente servicios:  Padre/Guardián  Amigo  Pariente (Tío, Tía, Primo, Abuelo, etc.)  Otro Por favor siga contestando preguntas en la siguiente página. Muy de acuerdo De acuerdo Neutro En Desacuerdo Muy en desacuerdo No Aplica POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 1. Me gustaron los servicios que recibí aquí. 2. Si yo tuviera otras opciones, yo todavía preferiría los servicios de esta agencia. 3. Yo recomendaría ésta agencia a un miembro de la familia o un amigo. 4. La ubicación de los servicios fue conveniente (estacionamiento, transporte público, distancia, etc.) 5. El personal quiso verme tan a menudo como sentí que era necesario.                               6. El personal devolvió mi llamada en 24 horas. 7. Los servicios estaban disponibles a las horas que eran convenientes para mí. 8. Recibí los servicios que pensé que necesitaba.                   9. Pude ver a un Psiquiatra cuando así lo solicité.       10. El personal aquí cree que puedo crecer, cambiar y recuperarme. 11. Me sentí cómodo haciendo preguntas acerca de mi tratamiento y medicación. 12. Me siento libre de presentar mis inquietudes sobre los servicios.                   13. Me dieron la información sobre mis derechos. 14. El personal me animó a tomar la responsabilidad de como vivo mi vida. 15. El personal me ayudó a entender acerca de los efectos secundarios sobre los que debo estar preparado. 16. El personal respetó mis deseos sobre a quién sí y a quien no deben dar información sobre mi tratamiento. 17. Yo, y no el personal, decidí las metas de mi tratamiento.                               18. El personal fue respetuoso de mis raíces culturales/étnicas. (Por ejemplo: mis valores, tradiciones, creencias, raza, lenguaje, etc.). 19. El personal me ayudó a obtener la información que necesité de modo que yo pudiera hacerme cargo de manejar mi enfermedad. 20. Fui animado a usar programas manejados por consumidores (grupos de                   Como un resultado directo de los servicios recibí: 21. Manejo de forma más eficaz los problemas diarios.       22. Soy más capaz de controlar mi vida.       23. Soy más capaz de lidiar con las crisis.       24. Me llevo mejor con los miembros de mi familia.       25. Me desenvuelvo mejor en situaciones sociales.       26. Tengo mejor desempeño en la escuela y/o trabajo.       27. Mi situación de vivienda ha mejorado.       A fin de proporcionar un mejor servicio en los cuidados de salud mental y de abuso de sustancias, tenemos que saber lo que usted piensa sobre los servicios que recibió DURANTE los 6 MESES PASADOS, la gente que lo proporcionó, y los resultados. apoyo, centros de ayuda informal, línea telefónica de crisis, etc.) 76 2013 Consumer Survey Report 28. Los síntomas no me molestan tanto como antes.       29. Hago cosas que son más significativas para mí.       30. Soy más capaz de ocuparme de mis necesidades.       31. Soy más capaz de manejar las cosas o situaciones cuando no van bien.       32. Soy más capaz de hacer cosas que quiero hacer.       Por favor siga contestando preguntas en la siguiente página. Muy de acuerdo De acuerdo Neutro En Desacuerdo Muy en desacuerdo No Aplica POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 33. Soy feliz con las amistades que tengo.       34. Tengo a personas con quienes puedo hacer cosas agradables.       35. Siento que pertenezco a mi comunidad.       36. En una crisis, yo tendría el apoyo que necesito de familia o amigos.       Preguntas adicionales del estado: 37. En la creación de mi plan de servicios, mis preferencias culturales fueron incluidas. (Por ejemplo; mis valores, tradiciones, creencias, raza, lenguaje, etc.). 38. Los servicios me fueron proporcionados en un lenguaje que entendí.             39. En la creación de mi plan de servicios las tradiciones y creencias de mi tribu fueron incluidas (Por ejemplo: me preguntaron si quería participar en sesiones de sanación tradicional). Si usted no participa de las tradiciones tribales, por favor marque la opción "No aplica".       Para las preguntas 33-36 por favor responda basado en las relaciones con personas diferentes de su proveedor (es) de salud mental. Por favor conteste las siguientes preguntas para hacernos saber cómo se encuentra. 40. ¿Durante cuánto tiempo ha recibido usted servicios de  Menos de un año (menos de 12 meses) salud mental de éste proveedor? (Continué con la pregunta 41)  1 año o más (al menos 12 meses) (Vaya a la pregunta 44) Si su respuesta es “menos de un año (menos de 12 meses)”, por favor complete las preguntas 41-43. 41. ¿Ha sido usted arrestado desde que comenzó a recibir  Sí  No servicios de salud mental? 42. ¿Ha sido usted arrestado durante los 12 meses  Sí  No anteriores a esto? 43. Desde que usted comenzó a recibir servicios de salud  han disminuido (por ejemplo, no he sido mental, ha tenido incidentes con la policía… arrestado, molestado por la policía, enviado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (yo no he tenido ningún incidente con la policía durante este año o el año pasado) Si su respuesta es “1 año o más (al menos 12 meses)”, por favor complete las preguntas 44 - 46. 77 2013 Consumer Survey Report 44. ¿Ha sido arrestado durante los 12 meses pasados? 45. ¿Ha sido arrestado durante los 12 meses anteriores a esto? 46. Durante el año pasado, sus incidentes con la policía …  Sí  No  Sí  No  han disminuido (por ejemplo, no he sido arrestado, molestado por policía, enviado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (yo no he tenido ningún encuentro con la policía durante este año o el año pasado) Por favor siga contestando preguntas en la siguiente página. 78 2013 Consumer Survey Report Por favor siéntase libre de usar el espacio proporcionado abajo para comentar sobre cualquiera de sus respuestas. También, si hay áreas que no fueron cubiertas por este cuestionario que usted siente que deberían haber sido incluidas, por favor escríbalas en la sección de comentarios. Gracias por su tiempo y cooperación en completar éste cuestionario. ¿Cuáles han sido algunas cosas más provechosas sobre los servicios que usted recibió durante los 6 meses pasados? ¿Qué mejoraría los servicios qué usted recibe aquí? Por favor escriba cualquier otro comentario que usted quiera compartir: Gracias por completar este cuestionario. 79 2013 Consumer Survey Report APPENDIX-2 (See attached) Year 2013 Youth Consumer Survey for Families (English and Spanish) 80 2013 Consumer Survey Report THIS SECTION MUST BE COMPLETED BY T/RBHA OR PROVIDER! Name of Service Agency: ____________________________________________________________________ T/RBHA NAME: ________________________ Client’s Entitlement Status: TXIX  Facility ID:_______________ TXXI  Survey ID: ____________________________ Survey completed at: Home  Clinic  By Phone  Client enrolled in DDD: Yes ______ No ______ YEAR 2013 YOUTH SERVICES SURVEY FOR FAMILIES Please help us improve our program by completing this survey about the services you have received in the last six months. We are interested in your honest opinion. All responses will be treated as confidential. Any personal information will be excluded in the presentation of the survey results. Your child and family’s current and/or future services will not be affected if you decide not to participate in this survey. If you have already taken this survey in the months of April or May 2011, then you do not need to complete it again. After you have completed the survey, please fold and drop it in the survey box before you leave the office today. Thank you. Use Pen or Pencil Please fill in the bubbles completely with your answers to the following questions: Information about the Person Receiving Services: Child’s Age: Child’s Sex:  Male Child’s Ethnicity:  Hispanic or Latino  Female  Not Hispanic or Latino Please check all applicable race categories: Child’s Race:  White  Black/African American  Native Hawaiian/Pacific Islander  Asian  American Indian/Alaska Native How long has your child been receiving mental health and/or substance abuse services? (from this provider)  0 - 6 months  7 - 11 months  1 - 2 years  3 - 5 years  more than 5 years  Parent/Guardian  Friend  Relative (Uncle, Aunt, Cousin, Grandparent, etc.)  Other Does your family have a Child and Family Team?  Yes  No Please check your relationship with the child: 81 2013 Consumer Survey Report Please continue to answer questions on the next page.PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable COMPLETELY 1. Overall, I am satisfied with the services my child received.       2. I helped to choose my child’s services.       3. I helped to choose my child’s treatment goals.       4. The people helping my child stuck with us no matter what.       5. I felt my child had someone to talk to when he/she was troubled.       6 I participated in my child’s treatment.       7. The services my child and/or family received were right for us.       8. The location of services was convenient for us.       9. Services were available at times that were convenient for us.       10. My family got the help we wanted for my child.       11. My family got as much help as we needed for my child.       12. Staff treated me with respect.       13. Staff respected my family’s religious/spiritual beliefs.       14. Staff spoke with me in a way that I understood. 15. Staff were sensitive to my cultural/ethnic background. (For example: values, traditions, beliefs, race, language, etc.)             16. My child is better at handling daily life.       17 My child gets along better with family members.       18. My child gets along better with friends and other people.       19. My child is doing better in school and/or work.       20. My child is better able to cope when things go wrong.       21 I am satisfied with our family life right now.       22. My child is better able to do things he or she wants to do.       Please help our agency make services better by answering some questions about the services your child received OVER THE LAST 6 MONTHS. As a result of the services my child and/or family received: As a result of the services my child and/or family received: please answer for relationships with persons other than your mental health provider(s). 23 I know people who will listen and understand me when I need to talk. 24. I have people that I am comfortable talking with about my child’s problems. 25 In a crisis, I would have the support I need from family or friends.                   26. I have people with whom I can do enjoyable things.       27. What has been the most helpful thing about the services you and your child received over the last 6 months? ________________________________________________________________________________________________ ________________________________________________________________________________________________ 28. What would improve the services here? _____________________________________________________________ ________________________________________________________________________________________________ 82 2013 Consumer Survey Report ________________________________________________________________________________________________ Please continue to answer questions on the next page. Agree I am Neutral Disagree Strongly Disagree Not Applicable State-added Questions: 29. In creating my child’s service plan our cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) 30. Services were provided in a language my child understood. 31. In creating my child’s service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) If you do not participate in tribal customs, please mark Not Applicable. Strongly Agree PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY                   Please answer the following questions to let us know how your child is doing. 32. Is your child currently living with you?  Yes  No 33. Has your child lived in any of the following places in the last 6 months? (CHECK ALL THAT APPLY)  With one or both parents  With another family member  Foster home  Therapeutic foster home  Crisis shelter  Homeless shelter  Group home  Residential treatment facility  Hospital  Local jail or detention facility  State correctional facility  Runaway/homeless/on the streets  Other (describe):________________________________________________________________________________ 34. In the last year, did your child see a medical doctor (or nurse) for a health check up or because he/she was sick? (Check one)  Yes, in a clinic or office  Yes, but only in a hospital emergency room  No  Do not remember 35. Is your child on medication for emotional/behavioral problems? 36. If yes, did the doctor or nurse tell you and/or your child what side effects to watch for? 37. Is your child still getting services from this provider? 38. How long did your child receive services from this provider?  Yes  No  Yes  No  Yes  No  Less than 1 month  1-5 Months  6 months to 1 year  More than 1 year (skip to Question 45) 39. Was your child arrested since beginning to receive mental health services? 40. Was your child arrested during the 12 months prior to that? 41. Since your child began to receive mental health services, have their encounters with the police…  Yes  No  Yes  No  been reduced (for example, they have not been arrested, hassled by the police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (They had no police encounters this year or last year) 83 2013 Consumer Survey Report 42. Was your child expelled or suspended from school since beginning services? 43. Was your child expelled or suspended from school during the 12 months prior to that?  Yes  No  Yes  No Please continue to answer questions on the next page. PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 44. Since starting to receive services, the number of days my child was in school is…  greater  about the same  less  does not apply (please select why this does not apply)  child did not have a problem with attendance before starting services  child is too young to be in school  child was expelled from school  child is home schooled  child dropped out of school  Other: 45. Was your child arrested during the last 12 months? 46. Was your child arrested during the 12 months prior to that? 47. Over the past year, have your child’s encounters with the police…  Yes  No  Yes  No  been reduced (for example, they have not been arrested, hassled by the police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (They had no police encounters this year or last year) 48. Was your child expelled or suspended from school during the last 12 months? 49. Was your child expelled or suspended from school during the 12 months prior to that? 50. Over the last year, the number of days my child was in school is…  Yes  No  Yes  No  greater  about the same  less  does not apply (please select why this does not apply)  child did not have a problem with attendance before starting services  child is too young to be in school  child was expelled from school  child is home schooled  child dropped out of school  Other: 84 2013 Consumer Survey Report Please continue to answer questions on the next page. Please feel free to use the space provided below to comment on any of your answers. Also, if there are areas which were not covered by this questionnaire which you feel should have been, please write them in the comments section. Thank you for your time and cooperation in completing this questionnaire. Please list any other comments you may want to share: ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ 85 2013 Consumer Survey Report ¡¡ESTA SECCIÓN DEBE SER COMPLETADA POR LA RBHA O EL PROVEEDOR!! Nombre de Agencia de Servicio __________________________________________ NOMBRE de la RBHA: __________________ TXIX  El Estado de Elegibilidad del Cliente: Encuesta tomada en : Casa  ID de agencia o proveedor: _____________ TXXI  Clínica  Por teléfono  ID de la encuesta: ___________________ Cliente inscrito en DDD: Si____ No_____ ENCUESTA PARA FAMILIAS DE JÓVENES QUE RECIBIERON SERVICIOS DURANTE EL AÑO 2013 Por favor ayúdenos a mejorar nuestro programa completando ésta encuesta sobre los servicios que usted ha recibido en los últimos seis meses. Estamos interesados en su opinión honesta. Todas las respuestas serán tratadas como confidenciales. Cualquier información personal será excluida en la presentación de los resultados de ésta encuesta. Los servicios actuales y/o futuros de su hijo(a) o la familia no se verán afectados si usted decide no participar en esta encuesta. Si usted ha tomado ya esta encuesta en los meses de abril o mayo de 2011, no tiene que completarla otra vez. Después de que usted haya finalizado, por favor doble la hoja y deposítela en la caja de encuestas, antes de usted salga de la oficina hoy. Gracias. Use pluma o lápiz Por favor rellene las burbujas completamente con sus respuestas a las preguntas siguientes: Información acerca de la persona que recibe los servicios: La Edad del Niño: El Sexo del Niño:  Masculino La Pertenencia étnica del Niño  Femenino  Hispano o Latino  No Hispano o Latino Por favor compruebe todas las categorías de raza aplicables: La Raza del Niño:  Blanco  Negro/Africano Americano  Asiático  Nativo Americano / Nativo de Alaska  Nativo de Hawai/Isleño del Pacífico ¿Por cuánto tiempo ha estado su hijo recibiendo servicios de salud mental y/o de abuso de sustancias? (De éste proveedor)  0 - 6 meses  7 - 11 meses  1 - 2 años  3 - 5 años  más de 5 años  Padre/Guardián  Amigo  Pariente (Tío, Tía, Primo, Abuelo, etc.) Otro ¿Tiene su familia un Equipo del Niño y la Familia?  Sí  No Por favor indique su relación con el niño: Por favor siga contestando preguntas en la siguiente página. POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 86 Muy De acuerdo De acuerdo Neutro En desacuerdo Muy en desacuerdo No Aplica 2013 Consumer Survey Report 1. En general, estoy satisfecho con los servicios que mi hijo recibió.       2. Ayudé elegir los servicios de mi hijo.       3. Ayudé elegir los objetivos de tratamiento de mi hijo.       4. La gente que ayuda a mi hijo nos apoyó sin falta.       5. Sentí que mi hijo tenía a alguien para dirigirse cuando estaba preocupado.       6. Participé en el tratamiento de mi hijo. 7. Los servicios que mi hijo y/o la familia recibieron fueron los apropiados para nosotros. 8. La ubicación de los servicios fue conveniente para nosotros. 9. Los servicios estuvieron disponibles en momentos en que fue conveniente para nosotros. 10. Mi familia recibió la ayuda que estábamos buscando para mi hijo.                               11. Mi familia recibió tanta ayuda como necesitábamos para mi hijo.       12. El personal me trató con respeto.       13. El personal respetó las creencias religiosas/espirituales de mi familia.       14. El personal habló conmigo de tal forma que entendí. 15. El personal fue respetuoso de mis raíces culturales/étnicas. (Por ejemplo: mis valores, tradiciones, creencias, raza, lenguaje, etc.).             Como resultado de los servicios que mi niño y/o la familia recibieron: 16. Mi hijo maneja de forma más eficaz los problemas diarios.       17. Mi hijo se lleva mejor con los otros miembros de la familia.       18. Mi hijo se lleva mejor con los amigos y otras personas.       19. A mi hijo le va mejor en la escuela y/o el trabajo.       20. Mi hijo está más dispuesto a enfrentar las situaciones difíciles.       21. Estoy satisfecho con nuestra vida familiar actualmente.       22. Mi hijo es más capaz de hacer las cosas que quiere hacer.       Por favor ayude a nuestra agencia a mejorar los servicios, contestando algunas preguntas sobre los servicios que su niño recibido DURANTE LOS ÚLTIMOS 6 MESES. Como resultado de los servicios que mi niño y/o la familia recibieron: por favor responda basado en las relaciones con personas diferentes de su proveedor(es) de salud mental. 23. Conozco gente que escuchará y me entenderá cuando necesito      hablar. 24. Tengo personas con las que me siento cómodo hablando acerca del      problema de mi hijo.      25. En una crisis, yo tendría el apoyo que necesito de mi familia o amigos.      26. Cuento con personas con quienes puedo hacer cosas agradables. 27. ¿Cuál ha sido la cosa más provechosa sobre los servicios usted y su hijo recibieron durante los 6 meses pasados? 28. ¿Qué mejoraría los servicios aquí? 87     2013 Consumer Survey Report Por favor siga contestando preguntas en la siguiente página. De acuerdo Neutro En desacuerdo Muy en desacuerdo No Aplica Pregunta adicional del estado: 29. En la creación del plan de servicios de mi hijo, nuestras preferencias culturales fueron incluidas. (Por ejemplo; mis valores, tradiciones, creencias, raza, lenguaje, etc.). 30. Los servicios fueron proporcionados en un lenguaje que mi hijo entendiera. 31. En la creación del plan de servicios de mi hijo nuestras tradiciones y creencias tribales fueron incluidas (Por ejemplo: se nos preguntó si queríamos participar en sesiones de sanación tradicional). Si usted no participa de las tradiciones tribales, por favor marque la opción "No aplica". Muy De acuerdo POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE                   Por favor conteste las siguientes preguntas, para informarnos acerca de cómo va su hijo. 32. ¿Vive actualmente su hijo con usted?  Sí 33. ¿Ha vivido su hijo en cualquiera de los siguientes sitios en los últimos 6 meses?  No (MARQUE TODAS LAS QUE APLICAN)  Con uno o ambos padres  Con otro miembro de familia  Familia adoptiva terapéutica  Refugio de crisis  Centro de tratamiento  Casa de Grupo  Cárcel local o instalación de detención  Otro (describa)  Instalación correccional 34. ¿Durante el año pasado, visitó su hijo un médico (o enfermera) para un control de salud o porque él/ella estaba enfermo? (Elija una de las opciones)  Familia adoptiva  Refugio para los sin hogar  Hospital  Escapó/o vive en las calles  Sí, en una clínica o oficina  Sí, pero sólo en un cuarto de emergencia de hospital  No  No recuerdo 35. ¿Está su hijo tomando medicamentos para problemas de conducta / emocionales? 36. ¿Si la respuesta es afirmativa, responda si el médico o la enfermera le explicaron con qué clase de efectos secundarios debe estar alerta? 37. ¿Todavía está su hijo recibiendo servicios de éste proveedor? 38. ¿Por cuánto tiempo recibió su hijo servicios de éste proveedor?  Sí  No  Sí  No  Sí  No  Menos de 1 mes  1-5 Meses  6 Meses a 1 año  Más de 1 año (vaya a la pregunta 45) 39. ¿Ha sido su hijo arrestado desde que comenzó a recibir servicios de salud mental? 40. ¿Estuvo arrestado su hijo durante los 12 meses anteriores a ésto? 41. Desde que su hijo comenzó a recibir servicios de salud mental, los incidentes con la policía...  Sí  No  Sí  No  han disminuido (por ejemplo, no ha sido arrestado, molestado por la policía, llevado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (no tuvo ningún incidente con la policía éste año o el año pasado) 88 2013 Consumer Survey Report 42. ¿Ha sido su hijo expulsado o suspendido de su escuela desde que comenzó a recibir los servicios? 43. ¿Fue su hijo expulsado o suspendido de su escuela durante los 12 meses anteriores a ésto?  Sí  No  Sí  No Por favor siga contestando preguntas en la siguiente página. POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 44. Desde que comenzó a recibir servicios, el número de días que mi hijo estuvo en la escuela fue…  mayor  el mismo  menos  no se aplica (por favor seleccione por qué éste no se aplica)  el niño no tenía un problema con la asistencia antes de los servicios iniciales  el niño es demasiado joven para estar en la escuela  el niño fue expulsado de la escuela  el niño está siendo enseñado en la casa  el niño abandonó la escuela 45. ¿Ha sido su hijo arrestado durante los 12 meses pasados? 46. ¿Estuvo arrestado su hijo durante los 12 meses antes a ésto? 47. Durante el año pasado, los incidentes de su niño con la policía…  Otro  Sí  No  Sí  No  han disminuido (por ejemplo, no ha sido arrestado, molestado por la policía, llevado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable ( no tuvo ningún encuentro con policía éste año o el año pasado) 48. ¿Fue expulsado su niño o suspendido de la escuela durante los 12 meses pasados? 49. ¿Fue expulsado su niño o suspendido de la escuela durante los 12 meses antes de esto? 50. Durante el año pasado, el número de días mi niño fue a la escuela era…  Sí  No  Sí  No  mayor  el mismo  menos  no se aplica (por favor seleccione por qué éste no se aplica)  El niño no tenía un problema con la asistencia antes de servicios iniciales  El niño es demasiado joven para estar en la escuela  El niño fue expulsado de la escuela  El niño está siendo enseñado en la casa  El niño abandonó la escuela  Otro 89 2013 Consumer Survey Report Por favor siga contestando preguntas en la siguiente página. Por favor siéntase libre de usar el espacio proporcionado abajo para comentar sobre cualquiera de sus respuestas. También, si hay áreas que no fueron cubiertas por éste cuestionario que usted siente deberían haber sido incluidas, por favor escríbalas en la sección de comentarios. Gracias por su tiempo y cooperación en completar éste cuestionario. Por favor escriba cualquier otro comentario que usted quiera compartir: 90