2014 Annual Consumer Survey Report December 2014 Office of Performance Improvement Arizona Department of Health Services Division of Behavioral Health Services Bureau of Quality and Integration 2014 Consumer Survey Report TABLE OF CONTENTS EXECUTIVE SUMMARY…………………………………………..……..………………………….4 INTRODUCTION……………………………………………………………...............................7 SURVEY SAMPLE AND RESPONSE RATES...……………………………………….………7 METHODOLOGY AND SURVEY ADMINISTRATION………………….…………...……....8 DATA ANALYSIS………………….…………………………………………………….………9 STATEWIDE SURVEY DATA LIMITATIONS……………………………...………………..10 ADULT SURVEY ANALYSIS AND RESULTS........................................................................10 YSS-F SURVEY ANALYSIS AND RESULTS...………………………………………………16 STATEWIDE IMPROVEMENT OPPORTUNITIES…………………………………………..21 RBHA SPECIFIC INITIATIVES……………………………………………………..................22 ADHS/DBHS INITIATIVES…………………………………………………….……...............23 CONCLUSION……………………………………………………………………….……...…..23 ATTACHMENTS Attachment A: 2014 Tribal Behavioral Health Authority Health Survey Results Attachment B: 2014 Statewide Consumer Survey Protocol Attachment C: 2014 Raw Data Attachment D: 2014 Adult Demographics Attachment E: 2014 Adult Line Item Attachment F: 2014 YSS-F Demographics Attachment G: 2014 YSS-F Line Item Attachment H: List of Acronyms 2 2014 Consumer Survey Report APPENDICES YEAR 2014 ADULT CONSUMER SURVEY (ENGLISH AND SPANISH) YEAR 2014 YOUTH CONSUMER SURVEY FOR FAMILIES (ENGLISH AND SPANISH) 3 2014 Consumer Survey Report I. Executive Summary The Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS), Tribal/Regional Behavioral Health Authorities (T/RBHAs) and contracted service providers jointly conduct statewide consumer satisfaction surveys each fiscal year. Each year, two surveys are administered based on the Substance Abuse and Mental Health Services Administration’s (SAMSHA’s) Mental Health Statistics Improvement Program (MHSIP) consumer surveys: The Adult Consumer Survey and The Youth Services Survey for Families (YSS-F). The surveys request independent feedback from Title XIX/XXI adults and families of youth receiving services through Arizona’s publicly funded behavioral health system. The surveys measure consumers’ perceptions of behavioral health services in relation to the following domains: General Satisfaction Access to Services Service Quality/Appropriateness Participation in Treatment Outcomes Cultural Sensitivity Improved Functioning Social Connectedness Within ADHS/DBHS, Consumer Survey results are: • Reviewed by the ADHS/DBHS Quality Management (QM) Committee • Used by the QM Committee as a critical source of information to drive system improvements through RBHA contracts and QM Plans • Used for certain Performance Incentive System calculations • Tracked and trended by Network development to determine network capacity by measuring member perception of behavioral health services in relation to the established survey domains. • Analyzed by the Office of Information Management and the Office of Performance improvement to identify and improve on performance in the various survey domains. Survey outcomes are also: • Disseminated to ADHS/DBHS stakeholders and consumers via the ADHS/DBHS website and Performance Dashboard; • Submitted to these national organizations as part of federal grant requirements for use in national reporting and evaluation: o The National Association of State Mental Health Program Directors’ (NASMHPD) o National Research Institute (NRI) o Western States Decision Support Group (WSDSG) o Substance Abuse and Mental Health Services Administration’s (SAMHSA) Mental Health Statistics Improvement Program (MHSIP) 4 2014 Consumer Survey Report The fiscal year 2014 survey was conducted during an 8-week period from March 31 through May 23, 2014. Adult and Youth survey results are summarized below. With regard to the statewide results, it should be noted that this is the first year ADHS/DBHS did not engage all its contractors throughout the state to take part in the survey. GSA 6, which has the largest population base of GSAs throughout the state, did not participate in the 2014 survey; explanation follows in Section II of this report. However, because GSA 6 was excluded, the statewide results were not weighted as they have been in previous years. Therefore, direct comparison of 2014 statewide scores with scores of prior years should be considered with caution. That being said, this year’s statewide domain scores show no statistically significant changes when compared to 2013 scores, with the exception of the Social Connectedness domain that increased 5 percentage points for the adult population. Adult Consumer Survey results: Statewide: The Service Access domain remained unchanged from 2013. The Participation in Treatment Planning, Service Quality & Appropriateness, Outcomes, and Improved Functioning domains showed increases of 1 to 2 percentage points. The Social Connectedness domain increased by 5 percentage points, a statistically significant increase. The General Satisfaction domain decreased by 2 percentage points. RBHA Specific: NARBHA (GSA 1) showed increases or decreases of 1 to 3 percentage points in all domains; none were statistically significant. Cenpatico’s (GSA 2, 3, 4) scores varied from -8 percentage points to +6 percentage points as compared to domain scores in 2013, with a statistically significant increase in GSA 3’s Social Connectedness domain and a statistically significant decrease in GSA 4’s Outcomes domain. GSA 4 experienced a decrease in all domains this year. CPSA’s (GSA 5) score for the General Satisfaction domain remained unchanged from 2013 and all other domains increased from 2 to 8 percentage points, with the 8 point increase in the Social Connectedness domain being statistically significant. YSS-F Consumer Survey results: Statewide: All domains remained the same or experienced increases in positive scores, none of which were statistically significant. RBHA Specific: NARBHA experienced an increase of 2 percentage points in General Satisfaction with no changes in the Outcomes or Social Connectedness domains. Decreases in the remaining domains were noted, none of which were statistically significant. Cenpatico’s domain scores varied from -7 to +5 percentage points as compared to last year, with a statistically significant increase in GSA 2’s Social Connectedness domain and a statistically significant decrease in GSA 4’s Outcomes domain. 5 2014 Consumer Survey Report CPSA (GSA 5) experienced changes ranging from -4 to +3 percentage points, none of which were statistically significant. Regional Behavioral Health Authorities (RBHA) and Tribal Behavioral Health Authorities (TBHA) results from the 2014 Consumer Survey are presented separately to preserve the integrity of findings as two different sampling methodologies were used to get the survey sample. Differences between the two survey modalities are discussed in Section III of this report. 6 2014 Consumer Survey Report II. Introduction The Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS) and the Regional Behavioral Health Authorities (RBHAs) in collaboration with their contracted providers administered the statewide consumer survey between March 31 and May 23, 2014. Within ADHS/DBHS, Consumer Survey results are: • Reviewed by the ADHS/DBHS Quality Management (QM) Committee; • Used by the QM Committee as a critical information source to drive system improvements through RBHA contracts and QM Plans; • Used for certain Performance Incentive System calculations. • Tracked and trended by the office of Network Development to determine network capacity by measuring member perception of behavioral health services in relation to the established survey domains and, • Analyzed by the office of Information Management and the Office of Performance improvement to identify and improve on performance gabs in the various survey domains. • Used by the Office of Information Management to conduct correlation analyses between survey domains and performance indicators. Survey outcomes are also: • Disseminated to ADHS/DBHS stakeholders and consumers via the ADHS/DBHS website and Performance Dashboard; • Submitted to these national organizations as part of federal grant requirements for use in national reporting and evaluation: o The National Association of State Mental Health Program Directors’ (NASMHPD) o National Research Institute (NRI) o Western States Decision Support Group (WSDSG) o Substance Abuse and Mental Health Services Administration’s (SAMHSA) Mental Health Statistics Improvement Program (MHSIP) III. Survey Sample and the Response Rates A total of 4,800 RBHA-enrolled consumers were included for the MHSIP Consumer Survey random sample pool; 3,677 consumers were contacted and asked to complete a survey and 3,011 surveys were actually completed and returned to be included in tally of results. The response rate is calculated by dividing the number of surveys returned by the number of enrollees asked to participate. Statewide response rates based on surveys offered were 83% for adults and 80% for youth, as seen in TABLE 1. Please see Attachment B for a complete discussion of the survey protocol. The Tribal Behavioral Health Authorities (TBHA) scores are not included in the statewide results due to the TBHA convenience sampling methodology which solicits responses at the convenience of the recipients rather than from a random sample of TBHA enrolled recipients. Therefore, the results are reported separately in Attachment A – 2014 Tribal Behavioral Health Authority Health Survey Results. 7 2014 Consumer Survey Report TABLE 1: Survey Response Rate Based on Surveys Offered for Each RBHA/GSA Consumer Survey 2014 Surveys Offered (a) RBHA/GSA GSA 1 NARBHA GSA 2 Cenpatico GSA 3 Cenpatico GSA 4 Cenpatico GSA 5 – CPSA Statewide RBHA Surveys Completed (b) Response Rate (b/a) Overall 825 Adult 419 Youth 406 Overall 713 Adult 348 Youth 365 Overall 86% Adult 83% Youth 90% 610 365 245 515 303 212 84% 83% 87% 729 425 304 638 371 267 88% 87% 88% 761 366 395 583 274 309 77% 75% 78% 752 425 327 562 368 194 75% 87% 59% 3,677 2,000 1,677 3,011 1,664 1,347 82% 83% 80% IV. Methodology and Survey Administration Two survey populations (sampling frames) were identified for use in the Adult Consumer Survey and the YSS-F: Adults - Title XIX/XXI behavioral health recipients who are 18 years of age or older, and are enrolled in any of the adult programs: Serious Mental Illness (SMI), Substance Abuse or General Mental Health (Non-SMI). Youth - Title XIX/XXI behavioral health recipients under age 18 and enrolled in the Child/Adolescent program. A total of 30,414 adult and 18,231 youth RBHA-enrolled Title XIX/XXI consumers were eligible to participate in the 2014 survey. Please refer to Attachment B (2014 Consumer Survey Protocol) for details on sampling frame development, inclusion/exclusion criteria, survey instruments, and survey administration guidelines. The ADHS/DBHS utilized a Likert Scale model to have respondents rate the domain questions of the Consumer Survey from strongly agree (numeric value of 5) to strongly disagree (numeric value of 1), and an option for Not Applicable (numeric value 0). Consumer perception was determined as positive for those domains where the question responses equaled 3.5 to 5.0. TABLEs 2 & 3 show the number and percentage of sample surveys that were not included in the domain scores. These are presented by RBHA/GSA and domain. Surveys are excluded from a domain’s score when more than one-third of the questions have a response that cannot be determined. This occurs most often when no response is marked. It also occurs when more than one response is marked. See Attachment B for a full discussion of the survey protocol. 8 2014 Consumer Survey Report TABLE 2: Adult Number and Percent of Surveys Excluded Due to Non-Responses by Domain and GSA/RBHA GSA/RBHA GSA 1 NARBHA GSA 2 Cenpatico GSA 3 Cenpatico GSA 4 Cenpatico GSA 5 CPSA Statewide RBHA General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness 2 (0.6%) 3 (0.9%) 20 (5.7%) 5 (1.4%) 11 (3.2%) 12 (3.4%) 13 (3.7%) 2 (0.7%) 2 (0.7%) 12 (4.0%) 5 (1.7%) 10 (3.3%) 10 (3.3%) 12 (4.0%) 5 (1.3%) 9 (2.4%) 19 (5.1%) 9 (2.4%) 17 (4.6%) 15 (4.0%) 15 (4.0%) 1 (0.4%) 3 (1.1%) 16 (5.8%) 5 (1.8%) 12 (4.4%) 10 (3.6%) 7 (2.6%) 19 (5.0%) 20 (5.3%) 32 (8.4%) 25 (6.6%) 34 (9.0%) 27 (7.1%) 29 (7.7%) 29 (1.7%) 37 (2.2%) 99 (5.9%) 49 (2.9%) 84 (5.0%) 74 (4.4%) 76 (4.5%) TABLE 3: Youth Number and Percent of Surveys Excluded Due to Non-Responses by Domain and GSA/RBHA GSA/RBHA GSA 1 NARBHA GSA 2 Cenpatico GSA 3 – Cenpatico GSA 4 – Cenpatico GSA 5 CPSA Statewide RBHA General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 2 (0.6%) 6 (1.7%) 4 (1.1%) 7 (1.9%) 6 (1.7%) 6 (1.7%) 7 (1.9%) 3 (1.4%) 3 (1.4%) 2 (0.9%) 3 (1.4%) 2 (0.9%) 2 (0.9%) 5 (2.4%) 2 (0.7%) 5 (1.9%) 2 (0.7%) 9 (3.4%) 5 (1.9%) 4 (1.5%) 8 (3.0%) 2 (0.6%) 3 (1.0%) 5 (1.6%) 8 (2.6%) 6 (1.9%) 6 (1.9%) 10 (3.2%) 1 (0.5%) 3 (1.6%) 1 (0.5%) 9 (4.7%) 2 (1.1%) 2 (1.1%) 4 (2.1%) 10 (0.7%) 20 (1.5%) 14 (1.0%) 36 (2.7%) 21 (1.6%) 20 (1.5%) 34 (2.5%) V. Data Analysis ADHS/DBHS provided each RBHA with an Access database to compile survey responses. All completed surveys were entered by the RBHAs and the databases were submitted to ADHS/DBHS. Each RBHA analyzed its respective survey data using an SPSS script designed and provided by ADHS/DBHS to ensure consistency in data analysis. ADHS/DBHS calculated GSA-specific and statewide results presented in this report from the databases submitted by each RBHA. Statewide survey data is analyzed as follows: By Domain; By Domain Line Item; Sub-group Analysis; and Comparison to Past Survey Performance. 9 2014 Consumer Survey Report Historically, weights have been applied for the calculation of statewide survey performance. Each GSA is provided a similar number of randomly selected members to complete surveys but GSAs can have very different population sizes. In previous years, weighting was applied because GSA 6 has a relatively larger population base of enrolled members than the other GSAs. A change in RBHA contractor for GSA 6 occurred in FY2014 and prior to implementation of the survey period, the feasibility of GSA 6’s participation in the survey was discussed amongst management staff at AHCCCS, ADHS/DBHS, and Magellan Health Services, the contractor responsible for providing services through March 31, 2014. It was decided that Magellan would not be obligated to participate. Mercy Maricopa Integrated Health Services was awarded the contract to serve GSA 6 behavioral health recipients effective April 1, 2014. Because the survey asks RBHA members to provide input based on their perception of services received during the six months prior to completing the survey questionnaire, participants could not comment on services they had received from Mercy Maricopa. This circumstance prevented survey participation of GSA 6 members in the 2014 survey. Therefore, statewide results were not weighted this year. VI. Statewide Survey Data Limitations Two different sampling methodologies were used to obtain the samples for the 2014 Consumer Survey: one for the Regional Behavioral Health Authorities (RBHA) and another for the Tribal Behavioral Health Authorities (TBHA). The TBHA scores were not included in the statewide results due to the TBHA convenience sampling methodology which solicits responses at the convenience of the recipients and providers rather than from a random sample of TBHA enrolled recipients. VII. Adult Survey Analysis and Results Demographics A total of 1,664 completed adult surveys were analyzed. Responses are broken out by gender, age, ethnicity, race, length of enrollment in services, length of enrollment and behavioral health category. Please see Attachment D, 2014 Adult Demographics for complete demographics information. Domain Line Item Analysis A specific set of questions was developed to gather scores for each domain. Line items are specific questions pertaining to each survey domain (Attachment E, 2014 Adult Line Item); analysis of the respondents’ answers to a question under a domain, either positive or negative, affects the overall domain score. TABLE 4 demonstrates the results of the 2014 Adult Survey and compares them to previous surveys. The t-test was used to compare survey results for 2013 and 2014 and determine whether there were any statistically significant differences between the mean of the samples of each domain. A P-value of 0.05 with 95% confidence interval is used to determined statistical significance. A score is therefore considered to be of statistical significance when it is less than or equals to 0.05. The initials: “ns” (Not Significant) are used to denote changes in the results between the current 10 2014 Consumer Survey Report and previous year that are not statistically significant. Again, direct comparison of 2013 and 2014 statewide scores should take into consideration that the 2013 statewide data was weighted and the 2014 data was not. For the purpose of the t-test to determine the statistical significance of differences between 2013 and 2014 domain scores, 2013 scores were re-calculated to exclude GSA 6 data and remove weighting. TABLE 4 Statewide RBHA Adult Consumer Survey Results by Domain for 2007– 2014 2007 % Domain General Satisfaction Service Access 86 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 79 Improved Functioning Social Connectedness 66 Color 77 88 74 65 2008 % (Pvalue) 82 (ns) 73 (ns) 89 (0.01) 2009 % (Pvalue) 84 (ns) 78 (ns) 82 (0.03) 2010 % (Pvalue) 86 (ns) 81 (ns) 92 (0.01) 2011 % (Pvalue) 86 (ns) 84 (ns) 87 (ns) 2012 % (Pvalue) 85 (ns) 83 (ns) 90 (0.01) 2013 % (Pvalue) 90 (0.01) 87 (0.06) 91 (ns) 2014 % (Pvalue) 88 (ns) 87 (ns) 92 (ns) 84 (ns) 68 (ns) 68 (ns) 68 (ns) 89 (ns) 70 (ns) 68 (ns) 68 (ns) 90 (ns) 75 (ns) 71 (ns) 79 (0.02) 88 (ns) 72 (ns) 69 (ns) 76 (ns) 87 (ns) 74 ns) 69 (ns) 76 (ns) 91 (0.00) 72 (ns) 71 (ns) 77 (ns) 92 (ns) 74 (ns) 72 (ns) 82 (0.02) Key statistically significant increase statistically significant decrease Figure 1-The 2014 Statewide Adult Survey Results: Rate Change Analysis Social Connectedness Improved Functioning Outcomes 2014 Service Quality & Appropriateness 2013 Participation in Treatment Planning Service Access General Satisfaction 0 10 20 30 40 50 60 70 80 90 100 General Satisfaction: Three questions are included in the General Satisfaction domain The 2014 score for this domain is 88%, representing a decrease of 2 percentage points compared to the 2013 survey results which is not statistically significant (see TABLE 4). 11 2014 Consumer Survey Report Service Access: Six questions in the survey gathered information regarding Access to Services. The average score for this domain is 87%, unchanged from last year. Participation in Treatment Planning: Two survey questions were used to determine the score for Participation in Treatment Planning. The average score for this domain is 92%, a slight increase of one percentage point from the 2013 results. Service Quality and Appropriateness: Nine questions in the survey focused on Service Quality and Appropriateness. The average score is 92%, again an increase of one percentage point over 2013. Outcomes: Eight questions were asked to obtain consumers’ perception regarding Outcomes. After last year’s drop by 2 percentage points as compared to 2012, the 2014 results indicate an increase of 2 percentage points, returning to 74% for the overall domain score. Improved Functioning: Five questions are included in the Improved Functioning domain. The overall score for this domain is 72%, indicating a one percentage point increase compared to last year. Social Connectedness: Four questions were asked in the survey to ascertain Social Connectedness. This domain increased to 82%, experiencing a statistically significant increase of 5 percentage points. RBHA and GSA specific Adult survey findings are presented in TABLES 5 through 9 (page 1315); the Tables include percentages and P-values for each domain by RBHA and GSA covering 2007 through 2014. TABLE 5: GSA 1 - NARBHA Adult Consumer Survey Results by Domain for 2007 - 2014 2007 % Domain General Satisfaction 84 Service Access 73 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 78 Improved Functioning 63 Social Connectedness 61 12 87 74 2008 % (Pvalue) 85 (ns) 80 (ns) 92 (0.00) 88 (ns) 71 (ns) 68 (ns) 65 (ns) 2009 % (Pvalue) 84 (ns) 79 (ns) 80 (0.00) 88 (ns) 64 (ns) 63 (ns) 68 (ns) 2010 % (Pvalue) 88 (ns) 83 (ns) 92 (0.00) 90 (ns) 72 (ns) 71 (ns) 78 (0.03) 2011 % (Pvalue) 90 (ns) 85 (ns) 91 (ns) 93 (ns) 76 (ns) 74 (ns) 80 (ns) 2012 % (Pvalue) 86 (ns) 86 (ns) 93 (ns) 92 (ns) 72 (ns) 69 (ns) 78 (ns) 2013 % (Pvalue) 89 (ns) 85 (ns) 93 (ns) 94 (ns) 71 (ns) 71 (ns) 80 (ns) 2014 % (Pvalue) 88 (ns) 86 (ns) 90 (ns) 93 (ns) 73 (ns) 73 (ns) 81 (ns) 2014 Consumer Survey Report TABLE 6: GSA 2 – Cenpatico Adult Consumer Survey Results by Domain for 2007 - 2014 2007 % Domain General Satisfaction 87 Service Access 80 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 79 Improved Functioning 73 Social Connectedness 74 Color 87 78 2008 % (Pvalue) 84 (ns) 82 (ns) 86 (ns) 88 (ns) 71 (ns) 75 (ns) 75 (ns) 2009 % (Pvalue) 86 (ns) 81 (ns) 82 (ns) 89 (ns) 71 (ns) 67 (ns) 71 (ns) 2010 % (Pvalue) 91 (ns) 90 (0.02) 93 (0.00) 92 (ns) 81 (0.03) 75 (ns) 83 (0.01) 2011 % (Pvalue) 89 (ns) 88 (ns) 91 (ns) 90 (ns) 82 (ns) 77 (ns) 83 (ns) 2012 % (Pvalue) 88 (ns) 83 (ns) 92 (ns) 91 (ns) 69 (0.00) 70 (0.03) 80 (ns) 2013 % (Pvalue) 91 (ns) 88 (ns) 91 (ns) 91 (ns) 73 (ns) 71 (ns) 78 (ns) 2014 % (P-value) 87 (ns) 88 (ns) 94 (ns) 92 (ns) 74 (ns) 72 (ns) 81 (ns) Key statistically significant increase statistically significant decrease TABLE 7: GSA 3 - Cenpatico Adult Consumer Survey Results by Domain for 2007 - 2014 Domain 2007 % General Satisfaction 90 Service Access 83 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 82 Improved Functioning 72 Social Connectedness 71 13 89 79 2008 % (Pvalue) 82 (0.01) 77 (ns) 87 (ns) 86 (ns) 68 (0.01) 73 (ns) 72 (ns) 2009 % (Pvalue) 83 (ns) 79 (ns) 79 (0.02) 88 (ns) 71 (ns) 71 (ns) 71 (ns) 2010 % (Pvalue) 86 (ns) 84 (ns) 88 (0.03) 88 (ns) 77 (ns) 72 (ns) 80 (0.05) 2011 % (Pvalue) 86 (ns) 86 (ns) 91 (ns) 88 (ns) 75 (ns) 73 (ns) 81 (ns) 2012 % (Pvalue) 89 (ns) 91 (ns) 94 (ns) 93 (ns) 81 (ns) 78 (ns) 88 (ns) 2013 % (Pvalue) 88 (ns) 88 (ns) 93 (ns) 92 (ns) 79 (ns) 75 (ns) 79 (0.005) 2014 % (Pvalue) 88 (ns) 89 (ns) 93 (ns) 93 (ns) 80 (ns) 77 (ns) 85 (0.043) 2014 Consumer Survey Report TABLE 8: GSA 4 - Cenpatico Adult Consumer Survey Results by Domain for 2007 - 2014 2007 % Domain General Satisfaction 87 Service Access 86 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 81 Improved Functioning 76 Social Connectedness 73 Color 91 78 2008 % (Pvalue) 87 (ns) 83 (ns) 92 (0.01) 90 (ns) 73 (ns) 67 (0.04) 67 (ns) 2009 % (Pvalue) 86 (ns) 83 (ns) 84 (0.00) 89 (ns) 70 (ns) 64 (ns) 66 (ns) 2010 % (Pvalue) 87 (ns) 87 (ns) 92 (0.03) 90 (ns) 75 (ns) 73 (ns) 77 (0.02) 2011 % (Pvalue) 91 (ns) 89 (ns) 95 (ns) 93 (ns) 76 (ns) 70 (ns) 79 (ns) 2012 % (Pvalue) 93 (ns) 93 (ns) 94 (ns) 95 (ns) 82 (ns) 76 (ns) 81 (ns) 2013 % (Pvalue) 91 (ns) 93 (ns) 92 (ns) 95 (ns) 76 (ns) 70 (ns) 82 (ns) 2014 % (Pvalue) 88 (ns) 88 (ns) 91 (ns) 91 (ns) 68 (0.036) 67 (ns) 78 (ns) Key statistically significant increase statistically significant decrease TABLE 9: GSA 5 - CPSA Adult Consumer Survey Results by Domain for 2007 – 2014 2007 % Domain General Satisfaction 85 Service Access 78 Participation in Treatment Planning Service Quality & Appropriateness Outcomes 78 Improved Functioning 64 Social Connectedness 64 89 70 Key statistically significant increase statistically significant decrease 14 2008 % (Pvalue) 83 (ns) 67 (0.01) 91 (0.00) 80 (0.00) 66 (ns) 64 (ns) 62 (ns) 2009 % (Pvalue) 78 (ns) 69 (ns) 78 (0.00) 87 (ns) 68 (ns) 65 (ns) 65 (ns) 2010 % (Pvalue) 87 (0.03) 79 (0.03) 92 (0.03) 90 (ns) 69 (ns) 68 (ns) 78 (0.01) 2011 % (Pvalue) 89 (ns) 85 (ns) 92 (ns) 91 (ns) 75 (ns) 72 (ns) 76 (ns) 2012 % (Pvalue) 87 (ns) 78 (0.02) 89 (ns) 85 (0.02) 75 (ns) 70 (ns) 75 (ns) 2013 % (Pvalue) 87 (ns) 79 (ns) 89 (ns) 87 (ns) 71 (ns) 68 (ns) 74 (ns) 2014 % (Pvalue) 87 (ns) 83 (ns) 91 (ns) 89 (ns) 74 (ns) 71 (ns) 82 (0.010) 2014 Consumer Survey Report Summary of the 2014 Adult Survey Results: Statewide:      The domains with the highest scores for adults that participated in the 2014 survey remain the same as last year: General Satisfaction, Participation in Treatment Planning, and Service Quality and Appropriateness. General Satisfaction decreased by 2 percentage points which is not a statistically significant change. Service Access remained static at 87%. Participation in Treatment Planning and Service Quality & Appropriateness both improved by one percentage point. Outcomes and Improved Functioning improved slightly this year, by two and one percentage points respectively. Social Connectedness was the only domain for statewide adults to demonstrate a statistically significant change this year, with an increase in score of 5 percentage points (P-value of 0.02). Figure 2 - Trending Annual Adult Survey Results by Domain FY2008 - FY2014 General Satisfaction 100% 95% Service Access 90% Participation in Treatment Planning Service Quality & Appropriateness Outcomes 85% 80% 75% Improved Functioing 70% Social Connectedness 65% 2008 2009 2010 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness 15 2011 2008 82% 73% 89% 84% 68% 68% 68% 2012 2009 84% 78% 82% 89% 70% 68% 68% 2013 2010 86% 81% 92% 90% 75% 71% 79% 2014 2011 86% 84% 87% 88% 72% 69% 76% 2012 85% 83% 90% 87% 74% 69% 76% 2013 90% 87% 91% 91% 72% 71% 77% 2014 88% 87% 92% 92% 74% 72% 82% 2014 Consumer Survey Report VIII. YSS-F Survey Analysis and Results Demographics A total of 1,347 completed youth surveys were analyzed. Responses are broken out by gender, age group, ethnicity, race, length of enrollment in services, and involvement in a Child and Family Team (CFT). Please see Attachment FY2014 YSS-F Demographics for complete demographics information. Domain Line Item Analysis Line items are specific questions pertaining to each survey domain. Analysis of answers to each domain specific line item indicates the aspects of service delivery the respondents report as either positively or negatively affecting the overall domain score (Attachment G, 2014 YSS-F Line Item). TABLE 11 demonstrates the results of the 2014YSS-F Survey and compares them to results from previous years. The t-test analysis was used to compare survey results for 2013 and 2014 to determine if there were any statistically significant differences between each domain’s average scores. A P-value of 0.05 with 95% confidence interval significance was used to determine statistical significance: if the p-value is less than or equal to 0.05, the result will be considered statistically significant and if it is more than 0.05, it will be considered not statistically significant. The abbreviation “ns” is used for “not significant” to denote no statistically significant change from the previous year. As addressed previously in this report, statewide results were not weighted this year. For the purpose of the t-test to determine the statistical significance of differences between 2013 and 2014 domain scores, 2013 scores were re-calculated to exclude GSA 6 data and remove weighting. TABLE 11: Statewide Youth Consumer YSS-F Survey Results by Domain for 2007 – 2014 2007 % Domain General Satisfaction 78 Service Access 78 Participation in Treatment Planning Cultural Sensitivity 90 Outcomes 58 Improved Functioning 61 Social Connectedness 82 16 92 2008 % (Pvalue) 77 (ns) 73 (ns) 87 (ns) 90 (ns) 60 (ns) 58 (ns) 79 (ns) 2009 % (Pvalue) 80 (ns) 76 (ns) 89 (ns) 90 (ns) 68 (ns) 66 (ns) 83 (ns) 2010 % (Pvalue) 85 (ns) 83 (ns) 92 (ns) 96 (0.05) 69 (ns) 71 (ns) 88 (ns) 2011 % (Pvalue) 82 (ns) 86 (ns) 94 (ns) 96 (ns) 65 (ns) 67 (ns) 89 (ns) 2012 % (Pvalue) 82 (ns) 86 (ns) 92 (0.02) 96 (ns) 62 (0.00) 68 (ns) 86 (0.03) 2013 % (Pvalue 82 (ns) 84 (0.03) 93 (ns) 95 (ns) 61 (ns) 64 (ns) 87 (ns) 2014 % (Pvalue) 86 (ns) 91 (ns) 94 (ns) 97 (ns) 68 (ns) 69 (ns) 91 (ns) 2014 Consumer Survey Report Color Key statistically significant increase statistically significant decrease Figure 3-The 2014 Statewide YSS-F Results: Rate Change Analysis Social Connectedness Improved Functioning Outcomes 2014 Cultural Sensitivity 2013 Participation in Treatment Planning Service Access General Satisfaction 0 Domains General satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 10 20 30 40 50 60 70 80 2013 82% 84% 93% 95% 61% 64% 87% 90 100 2014 86% 91% 94% 97% 68% 69% 91% General Satisfaction: Six questions were asked in the survey to ascertain General Satisfaction (Attachment G). The overall score for this domain is 86%, an increase of 4 percentage points. Service Access: Two questions in the survey gathered information regarding Access to Services (Attachment G). The score for this domain is 91%, an increase of 7 percentage points compared to 2013 results. Participation in Treatment Planning: The survey included three questions to indicate positivity with Participation in Treatment Planning (see Attachment G). The score for this domain is 94%, up one percent from 2013 results which is not statistically significant. Cultural Sensitivity: Four questions in the survey focused on Cultural Sensitivity (Attachment G). The average score for this domain is 97%, an increase of 2 percentage points compared to 2013 results. 17 2014 Consumer Survey Report Outcomes: Seven questions were asked to get consumers’ perception on Outcomes (Attachment G). The average score for this domain is 68%, an increase of 7 percentage points. Improved Functioning: Six questions indicate respondents’ impressions of Improved Functioning (Attachment G). The average score for this domain is 69%, up 5 percentage points from 2013 results. Social Connectedness: Four questions were asked in the survey to ascertain Social Connectedness (see Attachment G). This domain scored 91%, indicating an increase of 4 percentage points. RBHA and GSA specific YSS- F survey findings are presented in TABLES 12 through 16 (page 19 - 21); they include percentages and P-values for each domain by RBHA and GSA ranging from 2007 through 2014. TABLE 12: GSA 1 - NARBHA Youth Consumer YSS-F Survey Results by Domain for 2007 – 2014 Domain 2007 % General Satisfaction 76 Service Access 77 Participation in Treatment Planning Cultural Sensitivity 91 Outcomes 51 Improved Functioning 55 Social Connectedness 80 18 93 2008 % (Pvalue) 2009 % (Pvalue) 2010 % (Pvalue) 2011 % (Pvalue) 2012 % (Pvalue) 2013 % (Pvalue) 2014 % (PValue) 84 (ns) 81 (ns) 88 (ns) 93 (ns) 72 (0.00) 71 (0.00) 87 (ns) 83 (ns) 79 (ns) 92 (ns) 93 (ns) 67 (ns) 66 (ns) 87 (ns) 90 (ns) 90 (0.01) 93 (ns) 96 (ns) 70 (ns) 73 (ns) 91 (ns) 90 (ns) 94 (ns) 94 (ns) 97 (ns) 69 (ns) 71 (ns) 90 (ns) 85 (ns) 90 (ns) 93 (ns) 98 (ns) 65 (ns) 68 (ns) 92 (ns) 85 (ns) 93 (ns) 96 (ns) 97 (ns) 67 (ns) 71 (ns) 90 (ns) 87 (ns) 91 (ns) 94 (ns) 96 (ns) 67 (ns) 69 (ns) 90 (ns) 2014 Consumer Survey Report TABLE 13: GSA 2 - Cenpatico Youth Consumer YSS-F Survey Results by Domain for 2007 – 2014 2007 % Domain General Satisfaction 89 Service Access 86 Participation in Treatment Planning Cultural Sensitivity 88 Outcomes 76 Improved Functioning 78 Social Connectedness 89 Color 96 2008 % (Pvalue) 76 (0.00) 81 (ns) 86 (ns) 94 (ns) 70 (ns) 69 (0.03) 86 (ns) 2009 % (Pvalue) 81 (ns) 82 (ns) 94 (0.02) 93 (ns) 62 (ns) 61 (ns) 85 (ns) 2010 % (Pvalue) 82 (ns) 90 (0.04) 93 (ns) 99 (0.02) 69 (ns) 70 (ns) 89 (ns) 2011 % (Pvalue) 91 (0.02) 95 (ns) 96 (ns) 99 (ns) 71 (ns) 73 (ns) 97 (0.01) 2012 % (Pvalue) 89 (ns) 93 (ns) 93 (ns) 97 (ns) 75 (ns) 76 (ns) 94 (ns) 2013 % (Pvalue) 88 (ns) 91 (ns) 93 (ns) 95 (ns) 74 (ns) 76 (ns) 91 (ns) 2014 % (PValue) 88 (ns) 94 (ns) 94 (ns) 97 (ns) 72 (ns) 74 (ns) 96 (0.048) Key statistically significant increase statistically significant decrease TABLE 14: GSA 3 - Cenpatico Youth Consumer YSS-F Survey Results by Domain for 2007 – 2014 Domain 2007 % General Satisfaction 77 Service Access 74 Participation in Treatment Planning Cultural Sensitivity 88 Outcomes 65 Improved Functioning 65 Social Connectedness 77 19 90 2008 % (Pvalue) 74 (ns) 74 (ns) 80 (0.01) 89 (ns) 65 (ns) 62 (ns) 2009 % (Pvalue) 75 (ns) 76 (ns) 88 (0.05) 89 (ns) 66 (ns) 65 (ns) 2010 % (Pvalue) 87 (0.01) 90 (0.00) 90 (ns) 94 (ns) 77 (0.02) 79 (0.00) 2011 % (Pvalue) 86 (ns) 90 (ns) 89 (ns) 96 (ns) 73 (ns) 74 (ns) 2012 % (Pvalue) 88 (ns) 93 (ns) 96 (0.00) 97 (ns) 71 (ns) 74 (ns) 2013 % (Pvalue) 82 (ns) 92 (ns) 93 (ns) 96 (ns) 67 (ns) 68 (ns) 2014 % (PValue) 85 (ns) 94 (ns) 94 (ns) 97 (ns) 68 (ns) 71 (ns) 77 (ns) 78 (ns) 87 (0.03) 87 (ns) 91 (ns) 89 (ns) 92 (ns) 2014 Consumer Survey Report TABLE 15: GSA 4 - Cenpatico Youth Consumer YSS-F Survey Results by Domain for 2007 – 2014 2007 % Domain General Satisfaction 78 Service Access 86 Participation in Treatment Planning Cultural Sensitivity 92 Outcomes 66 Improved Functioning 68 Social Connectedness 82 Color 91 2008 % (Pvalue) 78 (ns) 80 (ns) 90 (ns) 92 (ns) 61 (ns) 62 (ns) 87 (ns) 2009 % (Pvalue) 85 (ns) 91 (0.01) 98 (0.01) 96 (ns) 66 (ns) 65 (ns) 87 (ns) 2010 % (Pvalue) 80 (ns) 87 (ns) 91 (0.00) 95 (ns) 66 (ns) 69 (ns) 88 (ns) 2011 % (Pvalue) 84 (ns) 94 (0.04) 94 (ns) 97 (ns) 70 (ns) 71 (ns) 91 (ns) 2012 % (Pvalue) 82 (ns) 92 (ns) 92 (ns) 96 (ns) 63 (ns) 67 (ns) 88 (ns) 2013 % (Pvalue) 88 (0.04) 91 (ns) 93 (ns) 95 (ns) 75 (0.007) 76 0.01 92 (ns) 2014 % (PValue) 89% (ns) 92% (ns) 96 (ns) 98 (0.03) 68 (0.048) 69 (ns) 91 (ns) Key statistically significant increase statistically significant decrease TABLE 16: GSA 5 - CPSA Youth Consumer YSS-F Survey Results by Domain for 2007 – 2014 2007 % Domain General Satisfaction 81 Service Access 76 Participation in Treatment Planning Cultural Sensitivity 89 Outcomes 65 Improved Functioning 66 Social Connectedness 86 Color 90 Key statistically significant increase statistically significant decrease 20 2008 % (Pvalue) 79 (ns) 73 (ns) 86 (ns) 90 (ns) 63 (ns) 62 (ns) 82 (ns) 2009 % (Pvalue) 82 (ns) 74 (ns) 86 (ns) 86 (ns) 72 (ns) 72 (0.04) 84 (ns) 2010 % (Pvalue) 81 (ns) 84 (0.02) 88 (ns) 94 (0.02) 62 (0.03) 66 (ns) 84 (ns) 2011 % (Pvalue) 88 (ns) 87 (ns) 93 (ns) 97 (ns) 68 (ns) 71 (ns) 90 (ns) 2012 % (Pvalue) 84 (ns) 81 (ns) 89 (ns) 91 (0.01) 62 (0.01) 66 (ns) 86 (ns) 2013 % (Pvalue) 85 (ns) 83 (ns) 94 (ns) 96 (ns) 60 (ns) 64 (ns) 85 (ns) 2014 % (PValue) 81 (ns) 81 (ns) 92 (ns) 96 (ns) 62 (ns) 63 (ns) 88 (ns) 2014 Consumer Survey Report Summary of the 2014 YSS-F Survey Results: Statewide:  On a statewide basis, all domains experienced increased scores in 2014 compared to scores of the previous year. Figure 2 - Trending YSS-F Survey Results by Domain FY2008 - FY2014 100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioing Social Connectedness 2008 2009 2010 2011 2012 2013 2014 RBHA Specific:  NARBHA demonstrated minor fluctuations in domain scores compared to 2013, ranging from -2 to +2 percentage points.  Cenpatico (GSA 2, 3, 4) experienced some score changes in all GSAs, all domains, with a statistically significant increase of 5 percentage points in GSA 2’s Social Connectedness domain and a statistically significant increase in GSA 4’s Cultural Sensitivity (3 percentage points) and statistically significant decrease of 7 percentage points in Outcomes.  CPSA (GSA 5) experienced percentage decreases or increases in all domains ranging from -4 to +3 percentage points, none of which were statistically significant. IX. Statewide Improvement Opportunities ADHS/DBHS recognizes the opportunity to use the information from the 2014 survey results to make system improvements, and will implement statewide initiatives with the RBHAs to continue to improve scores for each domain. Special attention will be placed around the Outcomes and Improved Functioning domains, which continue to receive the lowest scores from both Adults and Children. DBHS will also collaborate with the RBHAs to increase performance improvement initiatives to improve on the number of surveys responses that are returned, more so because this year (2014), the total response rate for the survey dropped. A review of the plan to re-engineer the consumer survey distribution process will be carried out and new initiatives will be developed to improve response rates for 2015 21 2014 Consumer Survey Report In addition to general statewide initiatives, RBHA-specific activities are undertaken as part of standard ADHS/DBHS processes, as appropriate. Survey results are shared with the RBHAs, and they are required to respond with analysis of any problem areas, along with plans for improvement. ADHS/DBHS reviews their conclusions and plans, and monitors implementation and progress. X. RBHA Specific Initiatives NARBHA reports its survey results to its Quality Management Committee, Leadership Council, ADHS/DBHS, its subcontractors, its Board of Directors, and all NARBHA staff. NARBHA provides continuous technical assistance throughout the survey process, assisting its subcontractors to identify opportunities to improve the survey process, response rates, and results. Targeted technical assistance is provided when there is a substantial decrease in overall results from the prior year. Although NARBHA experienced slight improvement in the Outcomes and Improved Functioning domains for its adult population, they remain the lowest scoring domains for NARBHA as well as across the state and NARBHA has undertaken the Outcomes domain as a Performance Improvement Project. Cenpatico initiated a Performance Improvement Plan in July 2013 to improve performance in the Outcomes and Improved Functioning domains. They sought improvement ideas from a monthly member survey. Cenpatico will continue use of the survey after modifying the questions in response to 2014 scores related to the Outcomes and Improved Functioning domain questions. Cenpatico will also continue to educate survey participants on the meaning of questions included in these two domains as another related PIP that was conducted a few years ago identified apparent confusion about the questions. CPSA uses its survey results as a standalone evaluation of its behavioral health system and, in conjunction with other measures, to evaluate its overall system of care. They share findings with all their subcontractors, requiring performance improvement plans from those that fail to meet compliance thresholds. CPSA plans to continue audits of provider medical charts to provide feedback on key survey indicators. In order to guide service planning and service provision in a way that aligns with the survey, CPSA’s providers continue to incorporate survey outcome indicators into forms used by clinical services utilized at the beginning of appointments to address where members feel they are in treatment relative to outcomes attributable to treatment. This is an opportunity for staff to address what may or may not be working in treatment, allowing the opportunity for members to address outcomes with their clinical team on a consistent basis. Additionally, the providers have incorporated internal surveys for members to complete. This PIP initiative was a huge focus for the Adult providers in 2013 and once again appeared to make a significant impact based on the Outcome domain scores. Some of CPSA’s youth providers are choosing to address the Outcomes Indicator language more specifically in the CFT agenda, by implementing this language into the CFT Agenda forms, and calling on CFT coaches to monitor the effectiveness of this language within the CFT. CPSA will evaluate the effectiveness of these tools in the 2014 Corrective Action Plans across the CPSA Children’s system. CPSA’s Quality Management staff continue to audit Comprehensive Service Provider medical charts and provide real-time feedback to Comprehensive Service Provider staff on key MHSIP indicators. 22 2014 Consumer Survey Report XI. ADHS/DBHS Initiatives The Adult and Children’s System of Care Strategic Plans are informed by Survey results. Standardized measures are shared with DBHS stakeholders through the Behavioral Health System Performance Framework and Dashboard. ADHS/DBHS engages in a variety of network development activities to support the T/RBHAs in expanding and enhancing their provider networks. DBHS’ network management process identifies network needs as well as successes. Goals and objectives established for DBHS’ FY2015 System of Care Plan are: • • Goal 1 - Promote a Recovery-oriented system of care that maximizes resilience and independence for members and families. o Objective 1.1 – Advance health and wellness initiatives within the behavioral health system o Objective 1.2 – Establish ratios of case managers to members sufficient for both children with complex needs and adults with serious mental illness o Objective 1.3 – Promote the Office of Individual and Family Affairs o Objective 1.4 – Develop sufficient availability of generalist direct support providers and specialty providers to deliver flexible, in-home, community based support and rehabilitation services o Objective 1.5 - Increase opportunities for members to engage in informed decision making regarding employment opportunities. o Objective 1.6 – Increase the number of youth who successfully transition to adulthood Goal 2 - Improve the quality of behavioral health interventions o Objective 2.1 - Expand the use of best practices to improve outcomes o Objective 2.2 - Improve Access to Care o Objective 2.3 - Improve the quality of services to Children age 0-5 o Objective 2.4 - Enhance Substance Abuse Services XIII. Conclusion The Annual Consumer Surveys are a valuable source of information, providing direct feedback from members about their experience with behavioral health services. The results identify areas of satisfaction as well as opportunities for improvement. For Adults, the Participation in Treatment Planning and Service Quality & Appropriateness domains continue to demonstrate the highest scores in terms of positive responses. For Youth, the highest scoring domains are Cultural Sensitivity and Participation in Treatment Planning with Service Access closely following. Social Connectedness shows the most improvement this year; up 5 percentage points for Adults and 4 percentage points for Youth. Although the Outcomes and Improved Functioning domains continue to score the lowest for both Adults and Youth, this year experienced increases in both domains for both populations. 23 2014 Consumer Survey Report General Satisfaction scores experienced a decrease of 2 percentage points for the Adult population and increased 4 percentage points for Youth. For future comparison of year-to-year domain scores, 2014 will be treated as an outlier year due to the fact that limited survey participation occurred this year and the 2013 data had to be manipulated to conduct the t-tests for statistically significant changes. 24 2014 Consumer Survey Report ATTACHMENT-A 25 2014 Consumer Survey Report 2014 Tribal Behavioral Health Survey Results Arizona Tribal Behavioral Health Authorities (TBHAs) Gila River Indian Community, Pascua Yaqui Centered Spirit Program, Navajo Nation and White Mountain Apache Tribe continued participation in the 2014 Consumer Survey. All Tribal BHAs conduct a convenience sampling of their enrolled members. The number of surveys completed by tribal participants usually varies from year to year from both an overall, statewide perspective and tribe-specific. The 2014 survey experienced a dramatic decrease in tribal participation, falling from a total of 775 completed surveys in 2013 to 481 completed this year. Navajo Nation returned only one completed YSS-F form which greatly impacts interpretation of the domain scores with regard to representing opinions of Navajo’s youth members. Number of Surveys Completed TBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Total TBHA Adult Survey 2013 2014 26 27 137 70 317 166 80 135 560 398 YSS-F 2013 2014 19 8 46 1 130 55 20 19 215 83 Total 2013 53 183 447 100 775 2014 35 71 221 154 481 Methodology and Survey Administration A convenience sampling methodology was used (rather than a random sampling); therefore, all persons enrolled with the TBHAs were eligible for participation in the survey. Data Analysis TBHAs results were submitted directly to ADHS/DBHS and entered into the database; analysis was conducted by ADHS/DBHS staff in the Office of Information Management. Summary of Results: For all tribal adults combined, domain scores remain similar to scores experienced in 2013 with the most significant changes being an increase of 4 percentage points in the Social Connectedness and Improved Functioning domains and a decrease of 2 percentage points in Participation in Treatment Planning. The Outcomes and Improved Functioning domains remain the lowest rate of satisfaction amongst TBHA adults at 80% and 81% respectively, although both domains show an increase compared to 2013. Families of youth receiving services through the Tribal BHAs reported levels of satisfaction in all domains as compared to 2013 also, with the most significant changes in the Participation in Treatment Planning domain that scored a decrease of 2 percentage points (86%) and the General Satisfaction domain that increased 4 percentage points (94%). Outcomes and Improved Functioning domains are the lowest scoring domains for tribal youth also, scoring the same as the adult population at 80% and 81% respectively. 26 2014 Consumer Survey Report Summary of 2012 to 2014 Tribal Behavioral Health Authorities Results for Adults General Satisfaction TBHA 2012 Gila River Navajo Nation Pascua Yaqui White Mtn Apache 2013 Participation in Treatment Planning Service Access 2014 2012 2013 2014 Service Quality & Appropriateness Improved Functioning Outcomes Social Connectedness 2012 2013 2014 2012 2013 2014 2012 2013 2014 2012 2013 2014 2012 2013 2014 89% 96% 85% 81% 88% 88% 85% 92% 88% 83% 88% 92% 68% 78% 75% 70% 88% 77% 85% 88% 92% 85% 82% 81% 85% 83% 83% 88% 88% 85% 87% 88% 84% 75% 72% 86% 80% 72% 79% 87% 77% 82% 94% 92% 91% 92% 90% 90% 89% 92% 93% 92% 91% 93% 82% 81% 85% 84% 79% 87% 91% 85% 88% 93% 94% 97% 87% 73% 81% 87% 74% 79% 88% 70% 85% 87% 72% 72% 80% 74% 74% 93% 88% 89% TBHA scores are not included in statewide numbers due to TBHA convenience sampling methodology. Summary of 2012 to 2014 Tribal Behavioral Health Authorities Results for Youth General Satisfaction TBHA Gila River Navajo Nation Pascua Yaqui White Mtn Apache Participation in Treatment Planning Service Access Cultural Sensitivity Social Connectedness 2012 2013 2014 2012 2013 2014 2012 2013 2014 2012 2013 2014 2012 2013 2014 2012 2013 2014 2012 2013 2014 86% 90% 100% 96% 90% 100% 91% 78% 100% 100% 95% 100% 68% 74% 75% 68% 74% 75% 96% 95% 100% 94% 98% 100% 88% 96% 100% 88% 91% 100% 94% 100% 100% 94% 93% 0% 94% 93% 0% 94% 98% 100% 93% 90% 98% 92% 89% 91% 92% 90% 89% 97% 95% 96% 93% 80% 85% 93% 80% 87% 95% 92% 96% 100% 76% 76% 100% 81% 76% 100% 86% 71% 100% 95% 88% 100% 67% 71% 100% 67% 71% 100% 91% 88% TBHA scores are not included in statewide numbers due to TBHA convenience sampling methodology. 27 Improved Functioning Outcomes 2014 Consumer Survey Report ATTACHMENT-B 28 STATEWIDE CONSUMER SURVEY PROTOCOL 2014 Arizona Department of Health Services Division of Behavioral Health Services Executive Summary The protocol for the 2014 consumer survey is based on the methodological and administrative guidelines followed in previous years. Two consumer surveys will be administered again in 2014; one for adults and one for the parents/guardians of children receiving behavioral health services. The survey will be available to consumers in two languages: English and Spanish. Survey data will be entered into an SPSS database by the Regional Behavioral Health Authorities (RBHAs) and forwarded to the Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS). Prior to implementation of the 2014 survey, ADHS/DBHS will provide each Geographic Service Area (GSA) with the CIS (Client Information System) Client IDs for 500 adult consumers and 400 to 500 youth consumers plus a 20 percent over sample who the RBHAs/providers will encourage to participate in the survey. Each CIS Client ID will be linked to a Survey ID reference number to allow for tracking of which consumers completed a survey. The survey will be administered during the eight (8) weeks between March 31, 2014 and May 23, 2014. As in previous years, the primary administration route will be to distribute the survey at the provider sites. As the consumer checks in for their appointment, the survey questionnaire will be handed to them. The consumer will be requested to complete the questionnaire prior to their appointment, but will be allowed to finalize the survey after the appointment if needed, or be provided with an addressed, stamped envelope to mail the survey in to the RBHA if they did not have time to complete it in the office. A drop box will be provided at each provider site for consumers to drop off their completed surveys. In cases where consumers who are selected for the sample have home appointments (as opposed to clinic appointments) during the survey timeframe, the questionnaire will be completed at home and mailed using the addressed, stamped envelope to be provided with the survey. If the consumer does not have a clinic or home appointment scheduled during the survey administration period or no-shows for a scheduled appointment, a non-clinical staff at the RBHA may conduct the survey by phone. Providers will never have access to completed surveys or individual survey results. ADHS has the statewide oversight responsibility for implementation and analysis of the survey data. The RBHAs will be responsible for ensuring that providers strictly adhere to the protocol. The providers are primarily responsible for the survey administration. Survey Instruments The two MHSIP survey instruments are the Adult Consumer Survey and the Youth Services Survey for Families (YSS-F). The adult survey will be administered to adult consumers of behavioral health services and the YSS-F will be administered to parents/guardians of children receiving behavioral health services. 1 The MHSIP Adult Consumer Survey measures seven domains: (1) Service Accessibility; (2) Service Quality or Appropriateness (which includes one item concerning cultural sensitivity); (3) Consumer Participation in Treatment Planning; (4) Outcomes; (5) General Satisfaction; (6) Improved Functioning; and (7) Social Connectedness. In addition, the questionnaire includes a module of questions to determine the impact of services received on the recipient‟s involvement with the criminal justice system. All questions are scored using a Likert Scale of 1 through 5 as follows: 5=Strongly Agree, 4=Agree, 3=Neutral, 2=Disagree, and 1=Strongly Disagree. A Not Applicable option is also available if the question does not apply. In addition to the state-added question related to cultural sensitivity, two state-added questions introduced in the 2011 Adult survey tool to gather information specific to National CLAS Standards (Culturally and Linguistically Appropriate Services) will be retained again this year. The three state-added questions in the Adult survey tool are: Q37 – In creating my service plan my cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) Q38 – Services were provided in a language I understood. Q39 – In creating my service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) The MHSIP YSS-F focuses on the following seven domain areas: (1) Service Accessibility; (2) Participation in Treatment Planning; (3) Cultural Sensitivity; (4) Satisfaction with Services; (5) Outcomes; (6) Social Connectedness; and (7) Improved Functioning. Additional questions solicit information about the youth‟s criminal justice contact and school attendance. Questions are scored with a five-point Likert Scale where 5=Strongly Agree, 4=Agree, 3=Neutral, 2=Disagree, and 1=Strongly Disagree. A Not Applicable option is also available if the question does not apply. As with the Adult tool, in addition to the state-added question related to cultural sensitivity, two state-added questions introduced in the 2011 YSS-F tool to gather information specific to National CLAS Standards (Culturally and Linguistically Appropriate Services) will be retained again this year. The three state-added questions in the YSS-F tool are: Q29 – In creating my child‟s service plan our cultural preferences were included (For example: values, traditions, beliefs, race, language, etc.) Q30 – Services were provided in a language my child understood. Q31 – In creating my child‟s service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) The RBHA and/or provider may pre-print the following information on the survey tool prior to distribution to client:  RBHA Name  Provider Name  Survey Id  Provider Facility ID 2     Behavioral Health Category Entitlement Status (Title XIX or XXI) Distribution Method (Home, Clinic, or Phone) Client enrollment with DDD Both survey tools have four main sections: (1) demographic section, (2) MHSIP survey questions, (3) state-added questions, and (4) open-ended qualitative section. The demographic section provides descriptive information about the consumer‟s age, gender, race, ethnicity, and relationship of the person completing the survey to the service recipient. The second section of the survey contains the MHSIP standardized questions. They serve as benchmark tools to compare consumer perception of behavioral health systems across the nation. The third section consists of state-added questions to elicit consumer input regarding the inclusion of cultural preferences in the consumer‟s treatment planning and to provide information relative to the National CLAS Standards. The fourth and final section of the survey contains open-ended questions to solicit consumer comments. Two questions are asked of consumers – focusing on identifying what has been most helpful with their services, and what the consumer believes would improve services. The section entitled other comments is intended to provide consumers with an additional area on the survey to provide open-ended feedback on any issue. Confidentiality The front page of the survey questionnaire addresses confidentiality of the responses. Survey results are aggregated and not presented at an individual consumer level. The providers will never have access to completed surveys or individual survey results. Thematic analysis of the written comments may be performed by RBHA. Non-mandatory (Voluntary) The survey questionnaire likewise informs the respondent that participation in the survey is voluntary; every individual can choose to participate or not. It also notifies the respondent that non-participation will not affect the services they are currently or will receive in the future. Participating in the survey is one way the consumer may help improve the system they use. Languages The survey will be available to consumers in English and Spanish languages. Each survey form will be printed in English on one side and Spanish on the reverse side. For consumers with limited English proficiency and speak a language other than Spanish, the RBHA and/or the provider administering the survey will extend their best effort to 3 translate the survey in the consumer‟s preferred language by utilizing the Language Line or other translation/interpretation services officially utilized by the RBHA or their provider. The extent of assistance provided in language translation should not attempt to define what the question means. Sampling Design Sample Frame The sample frame refers to the population eligible to take the survey. This is the pool from which ADHS/DBHS will randomly select the sampled population. ADHS/DBHS will create an adult and a youth sample for each GSA from the CIS database. The adult clients may be enrolled in any program: General Mental Health, Substance Abuse, or Seriously Mentally Ill. Clients turning age 18 between October 1, 2013 and the end of the survey period, May 23, 2014, will be filtered out of the sample frame. The sample frame will be composed of: (a) Client must have an open Episode of Care (EOC) during the 6 months previous to the sample pull; (b) Client must be Title XIX or Title XXI eligible during the 6 months previous to the sample pull; (c) Client must have received a mental health service other than inpatient, transportation, laboratory and/or radiology services, or crisis; (d) The service must have occurred in the 6 months previous to the sample pull. Drawing of sample ADHS/DBHS will create a random sample and provide each GSA with a list of 500 adult and 400 to 500 youth clients plus a 20% over sample to offset clients in the sample who are found to not qualify due to one of the criteria listed above. Administering survey ADHS/DBHS will provide the RBHAs with a list of randomly selected clients. The RBHA will duplicate the Survey ID number on the survey tool to be offered that specific consumer. To protect the confidentiality of the consumer, none of the parties privy to this information (DBHS, RBHA, Provider) will link the survey responses to consumers. ADHS/DBHS will provide the RBHAs with a database to store the survey results. The RBHA will disseminate the surveys; track which consumers have completed the survey; track why surveys were not completed; enter the survey data into the provided database; and submit to ADHS/DBHS: the completed paper surveys, the database containing all survey results, the database detailing which consumers completed a survey and for every consumer that did not complete a survey, the reason the survey was not 4 completed. The RBHAs will enter data into the two ADHS/DBHS provided databases but the RBHAs will not alter the databases in any way. The provider agency is responsible for identifying the specific provider location or site from which the consumer is presently receiving services. Survey Methodology Distribution Method The primary distribution method is a non-clinical staff at the provider office (i.e., clinic) handing the survey questionnaire to the consumer. As the consumer checks in for their appointment, s/he will be provided with a copy of the survey questionnaire to complete. If the consumer agrees to participate, s/he will be requested to complete the survey prior to his/her appointment. If the consumer is unable to complete the questionnaire, s/he will be allowed to finish it on site after the appointment or be provided with an addressed, stamped envelope to mail the survey in if they did not have time to complete it in the office. A drop box will be provided on site for completed surveys. Additionally, a specific area at the provider office will be designated for completing the survey. If the individual randomly selected has a scheduled appointment at home during the survey window, the provider staff will bring the survey questionnaire at the appointment date. If the consumer agrees to participate, s/he will be advised to complete the survey after the staff leaves and to mail the completed questionnaire to the RBHA using the preaddressed, stamped envelope provided with the survey. If the individual does not have an appointment during the survey window, a non-clinical staff at the RBHA may conduct the survey over the phone. As an alternative, a nonclinical staff at the provider site may contact the consumer by phone to ask for their participation in the survey, offering a return envelope for the completed survey to be mailed. All return envelopes provided as a means for the consumer to submit their completed survey must be addressed to the RBHA. A check box in the questionnaire will be used to track the distribution method. The adult survey will be administered to the adult consumer. If the individual requests assistance, a guardian may complete the questionnaire on the consumer‟s behalf. The YSS-F will be administered to the parent/guardian of the child receiving services. If the parent or guardian is not at the appointment, then the survey will not be provided. List of Survey Clients ADHS/DBHS will provide each RBHA with a list of survey consumers to track those who complete a survey or the reason they did not complete a survey. The list of survey consumers will contain fields for:   5 RBHA ID Contractor ID            Survey ID AHCCCS ID Last Name First Name Date of Birth Sex Survey Offered Date Survey Completed Date Reason Not Completed Provider ID Client contact phone number The RBHAs will submit their lists of survey clients to ADHS/DBHS bi-weekly throughout the survey administration period to keep ADHS/DBHS informed regarding the status of response rates and reasons for non-participation of sampled consumers. Due dates for submission of bi-weekly lists are included in the Timeline Attachment to this Protocol. Since this document contains protected health information, it will be treated as a confidential document. ADHS/DBHS Post-administration Follow-up During the survey administration period, RBHAs will attempt to achieve participation of all consumers on its list of survey clients. Every effort will be made to separate service delivery from sample collection. At the end of the administration period, RBHAs will provide ADHS/DBHS with the completed list of survey consumers indicating which consumers completed the survey and which did not, with the reasons and a contact phone number for those who did not participate. At that point, the RBHA will delete the list of survey consumers. RBHAs will submit to ADHS/DBHS the completed paper surveys, retaining copies for reference, and databases containing survey results by July 25, 2014. Tribal RBHAs participate in the Annual Consumer Survey by way of a convenience sampling of their enrolled consumers. These entities will be provided a master survey tool for each population that can be copied for distribution. Tribal RBHAs will submit to ADHS/DBHS the completed paper surveys, retaining copies for reference, by July 25, 2014. ADHS/DBHS will enter the survey data collected at the tribal RBHAs into an SPSS database and provide results to each Tribal BHAS in Excel spreadsheets. Survey Administration Survey Timeframe The survey will be administered for a period of eight weeks: March 31 through May 23, 2014. 6 Roles and Responsibilities ADHS/DBHS is responsible for the statewide oversight of the survey administration to ensure consistent implementation of the survey protocol. The protocol, client sample, survey instruments, and survey results database will be created by ADHS/DBHS. ADHS/DBHS will run the analysis to determine RBHA survey results and provide the results to each RBHA from which the RBHAs will write their reports. ADHS/DBHS will provide technical assistance throughout the survey process. Periodic monitoring, training, timelines, and use of checklist will be utilized to guide the T/RBHAs on critical points in the process. The T/RBHAs have the primary responsibility for ensuring that the protocol is precisely followed within their geographic regions. Direct oversight and assistance will be provided by the T/RBHAs to their providers. The T/RBHAs will ensure that the providers are appropriately trained and prepared to administer the survey. The RBHAs will enter data into the two ADHS/DBHS provided databases but the RBHAs will not alter the databases in any way including creating tables, changing the names of tables or fields, or creating a different value coding scheme. Each provider agency is primarily responsible for each of its sites in which the survey is to be administered. Each site will maintain all necessary materials for survey administration. At each site, a drop box and a designated area will be provided for consumers to complete the survey. Providers will also be responsible for the day-to-day operations – including having the survey tools, materials for completing the survey (pens, pencils, clipboards), envelopes for return of the survey if needed, assigned resources for administration and collection of data for the survey. Non-Randomly Selected Consumers (Walk-in Requests) ADHS/DBHS does not require the RBHAs to extend survey participation efforts beyond the randomly selected populations but at the individual discretion of each RBHA/GSA, the RBHA may furnish its provider agencies with survey forms to give consumers who express a desire to participate in the survey but whose names do not appear on the lists of survey consumers (i.e., the consumer was not randomly selected). However, to maintain the scientific rigor of the protocol, the survey questionnaire completed by this group of respondents will be tracked separately by the RBHA. Names of individuals who belong to this group will not be added to the survey client list. This group will be tracked in some other ways as described in the succeeding paragraph. If the RBHA elects to collect responses from non-selected consumers, the same protocol will be followed for this group of respondents but the staff administering the survey will ensure that these “voluntary” surveys are kept separate from those on the lists of survey clients (randomly selected consumers). Several control measures will be used. First, 7 consumers that are not randomly selected to participate in the survey (i.e., walk-in respondents) will be given a copy of the survey questionnaire with the pre-filled section of the questionnaire not completed. Second, the survey questionnaire that will be used for non-randomly selected consumers will be color-coded. (Note: Surveys for the random sample will be printed on white paper.) RBHAs will enter the responses on the voluntary surveys into the same database as the results of the selected respondents, using the correct indicator, “C” for control (selected) respondents and “V” for voluntary (nonselected) respondents. Pre-Survey Activities Notification to Consumers about the Survey To encourage greater participation, efforts will be made to inform consumers in advance about the survey. T/RBHAs and providers will be encouraged to utilize all or a combination of any of the following media: flyers, posters, website announcements, or other promotional materials. A staff member or members at each provider site will be assigned to work on the survey. In addition to daily survey administrative duties, a component of the staff member‟s role will include assisting consumers with the survey if necessary. Assistance may include: reading the survey to individuals unable to read, explaining the Likert scale used for scoring answers, emphasizing confidentiality of the survey, or ensuring consumers that participation in the survey is voluntary. Staff will be allowed to provide administrative assistance to the consumer, or provide encouragement to participate in the survey process. However, staff will not be able to explain the meaning of particular questions or provide interpretations on what particular questions mean. Data Management and Reporting Scoring Protocol The scoring protocol that will be utilized for evaluating the domain areas within the survey are as follows: 1. Recode ratings of „not applicable‟ as missing values 2. For each survey, exclude domains where more than one-third of the domain questions are missing 3. Calculate the mean of the items for each respondent 4. Calculate the percent of scores that are greater than or equal to 3.5 through 5 Technical assistance ADHS will provide technical assistance to the RBHAs as needed. Response Rate Calculation: The rate will be calculated for each population for each GSA using the formula: 8 Response rate = A / B Where: A= Total number of surveys returned B= Total number of clients in sample that were contacted and asked to participate in the survey Weighting methodology The statewide data will be weighted by GSA client population to compensate for the stratified sample collection. Weights will be applied to the survey data prior to any statewide data analysis. Dissemination of Findings The following methods have been identified to circulate findings: Reporting of survey results in management meetings – Executive Management, Quality Management, Human Rights Committees, Behavioral Health Planning Council, other consumer advocate groups, and other interest groups that may be identified  Dissemination of information by providers to their local communities  Having copies of the survey available at the provider sites  Publishing results of survey on ADHS, RBHA and provider websites  9 2014 Consumer Survey Report ATTACHMENT-C 29 Attachment C - Raw Data Arizona Department of Health Services Division of Behavioral Health Services 2014 Annual Consumer Survey Assumptions for interpreting data: 1 2 3 4 Denominator - Number of responses to specific domain or question on completed/analyzed surveys. Numerator - Number of positive responses to specific domain or question on completed/analyzed surveys. RBHA - Specific numbers and percentages are based on actual valid survey returns. Tribal BHA scores are not included in Statewide numbers/percentages due to Tribal BHA convenience sampling methodology. 2014 MHSIP Consumer Survey Numbers Used in Calculation of Performance ADULT SURVEY RESULTS BY T/RBHA & MHSIP DOMAIN Domain: General Satisfaction T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 346 301 366 273 360 1,646 27 70 164 129 Numerator 305 263 321 241 314 1,444 23 57 150 125 Positive Response Percentage 88% 87% 88% 88% 87% 88% 85% 81% 91% 97% Denominator 345 301 362 271 359 1,638 24 70 164 129 Numerator 296 265 321 239 297 1,418 21 58 148 104 Positive Response Percentage 86% 88% 89% 88% 83% 87% 88% 83% 90% 81% Numerator 295 273 326 235 317 1,446 22 58 144 91 Positive Response Percentage 90% 94% 93% 91% 91% 92% 88% 85% 93% 79% Numerator 319 273 336 245 315 1,488 Positive Response Percentage 93% 92% 93% 91% 89% 92% Domain: Service Access T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Participation in Treatment Planning T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 328 291 352 258 347 1,576 25 68 155 115 Domain: Service Quality & Appropriateness T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Denominator 343 298 362 269 354 1,626 Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache 26 70 163 123 24 59 151 104 92% 84% 93% 85% Denominator 337 293 354 262 345 1,591 24 69 162 128 Numerator 247 218 284 178 257 1,184 18 59 137 92 Positive Response Percentage 73% 74% 80% 68% 74% 74% 75% 86% 85% 72% Denominator 336 293 356 264 352 1,601 26 70 164 128 Numerator 244 210 273 176 250 1,153 20 55 143 95 Positive Response Percentage 73% 72% 77% 67% 71% 72% 77% 79% 87% 74% Denominator 335 291 356 267 350 1,599 26 68 162 131 Numerator 273 235 302 209 288 1,307 24 56 143 116 Positive Response Percentage 81% 81% 85% 78% 82% 82% 92% 82% 88% 89% Domain: Outcomes T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Domain: Improved Functioning T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Social Connectedness T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache STATEWIDE RESULTS FOR EACH QUESTION IN DOMAIN General Satisfaction: Q1 Q2 Q3 Service Access Q4 Q5 Q6 Denominator 1,649 1,625 1,617 Numerator 1,481 1,379 1,422 Positive Response Percentage 90% 85% 88% 1,625 1,638 1,599 1413 1428 1261 87% 87% 79% Q7 Q8 Q9 1,640 1,636 1,565 1468 1392 1,172 90% 85% 75% Participation in Treatment Planning Q11 1,627 Q17 1,596 1,470 1,365 90% 86% Service Quality & Appropriateness Q10 Q12 Q13 Q14 Q15 Q16 Q18 Q19 Q20 1,592 1,623 1,633 1,614 1,592 1,620 1,578 1,608 1,571 1,410 1,380 1,502 1,446 1,333 1,483 1,394 1,407 1,355 89% 85% 92% 90% 84% 92% 88% 88% 86% Outcomes Q21 Q22 Q23 Q24 Q25 Q26 Q27 Q28 1,621 1,623 1,617 1,560 1,598 1,197 1,486 1,581 1,260 1,256 1,208 1,163 1,082 755 1,041 1,022 78% 77% 75% 75% 68% 63% 70% 65% Improved Functioning Q28 Q29 Q30 Q31 Q32 1,581 1,598 1,601 1,607 1,608 1,022 1,193 1,232 1,122 1,176 65% 75% 77% 70% 73% Social Connectedness Q33 Q34 Q35 Q36 1,597 1,600 1,576 1,589 1,258 1,284 1,077 1,309 79% 80% 68% 82% 2014 MHSIP Consumer Survey Numbers Used in Calculation of Performance YOUTH SERVICES SURVEY FOR FAMILIES RESULTS BY T/RBHA & MHSIP DOMAIN Domain: General Satisfaction T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 361 209 265 307 189 1,331 8 1 55 17 Numerator 314 183 226 272 154 1,149 8 1 54 13 Positive Response Percentage 87% 88% 85% 89% 81% 86% 100% 100% 98% 76% Denominator 357 209 262 306 187 1,321 7 1 55 17 Numerator 326 196 245 283 151 1,201 7 1 50 13 Positive Response Percentage 91% 94% 94% 92% 81% 91% 100% 100% 91% 76% Denominator 359 210 265 304 189 1,327 6 1 54 17 Numerator 338 198 248 291 174 1,249 6 1 48 12 Positive Response Percentage 94% 94% 94% 96% 92% 94% 100% 100% 89% 71% Denominator 356 209 258 301 181 1,305 Numerator 343 203 251 296 174 1,267 Positive Response Percentage 96% 97% 97% 98% 96% 97% Domain: Service Access T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Participation in Treatment Planning T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Domain: Cultural Sensitivity T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache 8 1 53 17 8 1 51 15 100% 100% 96% 88% Denominator 357 210 262 303 188 1,320 8 1 53 17 Numerator 240 151 179 205 116 891 6 0 45 12 Positive Response Percentage 67% 72% 68% 68% 62% 68% 75% 0% 85% 71% Denominator 357 210 263 303 188 1,321 8 1 53 17 Numerator 247 156 187 210 118 918 6 0 46 12 Positive Response Percentage 69% 74% 71% 69% 63% 69% 75% 0% 87% 71% Denominator 356 207 259 299 186 1,307 7 1 54 17 Numerator 322 198 239 273 163 1,195 7 1 52 15 Positive Response Percentage 90% 96% 92% 91% 88% 91% 100% 100% 96% 88% STATEWIDE RESULTS FOR EACH QUESTION IN DOMAIN General Satisfaction: Denominator Numerator Q1 1,332 1,154 Q4 1,310 1,102 Q5 1,302 1,104 Q7 1,328 1,152 Q10 1,324 1,112 Q11 1,324 1,070 Positive Response Percentage 87% 84% 85% 87% 84% 81% Domain: Outcomes T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Domain: Improved Functioning T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mtn Domain: Social Connectedness T/RBHA GSA 1 - NARBHA GSA 2 - Cenpatico 2 GSA 3 - Cenpatico 3 GSA 4 - Cenpatico 4 GSA 5 - CPSA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mtn Service Access Q8 1,331 1,181 89% Q9 Participation in Treatment Planning Q2 Q3 Q6 1,323 1,165 88% 1,313 1,324 1,316 1,189 1,235 1,255 91% 93% 95% Cultural Sensitivity Q12 Q13 Q14 Q15 1,332 1,285 1,332 1,278 1,275 1,204 1,288 1,182 96% 94% 97% 92% Outcomes Q16 Q17 Q18 Q19 Q20 Q21 Q22 1,323 1,320 1,317 1,296 1,308 1,307 1,318 932 905 940 851 784 815 894 70% 69% 71% 66% 60% 62% 68% Improved Functioning Q16 Q17 Q18 Q19 Q20 Q22 1,323 1,320 1,317 1,296 1,308 1,318 932 905 940 851 784 894 70% 69% 71% 66% 60% 68% Social Connectedness Q23 Q24 Q25 Q26 1,309 1,306 1,304 1,293 1,156 1,184 1,133 1,125 88% 91% 87% 87% CLAS (Added in 2011) Q30 Q31 1,290 708 1,254 571 97% 81% 2014 Consumer Survey Report ATTACHMENT-D 30 2014 Adult Demographics 2014 Adult Consumer Survey, Percent of Positive Response by Domain and Subgroup Subgroup Gender Male Female Age Group 18-21 22-30 31-45 46-65 66-75 76+ Race White only African American only Asian only Am Indian/Al Native only Nat Hawaiian/Pacific Islander only Multiple Race Ethnicity Hispanic or Latino Not Hispanic or Latino Length of Services 0-6 months 7-11 months 1-2 years 3-5 years 5 years + Program SMI Non-SMI General Satisfaction Service Access Service Quality and Appropriateness Participation in Treatment Planning Outcomes Improved Functioning Social Connectedness N % N % N % N % N % N % N % 548 890 88% 88% 541 871 88% 86% 560 921 92% 92% 545 895 92% 92% 453 728 76% 74% 445 705 74% 71% 496 804 83% 81% 72 197 460 609 76 18 85% 87% 87% 89% 92% 95% 70 195 451 595 74 18 82% 86% 86% 88% 89% 95% 76 207 478 613 82 19 91% 92% 92% 91% 98% 100% 70 204 461 599 78 19 90% 94% 92% 91% 94% 100% 60 160 368 496 72 18 71% 74% 72% 75% 89% 95% 56 161 359 483 67 16 68% 74% 70% 72% 83% 84% 68 185 418 534 73 17 81% 83% 82% 81% 89% 94% 1066 46 3 88% 90% 100% 1034 49 2 86% 96% 67% 1097 47 2 92% 96% 67% 1060 45 3 92% 92% 100% 868 37 2 74% 76% 67% 849 33 3 72% 66% 100% 963 40 3 82% 80% 100% 47 83% 49 86% 51 88% 49 89% 36 66% 35 66% 42 78% 7 39 88% 81% 8 32 100% 67% 8 40 100% 83% 7 38 88% 83% 7 30 88% 65% 6 31 75% 65% 6 36 75% 77% 474 816 90% 87% 473 789 90% 84% 483 846 93% 91% 475 821 94% 91% 394 666 78% 73% 382 654 75% 71% 438 733 86% 80% 33 66 157 239 920 85% 90% 86% 89% 88% 34 61 148 241 903 87% 85% 82% 91% 86% 35 67 157 248 954 92% 93% 87% 94% 92% 35 62 155 235 930 92% 90% 90% 93% 92% 17 56 122 209 756 50% 78% 70% 80% 74% 17 51 116 204 737 47% 72% 67% 78% 72% 30 62 133 233 821 77% 86% 76% 88% 81% 738 703 87% 89% 729 686 86% 88% 766 718 90% 93% 760 683 92% 92% 604 577 73% 76% 592 557 71% 73% 658 645 80% 84% 2014 Consumer Survey Report ATTACHMENT-E 31 2014 Adult Line Item 2014 Adult Consumer Survey, Statewide Percent of Positive Response by Line Item (Line specific numbers are based on actual valid survey returns.) Survey Item General Satisfaction: 1. I like the services that I received here. 2. If I had other choices, I would still get services from this agency. 3. I would recommend this agency to a friend or family member. Service Access: 4. The location of services was convenient (parking, public transportation, distance, etc.) 5 Staff were willing to see me as often as I felt it was necessary.. 6. Staff returned my call in 24 hours. 7. Services were available at times that were good for me. 8. I was able to get all the services I thought I needed. 9. I was able to see a psychiatrist when I wanted to. Participation in Treatment Planning 11. I felt comfortable asking questions about my treatment and medication. 17. I, not staff, decided my treatment goals. Service Quality and Appropriateness 10. Staff here believe that I can grow, change and recover. 12. I feel free to complain. 13. I was given information about my rights. 14. Staff encouraged me to take responsibility for how I live my life. 15. Staff helped me to understand what side effects to watch out for. 16 Staff respected my wishes about who is and who is not to be given information about my treatment.. 18. Staff were sensitive to my cultural background (For example: values, traditions, beliefs, race, language, etc.) 19. Staff helped me obtain the information I needed so that I could take charge of managing my illness. 20. I was encouraged to use consumer-run programs (support groups, drop-in centers, crisis phone line, etc.) Outcomes 21. I deal more effectively with daily problems. 22. I am better able to control my life. 23. I am better able to deal with crisis. 24 I am getting along better with my family. 25. I do better in social situations. 26. I do better in school and/or work 27. My housing situation has improved 28. My symptoms are not bothering me as much. Number Percent 1,444 1,481 1,379 1,422 1,418 88% 90% 85% 88% 87% 1,413 1,428 1,261 1,468 1,392 1,172 1,446 1,470 1,365 1,488 1,410 1,380 1,502 1,446 1,333 87% 87% 79% 90% 85% 75% 92% 90% 86% 92% 89% 85% 92% 90% 84% 1,483 92% 1,394 88% 1,407 88% 1,355 1,184 1,260 1,256 1,208 1,163 1,082 755 1,041 1,022 86% 74% 78% 77% 75% 75% 68% 63% 70% 65% Improved Functioning 28. My symptoms are not bothering me as much. 29. I do things that are more meaningful to me. 30. I am better able to take care of my needs. 31. I am better able to handle things when they go wrong. 32 I am better able to do things that I want to do. Social Connectedness 33. I am happy with the friendships I have. 34. I have people with whom I can do enjoyable things. 35. I feel I belong in my community. 36. In a crisis, I would have the support I need from family or friends. 1,153 1,022 1,193 1,232 1,122 1,176 1,307 1,258 1,284 1,077 1,309 72% 65% 75% 77% 70% 73% 82% 79% 80% 68% 82% 2014 Consumer Survey Report ATTACHMENT-F 32 2014 YSS-F Demographics 2014 YSS-F, Percent of Positive Response by Domain and Subgroup General Satisfaction Subgroup Service Access Cultural Sensitivity Participation in Treatment Planning Outcomes Improved Functioning Social Connectedness N % N % N % N % N % N % N % 750 394 87% 86% 777 419 91% 92% 828 434 98% 96% 824 420 96% 92% 583 304 68% 67% 598 316 70% 69% 772 419 91% 92% 4 354 791 100% 86% 86% 4 389 808 100% 95% 89% 4 396 867 100% 98% 97% 4 388 857 100% 94% 94% 2 275 614 50% 67% 68% 2 281 635 50% 69% 70% 4 375 816 100% 93% 91% 840 26 1 45 87% 63% 50% 85% 874 38 2 46 91% 93% 100% 90% 919 37 2 50 98% 90% 100% 98% 913 36 2 48 95% 86% 100% 96% 652 25 1 29 68% 60% 50% 58% 672 24 1 30 70% 57% 50% 60% 864 34 2 44 91% 85% 100% 86% 3 56 100% 89% 3 57 100% 91% 2 60 67% 97% 2 59 67% 94% 2 44 67% 70% 1 47 33% 75% 3 59 100% 94% 490 592 88% 85% 511 623 92% 90% 537 661 97% 97% 521 659 93% 95% 391 447 70% 65% 398 467 72% 67% 520 607 94% 89% 70 88 219 277 460 83% 86% 86% 87% 87% 71 95 231 293 473 87% 93% 92% 93% 90% 80 98 244 298 508 94% 99% 98% 96% 98% 78 95 237 301 503 93% 94% 94% 95% 95% 52 61 174 217 360 61% 60% 69% 69% 68% 52 62 178 223 376 61% 61% 71% 71% 71% 80 90 224 288 480 95% 92% 90% 93% 91% 924 166 87% 85% 960 175 92% 89% 1012 181 97% 96% 1002 174 95% 90% 711 128 68% 65% 734 132 70% 67% 960 173 92% 89% Gender Male Female Age Group 0-4 5-12 13-17 Race White only African American only Asian only Am Indian/Al Native only Nat Hawaiian/Pacific Islander only Multiple Race Ethnicity Hispanic or Latino Not Hispanic or Latino Length of Services 0-6 months 7-11 months 1-2 years 3-5 years 5 years + CFT Yes No 2014 Consumer Survey Report ATTACHMENT-G 33 2014 YSS-F Line Item 2014 YSSF, Statewide Percent of Positive Response by Line Item (Line specific numbers are based on actual valid survey returns.) Survey Item General Satisfaction: 1. Overall, I am satisfied with the services my child received. 4. The people helping my child stuck with us no matter what. 5. I felt my child had someone to talk to when he/she was troubled. 7. The services my child and/or family received were right for us. 10. My family got the help we wanted for my child. 11. My family got as much help as we needed for my child. Service Access 8. The location of services was convenient for us. 9. Services were available at times that were convenient for us. Participation in Treatment Planning 2. I helped to choose my child’s services. 3. I helped to choose my child’s treatment goals. 6. I participated in my child’s treatment. Cultural Sensitivity 12 Staff treated me with respect. 13. Staff respected my family’s religious/spiritual beliefs. 14. Staff spoke with me in a way that I understood. 15. Staff were sensitive to my cultural/ethnic background. Outcomes 16 My child is better at handling daily life. 17. My child gets along better with family members. 18. My child gets along better with friends and other people. 19. My child is doing better in school and/or work. 20. My child is better able to cope when things go wrong. 21. I am satisfied with our family life right now. 22. My child is better able to do things he or she wants to do. Improved Functioning 16 My child is better at handling daily life. 17. My child gets along better with family members. 18. My child gets along better with friends and other people. 19. My child is doing better in school and/or work. 20. My child is better able to cope when things go wrong. 22. My child is better able to do things he or she wants to do. Social Connectedness 23. I know people who will listen and understand me when I need to talk. 24. I have people that I am comfortable talking with about my child’s problems. 25. In a crisis, I would have the support I need from family or friends. 26. I have people with whom I can do enjoyable things. Number Percent 1,149 1,154 1,102 1,104 1,152 1,112 1,070 1,201 1,181 1,165 1,249 1,189 1,235 1,255 1,267 1,275 1,204 1,288 1,182 891 932 905 940 851 784 815 894 918 932 905 940 851 784 894 1,195 1,156 1,184 1,133 1,125 86% 87% 84% 85% 87% 84% 81% 91% 89% 88% 94% 91% 93% 95% 97% 96% 94% 97% 92% 68% 70% 69% 71% 66% 60% 62% 68% 69% 70% 69% 71% 66% 60% 68% 91% 88% 91% 87% 87% 2014 Consumer Survey Report ATTACHMENT-H 34 2014 Consumer Survey Report List of Acronyms 2014Consumer Survey Report Acronyms Description ADHS/DBHS ASOC CBHS CFT CPSA CSOC CSOCPR CSR GMH GSA MHSIP MMWIA NARBHA NASMHPD NOMs NRI QM RBHA RSS SA SAMSHA SMI T/RBHA WSDSG YSS-F Arizona Department of Health Services/Division of Behavioral Health Services Adult System of Care Cenpatico Behavioral Health Services Child and Family Team Community Partnership of Southern Arizona Children’s System of Care Child System of Care Practice Review Consumer Survey Report General Mental Health Geographical Service Area Mental Health Statistics Improvement Program Meet Me Where I Am Northern Arizona Regional Behavioral Health Authority National Association of State Mental Health Program Directors National Outcomes Measures National Research Institute Quality Management Regional Behavioral Health Authority Recovery Support Specialist Substance Abuse Substance Abuse and Mental Health Services Administration Serious Mental Illness Tribal/Regional Behavioral Health Authority Western States Decision Support Group Youth Services Survey for Families 35 APPENDIX 1 Year 2014 Adult Consumer Survey (English and Spanish) THIS SECTION MUST BE COMPLETED BY T/RBHA OR PROVIDER! Name of Service Agency: ____________________________________________________________________ T/RBHA NAME: _________________________ SMI  Program/Fund Source: Client’s Entitlement Status: Survey completed at: Facility ID: ______________ Non-SMI (GMH or SA)  TXIX  TXXI  Survey ID: ________________________ Home  Clinic  By Phone  Client enrolled in DDD: Yes _____ No _____ YEAR 2014 ADULT CONSUMER SURVEY Please help us improve our program by completing this survey about the services you have received in the last six months. We are interested in your honest opinion. All responses will be treated as confidential. Any personal information will be excluded in the presentation of the survey results. Your current and/or future services will not be affected if you decide not to participate in this survey. If you have already taken this survey during 2014, you do not need to complete it again. After you have completed the survey, please fold and drop it in the survey box before you leave the office today. Thank you. Use Pen or Pencil Please fill in the bubbles completely with your answers to the following questions: Information about the Person Receiving Services: Age: Sex:  Male  Female Ethnicity:  Hispanic or Latino  Not Hispanic or Latino Please check all applicable race categories: Race:  White  Black/African American  Native Hawaiian/Pacific Islander  Asian  American Indian/Alaska Native How long have you been receiving mental health and/or substance abuse services? (from any provider)  0 - 6 months  7 - 11 months  1 - 2 years  3 - 5 years  more than 5 years About the Person Completing This Survey: I am the person directly receiving services.  Yes  No If not, please check your relation to the person who is directly receiving services:  Parent/Guardian  Friend  Relative (Uncle, Aunt, Cousin, Grandparent, etc.) Please continue to answer questions on the next page.  Other Por favor siéntase libre de usar el espacio proporcionado abajo para comentar sobre cualquiera de sus respuestas. También, si hay áreas que no fueron cubiertas por este cuestionario que usted siente que deberían haber sido incluidas, por favor escríbalas en la sección de comentarios. Gracias por su tiempo y cooperación en completar éste cuestionario. ¿Cuáles han sido algunas cosas más provechosas sobre los servicios que usted recibió durante los 6 meses pasados? ¿Qué mejoraría los servicios qué usted recibe aquí? Por favor escriba cualquier otro comentario que usted quiera compartir: Gracias por completar este cuestionario. Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 1. I like the services that I received here.       2. If I had other choices, I would still get services from this agency.       3. I would recommend this agency to a friend or family member. 4. The location of services was convenient (parking, public transportation, distance, etc.) 5. Staff were willing to see me as often as I felt it was necessary.                   6. Staff returned my call in 24 hours.       7. Services were available at times that were good for me.       8. I was able to get all the services I thought I needed.       9. I was able to see a psychiatrist when I wanted to.             In order to provide the best possible behavioral health services, we need to know what you think about the services you received DURING THE LAST 6 MONTHS, the people who provided it, and the results. 10. Staff here believe that I can grow, change and recover. 11. I felt comfortable asking questions about my treatment and medication.       12. I feel free to complain.       13. I was given information about my rights.       14. Staff encouraged me to take responsibility for how I live my life.       15. Staff helped me to understand what side effects I should watch out for. 16. Staff respected my wishes about who is and who is not to be given information about my treatment. 17. I, not staff, decided my treatment goals. 18. Staff were sensitive to my cultural background. (For example: values, traditions, beliefs, race, language, etc.) 19. Staff helped me obtain the information I needed so that I could take charge of managing my illness. 20. I was encouraged to use consumer-run programs (support groups, drop-in centers, crisis phone line, etc.)                                     21. I deal more effectively with daily problems.       22. I am better able to control my life.       23. I am better able to deal with crisis.       24. I am getting along better with my family.       25. I do better in social situations.       26. I do better in school and/or work.       27. My housing situation has improved.       28. My symptoms are not bothering me as much.       29. I do things that are more meaningful to me.       30. I am better able to take care of my needs.       31. I am better able to handle things when they go wrong.       32. I am better able to do things that I want to do.       As a direct result of the services I received: Please continue to answer questions on the next page. Muy de acuerdo De acuerdo Neutro En Desacuerdo Muy en desacuerdo No Aplica POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 33. Soy feliz con las amistades que tengo.       34. Tengo a personas con quienes puedo hacer cosas agradables.       35. Siento que pertenezco a mi comunidad.       36. En una crisis, yo tendría el apoyo que necesito de familia o amigos.       Preguntas adicionales del estado: 37. En la creación de mi plan de servicios, mis preferencias culturales fueron incluidas. (Por ejemplo; mis valores, tradiciones, creencias, raza, lenguaje, etc.). 38. Los servicios me fueron proporcionados en un lenguaje que entendí.             39. En la creación de mi plan de servicios las tradiciones y creencias de mi tribu fueron incluidas (Por ejemplo: me preguntaron si quería participar en sesiones de sanación tradicional). Si usted no participa de las tradiciones tribales, por favor marque la opción "No aplica".       Para las preguntas 33-36 por favor responda basado en las relaciones con personas diferentes de su proveedor (es) de salud mental. Por favor conteste las siguientes preguntas para hacernos saber cómo se encuentra. 40. ¿Durante cuánto tiempo ha recibido usted servicios de  Menos de un año (menos de 12 meses) salud mental de éste proveedor? (Continué con la pregunta 41)  1 año o más (al menos 12 meses) (Vaya a la pregunta 44) Si su respuesta es “menos de un año (menos de 12 meses)”, por favor complete las preguntas 41-43. 41. ¿Ha sido usted arrestado desde que comenzó a recibir  Sí  No servicios de salud mental? 42. ¿Ha sido usted arrestado durante los 12 meses  Sí  No anteriores a esto? 43. Desde que usted comenzó a recibir servicios de salud  han disminuido (por ejemplo, no he sido mental, ha tenido incidentes con la policía… arrestado, molestado por la policía, enviado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (yo no he tenido ningún incidente con la policía durante este año o el año pasado) Si su respuesta es “1 año o más (al menos 12 meses)”, por favor complete las preguntas 44 - 46. 44. ¿Ha sido arrestado durante los 12 meses pasados?  Sí  No 45. ¿Ha sido arrestado durante los 12 meses anteriores a esto? 46. Durante el año pasado, sus incidentes con la policía …  Sí  No  han disminuido (por ejemplo, no he sido arrestado, molestado por policía, enviado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (yo no he tenido ningún encuentro con la policía durante este año o el año pasado) Por favor siga contestando preguntas en la siguiente página. Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 33. I am happy with the friendships I have.       34. I have people with whom I can do enjoyable things.       35. I feel I belong in my community.       36. In a crisis, I would have the support I need from family or friends.                         For questions 33-36 please answer for relationships with persons other than your mental health provider(s) State-added Questions: 37. In creating my service plan my cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) 38. Services were provided in a language I understood. 39. In creating my service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) If you do not participate in tribal customs, please mark Not Applicable. Please answer the following questions to let us know how you are doing. 40. How long have you received mental health services from this provider?  Less than a year (less than 12 months) (Continue to Question 41)  1 year or more (at least 12 months) (Skip to Question 44) If you answered “Less than a year (less than 12 months)”, please complete questions 41 - 43. 41. Were you arrested since you began to receive mental health services? 42. Were you arrested during the 12 months prior to that? 43. Since you began to receive mental health services, have your encounters with police…  Yes  No  Yes  No  been reduced (for example, I have not been arrested, hassled by police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (I had no police encounters this year or last year) If you answered “1 year or more (at least 12 months)”, please complete questions 44 - 46. 44. Were you arrested during the last 12 months?  Yes 45. Were you arrested during the 12 months prior to that? 46. Over the last year, have your encounters with police…  No  Yes  No  been reduced (for example, I have not been arrested, hassled by police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (I had no police encounters this year or last year) Please continue to answer questions on the next page. Muy de acuerdo De acuerdo Neutro En Desacuerdo Muy en desacuerdo No Aplica POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 1. Me gustaron los servicios que recibí aquí. 2. Si yo tuviera otras opciones, yo todavía preferiría los servicios de esta agencia. 3. Yo recomendaría ésta agencia a un miembro de la familia o un amigo. 4. La ubicación de los servicios fue conveniente (estacionamiento, transporte público, distancia, etc.) 5. El personal quiso verme tan a menudo como sentí que era necesario.                               6. El personal devolvió mi llamada en 24 horas. 7. Los servicios estaban disponibles a las horas que eran convenientes para mí. 8. Recibí los servicios que pensé que necesitaba.                   9. Pude ver a un Psiquiatra cuando así lo solicité. A fin de proporcionar un mejor servicio en los cuidados de salud mental y de abuso de sustancias, tenemos que saber lo que usted piensa sobre los servicios que recibió DURANTE los 6 MESES PASADOS, la gente que lo proporcionó, y los resultados.       10. El personal aquí cree que puedo crecer, cambiar y recuperarme. 11. Me sentí cómodo haciendo preguntas acerca de mi tratamiento y medicación. 12. Me siento libre de presentar mis inquietudes sobre los servicios.                   13. Me dieron la información sobre mis derechos. 14. El personal me animó a tomar la responsabilidad de como vivo mi vida. 15. El personal me ayudó a entender acerca de los efectos secundarios sobre los que debo estar preparado. 16. El personal respetó mis deseos sobre a quién sí y a quien no deben dar información sobre mi tratamiento. 17. Yo, y no el personal, decidí las metas de mi tratamiento.                               18. El personal fue respetuoso de mis raíces culturales/étnicas. (Por ejemplo: mis valores, tradiciones, creencias, raza, lenguaje, etc.). 19. El personal me ayudó a obtener la información que necesité de modo que yo pudiera hacerme cargo de manejar mi enfermedad. 20. Fui animado a usar programas manejados por consumidores (grupos de                   Como un resultado directo de los servicios recibí: 21. Manejo de forma más eficaz los problemas diarios.       22. Soy más capaz de controlar mi vida.       23. Soy más capaz de lidiar con las crisis.       24. Me llevo mejor con los miembros de mi familia.       25. Me desenvuelvo mejor en situaciones sociales.       26. Tengo mejor desempeño en la escuela y/o trabajo.       27. Mi situación de vivienda ha mejorado.       28. Los síntomas no me molestan tanto como antes.       29. Hago cosas que son más significativas para mí.       30. Soy más capaz de ocuparme de mis necesidades.       31. Soy más capaz de manejar las cosas o situaciones cuando no van bien.       32. Soy más capaz de hacer cosas que quiero hacer.       apoyo, centros de ayuda informal, línea telefónica de crisis, etc.) Por favor siga contestando preguntas en la siguiente página. Please feel free to use the space provided below to comment on any of your answers. Also, if there are areas which were not covered by this questionnaire which you feel should have been, please write them in the comments section. Thank you for your time and cooperation in completing this questionnaire. What have been some of the most helpful things about the services you received over the last 6 months? ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ What would improve the services that you receive here? ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Please list any other comments you may want to share: ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Thank you for completing this questionnaire. ¡¡ESTA SECCIÓN DEBE SER COMPLETADA POR LA RBHA O EL PROVEEDOR!! Nombre de Agencia de Servicio ____________________________________________________ NOMBRE de LA RBHA: ___________________________ SMI  Fuente de Programa/Fondo: El Estado de Elegibilidad del Cliente: Encuesta tomada en: Casa  ID de la agencia o proveedor: ___________ No - SMI (GMH o SA)  TXIX  TXXI  Clínica  Por teléfono  ID de la encuesta: _________________ Cliente inscrito en DDD: Si_____ No_____ ENCUESTA DE SERVICIOS PARA ADULTOS QUE RECIBIERON SERVICIOS DURANTE EL AÑO 2014 Por favor ayúdenos a mejorar nuestro programa completando ésta revisión sobre los servicios que usted ha recibido en los últimos seis meses. Estamos interesados en su opinión honesta. Todas las respuestas serán tratadas como confidenciales. Cualquier información personal será excluida en la presentación de los resultados ésta encuesta. Sus servicios actuales y/o futuros no se verán afectados si usted decide no participar en ésta encuesta. Si usted ha tomado ya ésta encuesta en 2014, no tiene que completarla otra vez. Después de que usted haya finalizado, por favor doble la hoja y deposítela en la caja de encuestas, antes de usted salga de la oficina hoy. Gracias. Use pluma o lápiz Por favor rellene las burbujas completamente con sus respuestas a las siguientes preguntas: Información acerca de la persona que recibe los servicios: Edad: _______ Sexo:  Masculino  Femenino Pertenencia étnica:  Hispano o Latino  No Hispano o Latino Por favor compruebe todas las categorías de raza aplicables Raza:  Blanco  Negro/Africano Americano  Asiático  Nativo Americano / Nativo de Alaska  Nativo de Hawai/Isleño del Pacífico ¿Por cuánto tiempo ha estado recibiendo servicios de salud mental y/o de abuso de sustancias? (de cualquier proveedor )  0 - 6 meses  7 - 11 meses  1 - 2 años  3 - 5 años  más de 5 años Sobre la persona que completa ésta encuesta: Soy la persona que directamente recibe servicios:  Sí  No Si no, por favor marque la relación con la persona que recibe directamente servicios:  Padre/Guardián  Amigo  Pariente (Tío, Tía, Primo, Abuelo, etc.) Por favor siga contestando preguntas en la siguiente página.  Otro APPENDIX 2 Year 2014 YSS-F Consumer Survey (English and Spanish) THIS SECTION MUST BE COMPLETED BY T/RBHA OR PROVIDER! Name of Service Agency: ____________________________________________________________________ T/RBHA NAME: ________________________ Client’s Entitlement Status: TXIX  Facility ID:_______________ TXXI  Survey ID: ____________________________ Survey completed at: Home  Clinic  By Phone  Client enrolled in DDD: Yes ______ No ______ YEAR 2014 YOUTH SERVICES SURVEY FOR FAMILIES Please help us improve our program by completing this survey about the services you have received in the last six months. We are interested in your honest opinion. All responses will be treated as confidential. Any personal information will be excluded in the presentation of the survey results. Your child and family’s current and/or future services will not be affected if you decide not to participate in this survey. If you have already taken this survey in 2014, then you do not need to complete it again. After you have completed the survey, please fold and drop it in the survey box before you leave the office today. Thank you. Use Pen or Pencil Please fill in the bubbles completely with your answers to the following questions: Information about the Person Receiving Services: Child’s Age: Child’s Sex:  Male  Female Child’s Ethnicity:  Hispanic or Latino  Not Hispanic or Latino Please check all applicable race categories: Child’s Race:  White  Black/African American  Native Hawaiian/Pacific Islander  Asian  American Indian/Alaska Native How long has your child been receiving mental health and/or substance abuse services? (from this provider)  0 - 6 months  7 - 11 months  1 - 2 years Please check your relationship with the child: Does your family have a Child and Family Team?  3 - 5 years  more than 5 years  Parent/Guardian  Friend  Relative (Uncle, Aunt, Cousin, Grandparent, etc.)  Other  Yes  No Please continue to answer questions on the next page. Por favor siéntase libre de usar el espacio proporcionado abajo para comentar sobre cualquiera de sus respuestas. También, si hay áreas que no fueron cubiertas por éste cuestionario que usted siente deberían haber sido incluidas, por favor escríbalas en la sección de comentarios. Gracias por su tiempo y cooperación en completar éste cuestionario. Por favor escriba cualquier otro comentario que usted quiera compartir: Gracias por completar éste cuestionario. Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 1. Overall, I am satisfied with the services my child received.       2. I helped to choose my child’s services.       3. I helped to choose my child’s treatment goals.       4. The people helping my child stuck with us no matter what.       5. I felt my child had someone to talk to when he/she was troubled.       6 I participated in my child’s treatment.       7. The services my child and/or family received were right for us.       8. The location of services was convenient for us.       9. Services were available at times that were convenient for us.       10. My family got the help we wanted for my child.       11. My family got as much help as we needed for my child.       12. Staff treated me with respect.       13. Staff respected my family’s religious/spiritual beliefs.       14. Staff spoke with me in a way that I understood. 15. Staff were sensitive to my cultural/ethnic background. (For example: values, traditions, beliefs, race, language, etc.)             16. My child is better at handling daily life.       17 My child gets along better with family members.       18. My child gets along better with friends and other people.       19. My child is doing better in school and/or work.       20. My child is better able to cope when things go wrong.       21 I am satisfied with our family life right now.       22. My child is better able to do things he or she wants to do.       Please help our agency make services better by answering some questions about the services your child received OVER THE LAST 6 MONTHS. As a result of the services my child and/or family received: As a result of the services my child and/or family received: please answer for relationships with persons other than your mental health provider(s). 23 I know people who will listen and understand me when I need to talk. 24. I have people that I am comfortable talking with about my child’s problems. 25 In a crisis, I would have the support I need from family or friends.                   26. I have people with whom I can do enjoyable things.       27. What has been the most helpful thing about the services you and your child received over the last 6 months? ________________________________________________________________________________________________ ________________________________________________________________________________________________ 28. What would improve the services here? _____________________________________________________________ ________________________________________________________________________________________________ ________________________________________________________________________________________________ Please continue to answer questions on the next page. Por favor siga contestando preguntas en la siguiente página. POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 44. Desde que comenzó a recibir servicios, el número de días que mi hijo estuvo en la escuela fue…  mayor  el mismo  menos  no se aplica (por favor seleccione por qué éste no se aplica)  el niño no tenía un problema con la asistencia antes de los servicios iniciales  el niño es demasiado joven para estar en la escuela  el niño fue expulsado de la escuela  el niño está siendo enseñado en la casa  el niño abandonó la escuela 45. ¿Ha sido su hijo arrestado durante los 12 meses pasados? 46. ¿Estuvo arrestado su hijo durante los 12 meses antes a ésto? 47. Durante el año pasado, los incidentes de su niño con la policía…  Otro  Sí  No  Sí  No  han disminuido (por ejemplo, no ha sido arrestado, molestado por la policía, llevado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable ( no tuvo ningún encuentro con policía éste año o el año pasado) 48. ¿Fue expulsado su niño o suspendido de la escuela durante los 12 meses pasados? 49. ¿Fue expulsado su niño o suspendido de la escuela durante los 12 meses antes de esto? 50. Durante el año pasado, el número de días mi niño fue a la escuela era…  Sí  No  Sí  No  mayor  el mismo  menos  no se aplica (por favor seleccione por qué éste no se aplica)  El niño no tenía un problema con la asistencia antes de servicios iniciales  El niño es demasiado joven para estar en la escuela  El niño fue expulsado de la escuela  El niño está siendo enseñado en la casa  El niño abandonó la escuela  Otro Por favor siga contestando preguntas en la siguiente página. Agree I am Neutral Disagree Strongly Disagree Not Applicable State-added Questions: 29. In creating my child’s service plan our cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) 30. Services were provided in a language my child understood. 31. In creating my child’s service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) If you do not participate in tribal customs, please mark Not Applicable. Strongly Agree PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY                   Please answer the following questions to let us know how your child is doing. 32. Is your child currently living with you?  Yes  No 33. Has your child lived in any of the following places in the last 6 months? (CHECK ALL THAT APPLY)  With one or both parents  With another family member  Foster home  Therapeutic foster home  Crisis shelter  Homeless shelter  Group home  Residential treatment facility  Hospital  Local jail or detention facility  State correctional facility  Runaway/homeless/on the streets  Other (describe):________________________________________________________________________________ 34. In the last year, did your child see a medical doctor (or nurse) for a health check up or because he/she was sick? (Check one)  Yes, in a clinic or office  Yes, but only in a hospital emergency room  No  Do not remember 35. Is your child on medication for emotional/behavioral problems? 36. If yes, did the doctor or nurse tell you and/or your child what side effects to watch for? 37. Is your child still getting services from this provider? 38. How long did your child receive services from this provider?  Yes  No  Yes  No  Yes  No  Less than 1 month  1-5 Months  6 months to 1 year  More than 1 year (skip to Question 45) 39. Was your child arrested since beginning to receive mental health services? 40. Was your child arrested during the 12 months prior to that? 41. Since your child began to receive mental health services, have their encounters with the police…  Yes  No  Yes  No  been reduced (for example, they have not been arrested, hassled by the police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (They had no police encounters this year or last year) 42. Was your child expelled or suspended from school since beginning services? 43. Was your child expelled or suspended from school during the 12 months prior to that?  Yes  No  Yes  No Please continue to answer questions on the next page. Por favor siga contestando preguntas en la siguiente página. De acuerdo Neutro En desacuerdo Muy en desacuerdo No Aplica Pregunta adicional del estado: 29. En la creación del plan de servicios de mi hijo, nuestras preferencias culturales fueron incluidas. (Por ejemplo; mis valores, tradiciones, creencias, raza, lenguaje, etc.). 30. Los servicios fueron proporcionados en un lenguaje que mi hijo entendiera. 31. En la creación del plan de servicios de mi hijo nuestras tradiciones y creencias tribales fueron incluidas (Por ejemplo: se nos preguntó si queríamos participar en sesiones de sanación tradicional). Si usted no participa de las tradiciones tribales, por favor marque la opción "No aplica". Muy De acuerdo POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE                   Por favor conteste las siguientes preguntas, para informarnos acerca de cómo va su hijo. 32. ¿Vive actualmente su hijo con usted?  Sí 33. ¿Ha vivido su hijo en cualquiera de los siguientes sitios en los últimos 6 meses?  No (MARQUE TODAS LAS QUE APLICAN)  Con uno o ambos padres  Con otro miembro de familia  Familia adoptiva terapéutica  Refugio de crisis  Centro de tratamiento  Casa de Grupo  Cárcel local o instalación de detención  Otro (describa)  Instalación correccional 34. ¿Durante el año pasado, visitó su hijo un médico (o enfermera) para un control de salud o porque él/ella estaba enfermo? (Elija una de las opciones)  Familia adoptiva  Refugio para los sin hogar  Hospital  Escapó/o vive en las calles  Sí, en una clínica o oficina  Sí, pero sólo en un cuarto de emergencia de hospital  No  No recuerdo 35. ¿Está su hijo tomando medicamentos para problemas de conducta / emocionales? 36. ¿Si la respuesta es afirmativa, responda si el médico o la enfermera le explicaron con qué clase de efectos secundarios debe estar alerta? 37. ¿Todavía está su hijo recibiendo servicios de éste proveedor? 38. ¿Por cuánto tiempo recibió su hijo servicios de éste proveedor?  Sí  No  Sí  No  Sí  No  Menos de 1 mes  1-5 Meses  6 Meses a 1 año  Más de 1 año (vaya a la pregunta 45) 39. ¿Ha sido su hijo arrestado desde que comenzó a recibir servicios de salud mental? 40. ¿Estuvo arrestado su hijo durante los 12 meses anteriores a ésto? 41. Desde que su hijo comenzó a recibir servicios de salud mental, los incidentes con la policía...  Sí  No  Sí  No  han disminuido (por ejemplo, no ha sido arrestado, molestado por la policía, llevado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (no tuvo ningún incidente con la policía éste año o el año pasado) 42. ¿Ha sido su hijo expulsado o suspendido de su escuela desde que comenzó a recibir los servicios? 43. ¿Fue su hijo expulsado o suspendido de su escuela durante los 12 meses anteriores a ésto?  Sí  No  Sí  No PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 44. Since starting to receive services, the number of days my child was in school is…  greater  about the same  less  does not apply (please select why this does not apply)  child did not have a problem with attendance before starting services  child is too young to be in school  child was expelled from school  child is home schooled  child dropped out of school  Other: 45. Was your child arrested during the last 12 months? 46. Was your child arrested during the 12 months prior to that? 47. Over the past year, have your child’s encounters with the police…  Yes  No  Yes  No  been reduced (for example, they have not been arrested, hassled by the police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (They had no police encounters this year or last year) 48. Was your child expelled or suspended from school during the last 12 months? 49. Was your child expelled or suspended from school during the 12 months prior to that? 50. Over the last year, the number of days my child was in school is…  Yes  No  Yes  No  greater  about the same  less  does not apply (please select why this does not apply)  child did not have a problem with attendance before starting services  child is too young to be in school  child was expelled from school  child is home schooled  child dropped out of school  Other: Please continue to answer questions on the next page. Muy De acuerdo De acuerdo Neutro En desacuerdo Muy en desacuerdo No Aplica POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 1. En general, estoy satisfecho con los servicios que mi hijo recibió.       2. Ayudé elegir los servicios de mi hijo.       3. Ayudé elegir los objetivos de tratamiento de mi hijo.       4. La gente que ayuda a mi hijo nos apoyó sin falta.       5. Sentí que mi hijo tenía a alguien para dirigirse cuando estaba preocupado.       6. Participé en el tratamiento de mi hijo. 7. Los servicios que mi hijo y/o la familia recibieron fueron los apropiados para nosotros. 8. La ubicación de los servicios fue conveniente para nosotros. 9. Los servicios estuvieron disponibles en momentos en que fue conveniente para nosotros. 10. Mi familia recibió la ayuda que estábamos buscando para mi hijo.                               11. Mi familia recibió tanta ayuda como necesitábamos para mi hijo.       12. El personal me trató con respeto.       13. El personal respetó las creencias religiosas/espirituales de mi familia.       14. El personal habló conmigo de tal forma que entendí. 15. El personal fue respetuoso de mis raíces culturales/étnicas. (Por ejemplo: mis valores, tradiciones, creencias, raza, lenguaje, etc.).             Como resultado de los servicios que mi niño y/o la familia recibieron: 16. Mi hijo maneja de forma más eficaz los problemas diarios.       17. Mi hijo se lleva mejor con los otros miembros de la familia.       18. Mi hijo se lleva mejor con los amigos y otras personas.       19. A mi hijo le va mejor en la escuela y/o el trabajo.       20. Mi hijo está más dispuesto a enfrentar las situaciones difíciles.       21. Estoy satisfecho con nuestra vida familiar actualmente.       22. Mi hijo es más capaz de hacer las cosas que quiere hacer.       Por favor ayude a nuestra agencia a mejorar los servicios, contestando algunas preguntas sobre los servicios que su niño recibido DURANTE LOS ÚLTIMOS 6 MESES. Como resultado de los servicios que mi niño y/o la familia recibieron: por favor responda basado en las relaciones con personas diferentes de su proveedor(es) de salud mental. 23. Conozco gente que escuchará y me entenderá cuando necesito      hablar. 24. Tengo personas con las que me siento cómodo hablando acerca del      problema de mi hijo.      25. En una crisis, yo tendría el apoyo que necesito de mi familia o amigos.      26. Cuento con personas con quienes puedo hacer cosas agradables. 27. ¿Cuál ha sido la cosa más provechosa sobre los servicios usted y su hijo recibieron durante los 6 meses pasados? 28. ¿Qué mejoraría los servicios aquí?     Please feel free to use the space provided below to comment on any of your answers. Also, if there are areas which were not covered by this questionnaire which you feel should have been, please write them in the comments section. Thank you for your time and cooperation in completing this questionnaire. Please list any other comments you may want to share: ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Thank you for completing this questionnaire. ¡¡ESTA SECCIÓN DEBE SER COMPLETADA POR LA RBHA O EL PROVEEDOR!! Nombre de Agencia de Servicio __________________________________________ NOMBRE de la RBHA: __________________ El Estado de Elegibilidad del Cliente: Encuesta tomada en : Casa  ID de agencia o proveedor: _____________ TXIX  TXXI  Clínica  Por teléfono  ID de la encuesta: ___________________ Cliente inscrito en DDD: Si____ No_____ ENCUESTA PARA FAMILIAS DE JÓVENES QUE RECIBIERON SERVICIOS DURANTE EL AÑO 2014 Por favor ayúdenos a mejorar nuestro programa completando ésta encuesta sobre los servicios que usted ha recibido en los últimos seis meses. Estamos interesados en su opinión honesta. Todas las respuestas serán tratadas como confidenciales. Cualquier información personal será excluida en la presentación de los resultados de ésta encuesta. Los servicios actuales y/o futuros de su hijo(a) o la familia no se verán afectados si usted decide no participar en esta encuesta. Si usted ha tomado ya esta encuesta en 2014, no tiene que completarla otra vez. Después de que usted haya finalizado, por favor doble la hoja y deposítela en la caja de encuestas, antes de usted salga de la oficina hoy. Gracias. Use pluma o lápiz Por favor rellene las burbujas completamente con sus respuestas a las preguntas siguientes: Información acerca de la persona que recibe los servicios: La Edad del Niño: El Sexo del Niño:  Masculino La Pertenencia étnica del Niño  Femenino  Hispano o Latino  No Hispano o Latino Por favor compruebe todas las categorías de raza aplicables: La Raza del Niño:  Blanco  Negro/Africano Americano  Asiático  Nativo Americano / Nativo de Alaska  Nativo de Hawai/Isleño del Pacífico ¿Por cuánto tiempo ha estado su hijo recibiendo servicios de salud mental y/o de abuso de sustancias? (De éste proveedor)  0 - 6 meses  7 - 11 meses  1 - 2 años  3 - 5 años  más de 5 años Por favor indique su relación con el niño  Padre/Guardián  Amigo  Pariente (Tío, Tía, Primo, Abuelo, etc.)  Otro ¿Tiene su familia un Equipo del Niño y la Familia?  Sí  No Por favor siga contestando preguntas en la siguiente página.