2011 Annual Consumer Survey Report December 2011 Vanessa Cardenas, Data Specialist Carolyn Dempsey, Performance Improvement Specialist Julie Karcis, Consultant Arizona Department of Health Services Division of Behavioral Health Services Bureau of Quality Management Operations 2011 Consumer Survey Report TABLE OF CONTENTS EXECUTIVE SUMMARY……………………………………… .......................................................... 3 INTRODUCTION……………………………………………………………... ........................... 5 SURVEY RESPONSE RATES .…………………………………………………………… ........ 5 METHODOLOGY AND SURVEY ADMINISTRATION……………………………………………………………………………...6 DATA ANALYSIS………………….………………………………………………………… .... 7 STATEWIDE SURVEY DATA LIMITATIONS………………………………………………...7 SURVEY FINDINGS................................................................ ..................................................... 8 STATEWIDE IMPROVEMENT OPPORTUNITIES……………………………………………… ............................................ …18 ADHS/DBHS INITIATIVES…………………………………………………….……...............18 CONCLUSION…………………………………………………………………………………..21 ATTACHMENTS Attachment A: 2011 Tribal Behavioral Health Authority Health Survey Results Attachment B: 2011 Statewide Consumer Survey Protocol Attachment C: 2011 Raw Data Attachment D: 2011 Adult Demographics Attachment E: 2011 Adult Line Item Attachment F: 2011 YSS-F Demographics Attachment G: 2011 YSS-F Line Item Attachment H: List of Acronyms APPENDICES YEAR 2011 ADULT CONSUMER SURVEY (ENGLISH AND SPANISH) YEAR 2011 YOUTH CONSUMER SURVEY FOR FAMILIES (ENGLISH AND SPANISH) 2011 Consumer Survey Report I. Executive Summary The Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS), Tribal/Regional Behavioral Health Authorities (T/RBHAs) and contracted service providers jointly conduct statewide consumer surveys during each fiscal year. Each year, two surveys are administered based on the Substance Abuse and Mental Health Services Administration‟s (SAMSHA‟s) Mental Health Statistics Improvement Program (MHSIP) consumer surveys: The Adult Consumer Survey and The Youth Services Survey for Families (YSS-F). The surveys request independent feedback from Title XIX/XXI adults and families of youth receiving services through Arizona‟s publicly funded behavioral health system. The surveys measure consumers‟ perceptions of behavioral health services in relation to the following domains: General Satisfaction Access to Services Service Quality/Appropriateness Participation in Treatment Outcomes Cultural Sensitivity Improved Functioning Social Connectedness ADHS/DBHS Consumer Survey data results are routinely: Reviewed in the ADHS/DBHS Quality Management (QM) Committee meetings; Used as the basis for recommendations from the QM Committee, which are incorporated in RBHA contracts and QM Plans; Disseminated to ADHS/DBHS stakeholders and consumers via the ADHS/DBHS website. Survey outcomes are always reported to: The National Association of State Mental Health Program Directors‟ (NASMHPD) National Research Institute (NRI) Western States Decision Support Group (WSDSG) Substance Abuse and Mental Health Services Administration‟s (SAMHSA) Mental Health Statistics Improvement Program (MHSIP) The fiscal year 2011 survey was conducted from April 4 through May 27, 2011. For RBHA contractors, ADHS/DBHS identified the following findings for the Adult Consumer Survey results: At a statewide level, one domain (Service Access) improved by 3 percent, while five (Participation in Treatment Planning, Service Quality and Appropriateness, Outcomes, Improved Functioning, Social Connectedness) decreased from 2 to 5 percent. None of these statewide increases or decreases were statistically significant. 3 2011 Consumer Survey Report Magellan had statistically significant decreases from the 2010 survey of 6 to 10 percent in three domains (Participation in Treatment Planning, Service Quality and Appropriateness, Outcomes). For the YSS-F Survey ADHS/DBHS identified the following findings for RBHA contractors overall: At a statewide level, three domains had increases in scores (Service Access, Participation in Treatment Planning, Social Connectedness) by 1 to 3 percent. Statewide, three domains (General Satisfaction, Outcomes, Improved Functioning) decreased from the 2010 results by 3 to 4 percent. None of these increases or decreases were statistically significant. Cenpatico-2 had statistically significant increases from the 2010 survey in two domains (General Satisfaction, Social Connectedness) of 8 and 9 percent. Cenpatico-4 had a statistically significant increase from the 2010 survey in one domain (Service Access) by 7 percent. Magellan had statistically significant decreases from the 2010 survey in three domains (General Satisfaction, Outcomes, Improved Functioning) of 8 and 10 percent. Regional Behavioral Health Authorities (RBHA) and Tribal Behavioral Health Authorities (TBHA) results from the 2011 Consumer Survey are presented separately to preserve the integrity of findings as two different sampling methodologies were used to get the survey sample. (For details, please refer to Section III-Survey Response Rates.) 4 2011 Consumer Survey Report II. Introduction The Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS) and the Regional Behavioral Health Authorities (RBHAs) in collaboration with their contracted providers administered the statewide consumer survey during April and May of 2011. Survey data results are: Reviewed in the ADHS/DBHS Quality Management (QM) Committee meetings; Used as the basis for recommendations from the QM Committee, which are incorporated in RBHA contracts and QM Plans; Disseminated to ADHS/DBHS stakeholders and consumers via the ADHS/DBHS website. Survey outcomes are reported to: The National Association of State Mental Health Program Directors‟ (NASMHPD) National Research Institute (NRI) Western States Decision Support Group (WSDSG) Substance Abuse and Mental Health Services Administration‟s (SAMHSA) Mental Health Statistics Improvement Program (MHSIP) III. Survey Response Rates A total of 5,800 RBHA-enrolled consumers were included for the MHSIP Consumer Survey random sample pool; 3,473 consumers were contacted and asked to complete a survey. The response rate is calculated by dividing the number of surveys returned by the number of enrollees asked to participate. Statewide response rates based on surveys offered were 85% for adults and 86% for youth, as seen in TABLE-1. Please see Attachment B for a complete discussion of the survey protocol. The Tribal Behavioral Health Authorities (TBHA) scores are not included in the statewide results due to the TBHA convenience sampling methodology which solicits responses at the convenience of the recipients rather than from a random sample of TBHA enrolled recipients. Therefore the results are reported separately in Attachment A-2011 Tribal Behavioral Health Authority Health Survey Results. TABLE-1: Survey Response Rate Based on Surveys Offered for Each RBHA/GSA Consumer Survey 2011 Surveys Offered (a) Surveys Completed (b) Response Rate (b/a) RBHA/GSA Overall Adult Youth Overall Adult Youth Overall Adult Youth CBHS 2 CBHS 3 CBHS 4 CPSA 5 Magellan NARBHA Statewide RBHA 5 480 328 489 243 553 269 675 374 519 262 757 370 3,473 1,846 152 246 284 301 257 387 1,627 386 262 398 201 427 194 645 352 420 224 673 340 2,949 1,573 124 197 233 293 196 333 1,376 81% 82% 77% 95% 83% 90% 86% 80% 83% 72% 96% 84% 92% 85% 84% 82% 82% 94% 82% 87% 86% 2011 Consumer Survey Report IV. Methodology and Survey Administration Two survey populations (sampling frame) were identified leading to the use of the Adult Consumer Survey and the YSS-F: Adults - Title XIX/XXI behavioral health recipients who are 18 years of age or older, and are enrolled in any of the adult programs: Serious Mental Illness (SMI), Substance Abuse or General Mental Health (Non-SMI). Youth - Title XIX/XXI behavioral health recipients under age 18 and enrolled in the Child/Adolescent program. A total of 74,312 adult and 35,853 youth RBHA-enrolled Title XIX/XXI consumers were eligible to participate in the 2011 survey. Please refer to Attachment B (2011 Consumer Survey Protocol) for details on sampling frame development, inclusion/exclusion criteria, survey instruments, and survey administration guidelines. The ADHS/DBHS Consumer Survey utilized a Likert Scale model to have respondents rate the domain questions from strongly agree (numeric value of 1) to strongly disagree (numeric value of 5), and an option for Not Applicable (numeric value 0). Consumer perception was determined as positive for those domains where the question responses averaged 2.5 or below. TABLE-2 and TABLE-3 show the number and percentage of sample surveys that were not included in the domain score, presented by RBHA/GSA and domain. Surveys are excluded from a domain‟s score when more than one-third of the questions have a response that cannot be determined. This occurs most often when no response is marked, and can also occur if more than one response is marked. See Attachment B for a full discussion of the survey protocol. TABLE-2: Adult Number and Percent of Surveys Excluded Due to Non-Responses by Domain and RBHA /GSA RBHA Cenpatico-2 Cenpatico-3 Cenpatico-4 CPSA-5 Magellan NARBHA Statewide RBHA 6 General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriaten ess Outcomes Improved Functioning Social Connectedness 3 (1.2%) 13 (6.5%) 7 (3.6%) 24 (6.8%) 27 (12.1%) 15 (4.4%) 0 1 (0.5%) 4 (2.1%) 7 (2.0%) 11 (4.9%) 3 (0.9%) 16 (6.1%) 16 (8.0%) 10 (5.2%) 25 (7.1%) 29 (12.9%) 8 (2.4%) 2 (0.8%) 11 (5.5%) 4 (2.1%) 12 (3.4%) 15 (6.7%) 7 (2.1%) 10 (3.8%) 15 (7.5%) 4 (2.1%) 22 (6.3%) 24 (10.7%) 13 (3.8%) 5 (1.9%) 12 (2.6%) 5 (2.6%) 23 (6.5%) 22 (9.8%) 9 (2.6%) 5 (1.9%) 8 (4.0%) 6 (3.1%) 12 (3.4%) 17 (7.6%) 7 (2.1%) 89 (5.7%) 26 (1.7%) 104 (6.6%) 88 (5.6%) 76 (4.8%) 55 (3.5%) 51 (3.2%) 2011 Consumer Survey Report TABLE-3: Youth Number and Percent of Surveys Excluded Due to Non-Responses by Domain and RBHA/GSA RBHA Cenpatico-2 Cenpatico-3 Cenpatico-4 CPSA-5 Magellan NARBHA Statewide RBHA General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 0 3 (1.5%) 0 5 (1.7%) 15 (7.7%) 4 (1.2%) 0 4 (2.0%) 3 (1.3%) 10 (3.4%) 16 (8.2%) 3 (0.9%) 0 3 (1.5%) 0 10 (3.4%) 16 (8.2%) 6 (1.8%) 1 (0.8%) 10 (5.1%) 4 (1.7%) 23 (7.8%) 31 (15.8%) 17 (5.1%) 2 (1.6%) 3 (1.5%) 2 (0.9%) 13 (4.4%) 20 (10.2%) 8 (2.4%) 2 (1.6%) 2 (1.0%) 1 (0.4%) 13 (4.4%) 20 (10.2%) 8 (2.4%) 5 (4.0%) 6 (3.0%) 1 (0.4%) 15 (5.1%) 23 (11.7%) 7 (2.1%) 67 (4.9%) 77 (5.6%) 77 (5.6%) 157 (11.4%) 98 (7.1%) 97 (7.0%) 112 (8.1%) V. Data Analysis ADHS/DBHS provided each RBHA with an Access database to compile survey responses. All completed surveys were entered and the database was submitted to ADHS/DBHS. Each RBHA analyzed its respective survey data using an SPSS script designed and provided by ADHS/DBHS to ensure consistency in data analysis. Statewide survey data is analyzed as follows: By Domain; By Domain Line Item; Sub-group Analysis; and Comparison to Past Survey Performance. The statewide survey results are weighted to correct the sample design, taking the different population sizes for each Geographic Service Area (GSA) into consideration. The weighted averages were taken to avoid the over/under representation of highly/less populated GSAs. Please refer to Attachment C, 2011 Raw Data, for further specifics of the weighting methodology and weights applied to each GSA. VI. Statewide Survey Data Limitations Two different sampling methodologies were used to get the 2011 Consumer Survey sample. The Tribal Behavioral Health Authorities (TBHA) scores could not be included in the statewide results due to the TBHA convenience sampling methodology which solicits responses at the convenience of the recipients and providers rather than from a random sample of TBHA enrolled recipients. 7 2011 Consumer Survey Report VII. Survey Findings MHSIP ADULT Demographics A total of 1,573 completed adult surveys were analyzed. The breakout of the respondents is by gender, age, ethnicity, race, length of enrollment in services, length of enrollment and behavioral health category. Please see Attachment D, 2011 Adult Demographics for complete demographics information. Domain Line Item Analysis A specific set of questions was developed to gather scores for each domain. Line items are specific questions pertaining to each survey domain (Attachment E, 2011 Adult Line Item); analysis of the respondents‟ answers to a question under a domain, either positive or negative, affects the overall domain score. Table-4 demonstrates the results of the 2011 Adult Survey and compares them to previous surveys. A P-value of 95% significance when less than or equal to 0.05 or “ns” for “not significant” is listed to reflect the change from the prior year. The statewide statistics are based on weighted scores. TABLE-4 Statewide RBHA Adult Consumer Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness Color Key statistically significant increase statistically significant decrease 8 2007 % 86 77 79 88 74 66 65 2008 % (P-value) 82 (ns) 73 (ns) 89 (0.01) 84 (ns) 68 (ns) 68 (ns) 68 (ns) 2009 % (P-value) 84 (ns) 78 (ns) 82 (0.03) 89 (ns) 70 (ns) 68 (ns) 68 (ns) 2010 % (P-value) 86 (ns) 81 (ns) 92 (0.01) 90 (ns) 75 (ns) 71 (ns) 79 (0.02) 2011 % (P-value) 86 (ns) 84 (ns) 87 (ns) 88 (ns) 72 (ns) 69 (ns) 76 (ns) 2011 Consumer Survey Report The 2011 statewide adult survey findings identify: General Satisfaction: Three questions were asked in the survey to ascertain General Satisfaction (Attachment E). The overall score for this domain was 86%, representing an average score based on three (3) questions. No increases or decreases were found in the 2011 survey results for this domain. Service Access: Six questions in the survey gathered information regarding Access to Services (Attachment E). This domain scored at 84%, based on an average score of six (6) questions, representing a three (3) percent increase compared to 2010, however the increase was not statistically significant Participation in Treatment Planning: The survey comprised two (2) questions to indicate Participation in Treatment Planning (Attachment E). The overall score for this domain was 87% which is an average score based on two (2) questions. This domain had a five (5) percent decrease compared to 2010. However the decrease was not statistically significant. Service Quality and Appropriateness: Nine questions in the survey focused on Service Quality and Appropriateness (Attachment E). The overall score for this domain was 88%, representing an average score based on nine (9) questions. This domain had a two (2) percent decrease compared to 2010. However the decrease was not statistically significant. Outcomes: Eight questions were asked to get consumers‟ perception on Outcomes (Attachment E). The overall score for this domain was 72%. This indicates a three (3) percent decrease compared to 2010, however the decrease was not statistically significant. Improved Functioning: Five questions indicate respondent‟s impressions of Improved Functioning (Attachment E). The overall score for this domain was 69%. This indicates a two (2) percent decrease compared to 2010, however the decrease was not statistically significant. Social Connectedness: Four questions were asked in the survey to ascertain Social Connectedness (Attachment E). The score for this domain was 76%. This indicates a three (3) percent decrease compared to 2010, however the decrease was not statistically significant. In summary, score for the Service Access domain increased. The Improved Functioning domain remains the lowest and the Quality & Appropriateness domain scored the highest. Although these domains showed differences as compared to last year‟s survey, none of the differences is statistically significant. 9 2011 Consumer Survey Report While there were no significant increases or decreases in the statewide survey results from 2010 to 2011, at the RBHA/GSA level, ADHS/DBHS identified opportunities for improvement. TABLE-5 through TABLE-10 show positive responses for each RBHA/GSA by domain. Included are the years 2007 through 2011. TABLE-5: Cenpatico-2 Adult Consumer Survey Results by Domain for 2007 - 2011 2007 % Domain General Satisfaction 2008 % (P-value) 2009 % (P-value) 2010 % (P-value) 2011 % (P-value) 87 84 (ns) 86 (ns) 91 (ns) 89 (ns) Service Access 80 82 (ns) 81 (ns) 90 (0.02) 88 (ns) Participation in Treatment Planning 79 86 (ns) 82 (ns) 93 (0.00) 91 (ns) Service Quality & Appropriateness 87 88 (ns) 89 (ns) 92 (ns) 90 (ns) Outcomes 78 71 (ns) 71 (ns) 81 (0.03) 82 (ns) Improved Functioning 73 75 (ns) 67 (ns) 75 (ns) 77 (ns) Social Connectedness 74 75 (ns) 71 (ns) 83 (0.01) 83 (ns) TABLE-6: Cenpatico-3 Adult Consumer Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness 2007 % 90 83 82 89 79 72 71 2008 % (P-value) 82 (0.01) 77 (ns) 87 (ns) 86 (ns) 68 (0.01) 73 (ns) 72 (ns) 2009 % (P-value) 83 (ns) 79 (ns) 79 (0.02) 88 (ns) 71 (ns) 71 (ns) 71 (ns) 2010 % (P-value) 86 (ns) 84 (ns) 88 (0.03) 88 (ns) 77 (ns) 72 (ns) 80 (0.05) 2011 % (P-value) 86 (ns) 86 (ns) 91 (ns) 88 (ns) 75 (ns) 73 (ns) 81 (ns) TABLE-7: Cenpatico-4 Adult Consumer Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness Color Key statistically significant increase statistically significant decrease 10 2007 % 87 86 81 91 78 76 73 2008 % (P-value) 87 (ns) 83 (ns) 92 (0.01) 90 (ns) 73 (ns) 67 (0.04) 67 (ns) 2009 % (P-value) 86 (ns) 83 (ns) 84 (0.00) 89 (ns) 70 (ns) 64 (ns) 66 (ns) 2010 % (P-value) 87 (ns) 87 (ns) 92 (0.03) 90 (ns) 75 (ns) 73 (ns) 77 (0.02) 2011 % (P-value) 91 (ns) 89 (ns) 95 (ns) 93 (ns) 76 (ns) 70 (ns) 79 (ns) 2011 Consumer Survey Report TABLE-8: CPSA-5 Adult Consumer Survey Results by Domain for 2007 – 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness 2007 % 85 78 78 89 70 64 64 2008 % (P-value) 83 (ns) 67 (0.01) 91 (0.00) 80 (0.00) 66 64 62 (ns) (ns) (ns) 2009 % (P-value) 78 (ns) 69 (ns) 78 (0.00) 87 (ns) 68 65 65 (ns) (ns) (ns) 2010 % (P-value) 87 (0.03) 79 (0.03) 92 (0.00) 90 (ns) 69 68 78 2011 % (P-value) 89 (ns) 85 (ns) 92 (ns) 91 (ns) (ns) (ns) (0.01) 75 72 76 (ns) (ns) (ns) TABLE-9: Adult Consumer Survey Results by Domain for Value Options in 2007 and Magellan in 2008-2011 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness 2007 % 87 83 83 90 77 67 67 2008 % (P-value) 80 (0.04) 71 (0.00) 87 (ns) 83 (0.02) 68 (0.03) 68 (ns) 69 (ns) 2009 % (P-value) 86 (ns) 82 (0.02) 85 (ns) 90 (ns) 73 (ns) 71 (ns) 69 (ns) 2010 % (P-value) 85 (ns) 79 (ns) 92 (0.05) 90 (ns) 77 (ns) 72 (ns) 79 (0.03) 2011 % (P-value 83 (ns) 83 (ns) 82 (0.00) 84 (0.05) 68 (0.03) 65 (ns) 74 (ns) TABLE-10: NARBHA Adult Consumer Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness Color Key statistically significant increase statistically significant decrease 11 2007 % 84 73 78 87 74 63 61 2008 % (P-value) 85 (ns) 80 (ns) 92 (0.00) 88 (ns) 71 (ns) 68 (ns) 65 (ns) 2009 % (P-value) 84 (ns) 79 (ns) 80 (0.00) 88 (ns) 64 (ns) 63 (ns) 68 (ns) 2010 % (P-value) 88 (ns) 83 (ns) 92 (0.00) 90 (ns) 72 (ns) 71 (ns) 78 (0.03) 2011 % (P-value) 90 (ns) 85 (ns) 91 (ns) 93 (ns) 76 (ns) 74 (ns) 80 (ns) 2011 Consumer Survey Report Of the six RBHA/GSAs, one contractor, Magellan experienced statistically significant changes from the 2010 to 2011 survey. Specifically, there were significant decreases in three domains; Participation in Treatment Planning fell 10% (P=0.00), Service Quality & Appropriateness fell 6% (P=0.05), and Outcomes fell 9% (P=0.03) compared to 2010. Magellan has recognized the statistically significant decrease in the Outcomes domain and reported steps they are taking to address this area. Magellan‟s goal will be to implement a continuous quality improvement approach that utilizes customer input to plan, design, measure, assess and improve behavioral health services. Listed below are the steps Magellan will implement for the adult Outcomes domain population. These steps are also expected to improve Social Connectedness and Improved Functioning domains.      Magellan will share the 2010 and 2011 Adult Satisfaction results with Provider Network Organizations (PNO) leadership and utilize the PNO Clinical Workgroup to identify root causes of poor performance and develop key initiatives to address these deficiencies. Develop process to administer Adult Satisfaction Survey to all PNO clinics to identify baseline performance measures by clinic and PNO on a quarterly basis. Present most recent survey outcomes to each PNO clinics‟ Clinic Advisory Committee to assist with the development of key clinic specific initiatives to improve poor performance. Require PNOs to develop and submit Performance Improvement Plans for areas of poor performance that clearly delineate clinic processes that will improve outcomes. Magellan will review quarterly Adult Satisfaction Survey results with PNO executive leadership. ADHS/DBHS recognizes Magellan‟s initiatives/action items do not address two domains that indicated a statistically significant decrease, Service Quality & Appropriateness domain and Participation in Treatment Planning domain. Magellan has been notified to submit an initiatives/action items report to ADHS/DBHS in a timely manner. 12 2011 Consumer Survey Report MHSIP YSS-F Demographics A total of 1,376 completed youth surveys were analyzed. The breakout of the respondents is reported by gender, age group, ethnicity, race, length of enrollment in services, and involvement in a Child and Family Team (CFT). Please see Attachment F, 2011 YSS-F Demographics for complete demographics information. Domain Line Item Analysis Line items are specific questions pertaining to each survey domain. Analysis of answers to each domain specific line item indicates the aspects of service delivery the respondents report as either positively or negatively affecting the overall domain score (Attachment G, 2011 YSS-F Line Item). Table-11 demonstrates the results of the 2011 YSS-F Survey and compares them to previous surveys. A P-value of 95% significance when less than or equal to 0.05 or “ns” for “not significant” is listed to reflect the change from the prior year. The statewide statistics are based on weighted scores. TABLE-11: Statewide Youth Consumer YSS-F Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness Color Key statistically significant increase statistically significant decrease 13 2007 % 78 78 90 92 58 61 82 2008 % (P-value) 77 (ns) 73 (ns) 87 90 60 58 79 (ns) (ns) (ns) (ns) (ns) 2009 % (P-value) 80 (ns) 76 (ns) 89 90 68 66 83 (ns) (ns) (ns) (ns) (ns) 2010 % (P-value) 85 (ns) 83 (ns) 92 96 69 71 88 (ns) (0.05) (ns) (ns) (ns) 2011 % (P-value) 82 86 (ns) (ns) 94 96 65 67 89 (ns) (ns) (ns) (ns) (ns) 2011 Consumer Survey Report The 2011 statewide YSS-F survey findings indicate: General Satisfaction: Six questions were asked in the survey to ascertain General Satisfaction (Attachment G). This domain scored 82%, a decrease by three (3) percent compared to 2010. However this decrease is not statistically significant. Service Access: Two questions in the survey gathered information regarding Access to Services (Attachment G). This domain scored 86%, an increase by three (3) percent, representing an average score based on 2 questions. However, this increase is not statistically significant compared to 2010. Participation in Treatment Planning: The survey comprised three questions to indicate Participation in Treatment Planning (see Attachment G). The score for this domain scored 94% up two (2) percent from 2010, representing an average score. This increase is not statistically significant. Cultural Sensitivity: Four questions in the survey focused on Cultural Sensitivity (Attachment G). This domain scored 96%, representing an average score. No increases or decreases were found in the 2011 survey results for this domain. Outcomes: Eight questions were asked to get consumers‟ perception on Outcomes (Attachment G). The overall score for this domain was 65%; indicating a four (4) percent decrease from 2010, however this decrease was not statistically significant. Improved Functioning: Six questions indicate respondent‟s impressions of Improved Functioning (Attachment G). The overall score for this domain was 67%. This domain decreased by four (4) percent from 2010; however this decrease was not statistically significant. Social Connectedness: Four questions were asked in the survey to ascertain Social Connectedness (see Attachment G). This domain scored 89%. Representing an average score. The increase by one (1) percent compared to 2010 is not statistically significant. . In summary, scores for the Service Access and Participation in Treatment Planning domains increased. The Outcomes domain remains the lowest and the Cultural Sensitivity domain remains the highest. Although these domains showed differences as compared to last year‟s survey, none of the differences is statistically significant. 14 2011 Consumer Survey Report While there were no significant increases or decreases in the statewide survey results from 2010 to 2011, at the RBHA/GSA level, ADHS/DBHS identified opportunities for improvement and recognized areas of improvement. TABLE-12 through TABLE-17 show positive responses for each RBHA/GSA by domain. Included are the years 2007 through 2011. TABLE-12: Cenpatico-2 Youth Consumer YSS-F Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 2007 % 89 86 88 96 76 78 89 2008 % (P-value) 76 (0.00) 81 (ns) 86 (ns) 94 (ns) 70 (ns) 69 (0.03) 86 (ns) 2009 % (P-value) 81 (ns) 82 (ns) 94 (0.02) 93 (ns) 62 (ns) 61 (ns) 85 (ns) 2010 % (P-value) 82 (ns) 90 (0.04) 93 (ns) 99 (0.02) 69 (ns) 70 (ns) 89 (ns) 2011 % (P-value) 91 (0.02) 95 (ns) 96 (ns) 99 (ns) 71 (ns) 73 (ns) 97 (0.01) TABLE-13: Cenpatico-3 Youth Consumer YSS-F Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 2007 % 77 74 88 90 65 65 77 2008 % (P-value) 74 (ns) 74 (ns) 80 89 65 62 77 (0.01) (ns) (ns) (ns) (ns) 2009 % (P-value) 75 (ns) 76 (ns) 88 89 66 65 78 (0.05) (ns) (ns) (ns) (ns) 2010 % (P-value) 87 (0.01) 90 (0.00) 90 94 77 79 87 (ns) (ns) (0.02) (0.00) (0.03) 2011 % (P-value) 86 90 (ns) (ns) 89 96 73 74 87 (ns) (ns) (ns) (ns) (ns) TABLE-14: Cenpatico-4 Youth Consumer YSS-F Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness Color Key statistically significant increase statistically significant decrease 15 2007 % 78 86 92 91 66 68 82 2008 % (P-value) 78 (ns) 80 (ns) 90 (ns) 92 (ns) 61 (ns) 62 (ns) 87 (ns) 2009 % (P-value) 85 (ns) 91 (0.01) 98 (0.01) 96 (ns) 66 (ns) 65 (ns) 87 (ns) 2010 % (P-value) 80 (ns) 87 (ns) 91 (0.00) 95 (ns) 66 (ns) 69 (ns) 88 (ns) 2011 % (P-value) 84 (ns) 94 (0.04) 94 (ns) 97 (ns) 70 (ns) 71 (ns) 91 (ns) 2011 Consumer Survey Report TABLE-15: CPSA-5 Youth Consumer YSS-F Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 2007 % 81 76 89 90 65 66 86 2008 % (P-value) 79 (ns) 73 (ns) 86 90 63 62 82 (ns) (ns) (ns) (ns) (ns) 2009 % (P-value) 82 (ns) 74 (ns) 86 86 72 72 84 (ns) (ns) (ns) (0.04) (ns) 2010 % (P-value) 81 (ns) 84 (0.02) 88 94 62 66 84 (ns) (0.02) (0.03) (ns) (ns) 2011 % (P-value) 88 87 (ns) (ns) 93 97 68 71 90 (ns) (ns) (ns) (ns) (ns) TABLE-16: Youth Consumer YSS-F Survey Results by Domain for Value Options in 2007 and Magellan in 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 2007 % 77 75 86 93 62 63 79 % 75 70 87 90 56 53 75 2008 (P-value) (ns) (ns) (ns) (ns) (ns) (0.04) (ns) % 77 73 88 91 68 64 82 2009 (P-value) (ns) (ns) (ns) (ns) (0.02) (0.03) (ns) % 86 80 93 98 71 72 88 2010 (P-value) (0.03) (ns) (ns) (0.01) (ns) (ns) (ns) 2011 % (P-value) 78 (0.02) 82 (ns) 94 (ns) 96 (ns) 61 (0.03) 63 (0.05) 88 (ns) TABLE-17: NARBHA Youth Consumer YSS-F Survey Results by Domain for 2007 - 2011 Domain General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness Color Key statistically significant increase statistically significant decrease 16 2007 % 76 77 91 93 51 55 80 % 84 81 88 93 72 71 87 2008 (P-value) (ns) (ns) (ns) (ns) (0.00) (0.00) (ns) 2009 % (P-value) 83 (ns) 79 (ns) 92 (ns) 93 (ns) 67 (ns) 66 (ns) 87 (ns) % 90 90 93 96 70 73 91 2010 (P-value) (ns) (0.01) (ns) (ns) (ns) (ns) (ns) 2011 % (P-value) 90 94 94 97 69 71 90 (ns) (ns) (ns) (ns) (ns) (ns) (ns) 2011 Consumer Survey Report Cenpatico-2 experienced a statistically significant increase in two domains. General Satisfaction domain increased 9% (P=0.01) and Social Connectedness domain increased 8% (P=0.03) compared to 2010. Cenpatico-4 experienced a statistically significant increase in one domain. Service Access domain increased 7% (P=0.04) compared to 2010. Magellan experienced a statistically significant decrease in three domains. General Satisfaction domain fell 8% (P=0.02), Outcomes domain fell 10% (P=0.03), and Improved Functioning domain fell 9% (P=0.05) compared to 2010. Magellan acknowledges the statistically significant decreases and has addressed implementation for the General Satisfaction and Outcomes domains. Listed below are steps Magellan is taking to improve consumers‟ satisfaction. These steps are also expected to improve the Improved Functioning domain.     17 Magellan is working with the children‟s Provider Network Organizations (PNO) to meet expectations for expansion of high needs case management capacity. High Needs Case Managers (HNCM) have a limited caseload size and work with families of children with complex needs, including multi-system involvement, to coordinate services and supports. The PNOs have in place screening processes and high needs indicators to guide Qualified Service Providers (QSP) in referring children with complex needs for HNCM assignment. The PNOs are moving toward conducting screening and assessment of children with high needs indicators at the PNO level in order to streamline the process of identifying and implementing appropriate services. Through the “Meet Me Where I Am” Design Team, Magellan is monitoring service utilization and average length of stay in generalist direct support services and encouraging the providers to periodically review the service array with child and family teams to ensure that services are producing desired outcomes and that transitions occur appropriately as treatment objectives are met and intensive services are no longer needed, thereby freeing up program capacity to other families who are in need of the services. The Children‟s Dashboard is measuring individual provider and children‟s Provider Network Organizations (PNO) performance on the National Outcomes Measures as well as the Functional Outcomes Measures. The Dashboard is available on the Magellan of Arizona website. The System of Care Practice Review (SOCPR) is now moving toward the close of its second year as the statewide mechanism for monitoring fidelity of practice to system of care principles. Following the first year of SOCPR, Magellan assisted children‟s PNOs and provider agencies in developing Performance Improvement Plans (PIP) to address opportunities for improvement identified in the SOCPR for children with high needs and the Brief Practice Review for all other children. Magellan QM/QI has conducted interim monitoring of provider PIPs through requesting progress updates from the providers and has supported the DBHS contractor in sharing the findings of the 2nd round of SOCPR and Brief Practice Reviews with providers and has assisted providers in developing further PIPs when warranted by practice review findings. 2011 Consumer Survey Report  The Responsible and Responsive Service Position Paper soon to be released by the Magellan Responsible and Responsive Service Initiative includes information promoting an outcomes orientation. In the position paper, which has contributions from the children‟s PNOs and which will be distributed to children‟s provider agencies, provider agencies are encouraged to put in place mechanisms for measuring outcomes at the individual and family level and at the organizational level to drive performance improvement activities. An Outcomes Workgroup of the initiative will continue to meet on a monthly basis to address ways to improve outcomes in the children‟s system. VIII. Statewide Improvement Opportunities ADHS/DBHS recognizes the opportunity to improve efforts supported by the 2011 survey results; and intends to pursue initiatives with the RBHAs to continue to improve scores for each domain. Special attention will be placed around the Outcomes, Improved Functioning, and Social Connectedness domains. In addition to general statewide initiatives, RBHA-specific activities are undertaken as part of standard ADHS/DBHS processes, as appropriate. Survey results are shared with the RBHAs, and they are required to respond with analysis of any problem areas, along with plans for improvement. ADHS/DBHS reviews their conclusions and plans, and monitors implementation and progress. ADHS/DBHS Initiatives: Below are ADHS/DBHS initiatives for both the Children and Adult systems of care that address Outcomes, Improved Functioning, Service Access and Social Connectedness: Adult Initiatives:  The 2012-2014 ADHS/DBHS Adult System of Care (ASOC)-Strategic Plan (SP) establishes goals and objectives for statewide network development priorities in part through a review of State and GSA network analysis, review of data sets, community input, and the identification of best practices efforts that are designed to assist individuals moving through recovery and increasing the quality of life outcomes. The analyses of the ASOC-strategic plan and the 2010 and 2011 Adult and Youth Consumer Survey identified similar priorities. For example: o An objective in the ASOC-SP is to increase the employment rate for persons with Serious Mental Illness (SMI), General Mental Health (GMH) and Substance Abuse (SA). The Outcomes domain of the Consumer Survey addresses a person‟s ability to: 1) control his/her life, 2) operate in school, and work environment, and 3) do better in social situations; by achieving the improvement in employment rate in ASOC-SP, ADHS/DBHS expects an increase in the Outcomes domain. o The ASOC-SP includes an objective to increase the percentage of peers and family members satisfied with their access to services which was identified as an area needing improvement in the Consumer Survey Report. Additionally, ASOC-SP includes strategies to conduct statewide discussion on the awareness and community needs around Trauma Informed Care. 18 2011 Consumer Survey Report  ADHS/DBHS is in the process of finalizing policies on clinical supervision and have developed a Clinical Supervision Review Tool for T/RBHAs to ensure sufficient clinical supervision for their behavioral health staff, at least four hours monthly. The aim is to improve delivery of services and quality of care which will ultimately affect outcomes and access to care. System Initiatives:  In October, 2010 ADHS/DBHS modified its enrollment process to automatically enroll Title XIX/XXI persons into its behavioral health programs. AHHCCS members no longer need to undertake an additional step to enroll for behavioral health services. As a result, this process reduced a potential barrier to services.  ADHS/DBHS, along with family member and recipient input, developed the Behavioral Health System Outcomes Framework and Dashboard as a means to focus departmental planning efforts by evaluating the effectiveness of the system using standardized measures, such as family and recipient identified outcomes, the National Outcomes Measures (NOMs) and ADHS/DBHS performance metrics.  ADHS/DBHS foresees that the development of the Outcomes Framework will impact the future Consumer Survey results due to increased transparency, provider education and accountability in service provision. The development of the Outcomes Framework is in direct response to the identified need for transparency, accountability, improved family, recipient and provider education and tracking of outcomes as a means to drive both individual recipient and program outcomes. Children’s Initiatives: 19  ADHS/DBHS continues to use the System of Care Practice Review (SOCPR) process developed by the University of South Florida to assess Child and Family Team (CFT) practice. These intensive case reviews include outcomes-focused questions to ensure CFTs are identifying and tracking outcomes as well as evaluating the effectiveness of provided services and improved functioning. The reviews are conducted at the individual provider level and are used to develop annual performance improvement plans. The plans are approved by DBHS and monitored by the RBHA with regular updates provided to DBHS.  For FY10, ADHS/DBHS earmarked approximately five million dollars of additional funding to the RBHAs to increase the number of case managers with reduced caseloads. During the course of FY10, there was an increase of approximately 104 case managers statewide serving children with complex needs. In total this focused effort has developed approximately 469 case managers statewide as of the end of the FY11 fiscal year. ADHS/DBHS continues to monitor the availability of case managers with reduced case loads for children with complex needs.  The continued emphasis on in-home direct support services, as established by the Meet Me Where I Am (MMWIA) Campaign, has been a valuable resource to children and families with the most complex needs. These services put direct support workers in 2011 Consumer Survey Report homes, schools and communities at the times and for as many hours as necessary to meet the family‟s needs, leading to improved functioning and better outcomes. 20  ADHS/DBHS continues to promote family and youth involvement in the Children‟s Behavioral Health System. In partnership with family-run organizations, efforts are focused on building and sustaining an infrastructure and agency culture to support and involve youth and family members at all levels of the system.  The annual Children‟s System of Care (CSOC) Strategic Plan (SP) provides a vehicle for the Division and RBHAs to address key goals around CFT Practice Model monitoring through the Practice Review process, and development of youth and family involvement and leadership, Generalist Direct Support services and High Needs Case Management services to work with children and families with the most complex needs.  The strategic activities prioritized in CSOC-SP continue to promote family driven care as well as youth and family involvement in local and statewide committees and system development activities. Focus is placed on specific populations such as children ages birth to 5, transition aged youth, and youth with substance use disorders in order to more clearly define their needs and help providers develop expertise in working with these youth. 2011 Consumer Survey Report IX. Conclusion For the 2011 Adult Survey statewide, ADHS/DBHS established no statistically significant increases or decreases compared to the 2010 Survey results. However, the Service Access domain continues to indicate improvement by three (3) percent. For the 2011 Adult Survey RBHA/GSA level, ADHS/DBHS found statistically significant decreases in three domains from Magellan. Scores for the Participation in Treatment Planning, Service Quality and Appropriateness, and Outcomes domains fell from 6 to 10 percent. For the YSS-F Survey, ADHS/DBHS established no statistically significant increases or decreases compared to the 2010 Survey results. However, three domains Service Access, Participation in Treatment Planning, and Social Connectedness continue to indicate improvement by 1 to 3 percent. For the YSS-F Survey RBHA/GSA level, ADHS/DBHS found statistically significant increases and decreases. Cenpatico-2 increased in two domains General Satisfaction and Social Connectedness from 8 to 9 percent. Cenpatico-22, increased in one domain Service Access by seven (7) percent. Magellan decreased in three domains General Satisfaction, Outcomes, and Improved Functioning, from 8 to 10 percent. ADHS/DBHS is working with Magellan on initiatives for these problem areas and will monitor the implementation. As per the ADHS/DBHS Contract with the RBHAs, Consumer Satisfaction with Service Outcomes on the Annual Consumer Survey is a performance incentive; a score of 70% or greater on the Outcomes domain of the Consumer Survey makes up 25% of the incentive award. Adult and Youth survey results are evaluated independently, each contributing to half of the award. ADHS/DBHS believes that assisting members in the identification and tracking of treatment outcomes is imperative to improving behavioral health recipients‟ overall perception of the quality of services they receive. 21 2011 Consumer Survey Report ATTACHMENT-A 22 2011 Consumer Survey Report 2011 Tribal Behavioral Health Authority Survey Results Arizona Tribal Behavioral Health Authorities (TBHAs) Gila River Indian Community, Pascua Yaqui Centered Spirit Program, Navajo Nation and White Mountain Apache Tribe participated in the 2011 Consumers Survey. The Tribal BHAs have shown an overall increase in participation from 381 participants in 2010 to 438 in 2011. All TBHAs conduct a convenience sampling of their enrolled members. Overall, the number of surveys completed by tribal participants increased this year, particularly by Navajo Nation and White Mountain Apache Tribe. Number of Surveys Completed TBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Statewide TBHA Adult Survey 26 132 100 30 288 YSS-F 28 45 57 20 150 Total 54 177 157 50 438 Methodology and Survey Administration A convenience sampling methodology was used (rather than a random sampling); therefore, all persons enrolled with the TBHAs were eligible for participation in this survey. Data Analysis TBHAs results were submitted directly to ADHS/DBHS and entered into the database; analyses were conducted by ADHS/DBHS staff. Summary of Results: General Satisfaction and Participation in Treatment Planning domains reported positive responses for adults receiving services through TBHAs compared to 2010 results. Participation in Treatment Planning domain showed the highest rate of satisfaction and improvement. Outcomes and Improved Functioning domains remain the lowest rates of satisfaction for adults receiving services. Families of children receiving services through the TBHAs reported satisfaction and improvements in Cultural Sensitivity and Social Connectedness domains compared to 2010. Cultural Sensitivity domain scored the highest rate of satisfaction for the families of youth. Outcomes and Improved Functioning domains remain the lowest rates of satisfaction reported. 23 2011 Consumer Survey Report Summary of 2010 & 2011 TBHA Results for Adults TBHA General Satisfaction Service Access Participation in Treatment Planning Service Quality & Appropriateness Outcomes Improved Functioning Social Connectedness 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 Gila River 93% 92% 90% 85% 92% 92% 90% 96% 77% 84% 74% 84% 88% 85% Navajo Nation 89% 91% 80% 80% 79% 89% 85% 88% 73% 82% 76% 83% 75% 86% Pascua Yaqui 95% 94% 94% 86% 89% 90% 94% 93% 87% 82% 88% 78% 91% 85% White Mtn Apache 91% 93% 95% 96% 86% 86% 91% 86% 86% 76% 83% 86% 100% 90% TBHA scores are not included in statewide numbers due to TBHA convenience sampling methodology. Summary of 2010 & 2011 TBHA Results for Youth TBHA General Satisfaction Service Access Participation in Treatment Planning Cultural Sensitivity Outcomes Improved Functioning Social Connectedness 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 2010 2011 Gila River 91% 82% 91% 86% 97% 96% 97% 100% 57% 64% 57% 64% 94% 96% Navajo Nation 86% 84% 75% 72% 78% 87% 78% 91% 75% 73% 75% 75% 72% 91% Pascua Yaqui 84% 96% 100% 100% 84% 95% 98% 100% 58% 71% 65% 75% 93% 95% White Mtn Apache 89% 100% 100% 90% 89% 90% 100% 100% 67% 85% 67% 85% 78% 95% TBHA scores are not included in statewide numbers due to TBHA convenience sampling methodology. 24 2011 Consumer Survey Report ATTACHMENT-B 25 2011 Consumer Survey Report STATEWIDE CONSUMER SURVEY PROTOCOL 2011 Prepared by: Carolyn Dempsey, PI Specialist, Quality Management Operations, DBHS Daniel Crough, Acting Data Manager, DBHS Arizona Department of Health Services Division of Behavioral Health Services 26 2011 Consumer Survey Report Executive Summary The protocol for the 2011 consumer survey is based on the methodological and administrative guidelines followed in recent years. Two consumer surveys will be administered again in 2011; one for adults and one for the parents/guardians of children receiving behavioral health services. The survey will be available to consumers in two languages: English and Spanish. Survey data will be entered into an SPSS database by the Regional Behavioral Health Authorities (RBHAs) and forwarded to the Arizona Department of Health Services/Division of Behavioral Health Services (ADHS/DBHS). Prior to implementation of the 2011 survey, ADHS/DBHS will provide each Geographic Service Area (GSA) with the CIS (Client Information System) Client IDs for 500 adult consumers and 400 to 500 youth consumers who the RBHAs/providers will encourage to participate in the survey. Each CIS Client ID will be linked to a Survey ID reference number to allow for tracking of which consumers completed a survey. The survey will be administered during April and May 2011. As in previous years, the primary administration route will be to distribute the survey at the provider sites. As the consumer checks in for their appointment, the survey questionnaire will be handed to them. The consumer will be requested to complete the questionnaire prior to their appointment, but will be allowed to finalize the survey after the appointment if needed, or be provided with an addressed, stamped envelope to mail the survey in to the RBHA if they did not have time to complete it in the office. A drop box will be provided at each provider site for consumers to drop off their completed surveys. In cases where consumers who are selected for the sample have home appointments (as opposed to clinic appointments) during the survey timeframe, the questionnaire will be completed at home and mailed using the addressed, stamped envelope to be provided with the survey. If the consumer does not have a clinic or home appointment scheduled during the survey administration period or no-shows for a scheduled appointment, a non-clinical staff at the RBHA may conduct the survey by phone. Providers will never have access to completed surveys or individual survey results. ADHS has the statewide oversight responsibility for implementation and analysis of the survey data. The RBHAs will be responsible for ensuring that providers strictly adhere to the protocol. The providers are primarily responsible for the survey administration. Survey Instruments Two MHSIP survey instruments are the Adult Consumer Survey and the Youth Services Survey for Families (YSS-F). The adult survey will be administered to adult consumers of behavioral health services and the YSS-F will be administered to parents/guardians of children receiving behavioral health services. The MHSIP Adult Consumer Survey measures seven domains: (1) Service Accessibility; (2) Service Quality or Appropriateness (which includes one item concerning cultural sensitivity); (3) Consumer Participation in Treatment Planning; (4) Outcomes; (5) General Satisfaction; (6) Improved Functioning; and (7) Social Connectedness. In addition, the questionnaire includes a module of questions to determine the impact of services received on the recipient‟s involvement with the criminal justice system. All questions are scored using a Likert Scale of 1 through 5 as follows: 1=Strongly Agree, 2=Agree, 3=Neutral, 4=Disagree, and 5=Strongly Disagree. A Not Applicable option is also available if the question does not apply. 27 2011 Consumer Survey Report Two state-added questions were added to the 2011 Adult survey tool to gather information specific to National CLAS Standards (Culturally and Linguistically Appropriate Services): Q38 – Services were provided in a language I understood. Q39 – In creating my service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) Additionally, the clarifiers at the end of one MHSIP question (Q18) and one state-added question on the Adult tool (Q37) were modified to provide additional clarification. The MHSIP YSS-F focuses on the following seven domain areas: (1) Service Accessibility; (2) Participation in Treatment Planning; (3) Cultural Sensitivity; (4) Satisfaction with Services; (5) Outcomes; (6) Social Connectedness; and (7) Improved Functioning. Additional questions solicit information about the youth‟s criminal justice contact and school attendance. Questions are scored with a five-point Likert Scale where 1=Strongly Agree, 2=Agree, 3=Neutral, 4=Disagree, and 5=Strongly Disagree. A Not Applicable option is also available if the question does not apply. Two state-added questions were added to the 2011 YSS-F tool to gather information specific to National CLAS Standards (Culturally and Linguistically Appropriate Services): Q30 – Services were provided in a language my child understood. Q31 – In creating my child‟s service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) Additionally, the clarifiers at the end of one MHSIP question (Q15) and one state-added question on the YSS-F tool (Q29) were modified to provide additional clarification. The RBHA and/or provider may pre-print the following information on the survey tool prior to distribution to client:  RBHA Name  Provider Name  Survey Id  Provider Facility ID  Behavioral Health Category  Entitlement Status (Title XIX or XXI)   Distribution Method (Home, Clinic, or Phone) Client enrollment with DDD Both survey tools have four main sections: (1) demographic section, (2) MHSIP survey questions, (3) stateadded questions, and (4) open-ended qualitative section. The demographic section provides descriptive information about the consumer‟s age, gender, race, ethnicity, and relationship of the person completing the survey to the service recipient. The second section of the survey contains the MHSIP standardized questions. They serve as benchmark tools to compare consumer perception of behavioral health systems across the nation. 28 2011 Consumer Survey Report The third section consists of state-added questions to elicit consumer input regarding the inclusion of cultural preferences in the consumer‟s treatment planning and to provide information relative to the National CLAS Standards. The fourth and final section of the survey contains open-ended questions to solicit consumer comments. Two questions are asked of consumers – focusing on identifying what has been most helpful with their services, and what the consumer believes would improve services. The section entitled other comments is intended to provide consumers with an additional area on the survey to provide open-ended feedback on any issue. Confidentiality The front page of the survey questionnaire addresses confidentiality of the responses. Survey results are aggregated and not presented at an individual consumer level. The providers will never have access to completed surveys or individual survey results. Thematic analysis is conducted on written comments. Non-mandatory (Voluntary) The survey questionnaire likewise informs the respondent that participation in the survey is voluntary; every individual can choose to participate or not. It also notifies the respondent that non-participation will not affect the services they are currently or will receive in the future. Participating in the survey is one way the consumer may help improve the system they use. Languages The survey will be available to consumers in English and Spanish languages. Each survey form will be printed in English on one side and Spanish on the reverse side. For consumers with limited English proficiency who speak a language other than Spanish, the RBHA and/or the provider administering the survey will extend their best effort to translate the survey in the consumer‟s preferred language by utilizing the Language Line or other translation/interpretation services officially utilized by the RBHA or their provider. The extent of assistance provided in language translation should not attempt to define what the question means. Sampling Design Sample Frame The sample frame refers to the population eligible to take the survey. This is the pool from which ADHS/DBHS will randomly select the sampled population. ADHS/DBHS will create an adult and a youth sample from the CIS database for each GSA. Clients 18 or older on December 1, 2010 are grouped as adults, and clients under 18 are grouped as youth. The adult clients may be enrolled in any program: General Mental Health, Substance Abuse, or Seriously Mentally Ill. The sample frame will be composed of: (a) Client must be enrolled in FY2011; (b) Client must be Title XIX or Title XXI eligible in FY2011; (c) Client must have received a mental health service other than inpatient, transportation, laboratory and/or radiology services, or crisis; (d) The service must have occurred in the previous 6 months. Drawing of sample 29 2011 Consumer Survey Report ADHS/DBHS will create a random sample and provide each GSA with a list of 500 adult and 400 to 500 youth clients to be surveyed. Administering survey ADHS/DBHS will provide the RBHAs with a list of randomly selected clients. The RBHA will duplicate the Survey ID number on the survey tool to be offered that specific consumer. To protect the confidentiality of the consumer, none of the parties‟ privy to this information (DBHS, RBHA, Provider) will link the survey responses to consumers. ADHS/DBHS will provide the RBHAs with a database to store the survey results. The RBHA will disseminate the surveys; track which consumers have completed the survey; track why surveys were not completed; enter the survey data into the provided database; and submit to ADHS/DBHS: the completed paper surveys, the database containing all survey results, the database detailing which consumers completed a survey and for every consumer that did not complete a survey, the reason the survey was not completed. The RBHAs will enter data into the two ADHS/DBHS provided databases but the RBHAs will not alter the databases in any way. The provider agency is responsible for identifying the specific provider location or site from which the consumer is presently receiving services. Survey Methodology Distribution Method The primary distribution method is a non-clinical staff at the provider office (i.e., clinic) handing the survey questionnaire to the consumer. As the consumer checks in for their appointment, s/he will be provided with a copy of the survey questionnaire to complete. If the consumer agrees to participate, s/he will be requested to complete the survey prior to his/her appointment. If the consumer is unable to complete the questionnaire, s/he will be allowed to finish it on site after the appointment or be provided with an addressed, stamped envelope to mail the survey in if they did not have time to complete it in the office. A drop box will be provided on site for completed surveys. Additionally, a specific area at the provider office will be designated for completing the survey. If the individual randomly selected has a scheduled appointment at home during the survey window, the provider staff will bring the survey questionnaire at the appointment date. If the consumer agrees to participate, s/he will be advised to complete the survey after the staff leaves and to mail the completed questionnaire to the RBHA using the pre-addressed, stamped envelope provided with the survey. If the individual does not have an appointment during the survey window, a non-clinical staff at the RBHA may conduct the survey over the phone. As an alternative, a non-clinical staff at the provider site may contact the consumer by phone to ask for their participation in the survey, offering a return envelope for the completed survey to be mailed. All return envelopes provided as a means for the consumer to submit their completed survey must be addressed to the RBHA. 30 2011 Consumer Survey Report A check box in the questionnaire will be used to track the distribution method. The adult survey will be administered to the adult consumer. If the individual requests assistance, a guardian may complete the questionnaire on the consumer‟s behalf. The YSS-F will be administered to the parent/guardian of the child receiving services. If the parent or guardian is not at the appointment, then the survey will not be provided. List of Survey Clients ADHS/DBHS will provide each RBHA with a list of survey consumers to track those who complete a survey or the reason they did not complete a survey. The list of survey consumers will contain fields for:              RBHA ID Contractor ID Survey ID AHCCCS ID Last Name First Name Date of Birth Sex Survey Offered Date Survey Completed Date Reason Not Completed Provider ID Client contact phone number The RBHAs will submit their lists of survey clients to ADHS/DBHS bi-weekly throughout the survey administration period to keep ADHS/DBHS informed regarding the status of response rates and reasons for non-participation of sampled consumers. Due dates for submission of bi-weekly lists are included in the Task Timeline section on page 9 of this Protocol. Since this document contains protected health information, it will be treated as a confidential document. ADHS/DBHS Post-administration Follow-up During the survey administration period, RBHAs will attempt to achieve participation of all consumers on its list of survey clients. Every effort will be made to separate service delivery from sample collection. At the end of the administration period, RBHAs will provide ADHS/DBHS with the completed list of survey consumers indicating which consumers completed the survey and which did not, with the reasons and a contact phone number for those who did not participate. At that point, the RBHA will delete the list of survey consumers. RBHAs will submit to ADHS/DBHS the completed paper surveys and a database containing survey results by July 29, 2011. Tribal RBHAs participate in the Annual Consumer Survey by way of a convenience sampling of their enrolled consumers. These entities will be provided a master survey tool for each population that can be copied for distribution. Survey Administration 31 2011 Consumer Survey Report Survey Timeframe The survey will be administered for a period of two months: April 4 through May 27, 2011. Roles and Responsibilities ADHS/DBHS is responsible for the statewide oversight of the survey administration to ensure consistent implementation of the survey protocol. The protocol, client sample, survey instruments, and survey results database will be created by ADHS/DBHS. ADHS/DBHS will provide technical assistance throughout the survey process. Periodic monitoring, training, timelines, and use of checklist will be utilized to guide the T/RBHAs on critical points in the process. The T/RBHAs have the primary responsibility for ensuring that the protocol is precisely followed within their geographic regions. Direct oversight and assistance will be provided by the T/RBHAs to their providers. The T/RBHAs will ensure that the providers are appropriately trained and prepared to administer the survey. The RBHAs will enter data into the two ADHS/DBHS provided databases but the RBHAs will not alter the databases in any way including creating tables, changing the names of tables or fields, or creating a different value coding scheme. Each provider agency is primarily responsible for each of its sites in which the survey is to be administered. Each site will maintain all necessary materials for survey administration. At each site, a drop box and a designated area will be provided for consumers to complete the survey. Providers will also be responsible for the day-to-day operations – including having the survey tools, materials for completing the survey (pens, pencils, clipboards), envelopes for return of the survey if needed, assigned resources for administration and collection of data for the survey. Non-Randomly Selected Consumers (Walk-in Requests) ADHS/DBHS does not require the RBHAs to extend survey participation efforts beyond the randomly selected populations but at the individual discretion of each RBHA/GSA, the RBHA may furnish its provider agencies with survey forms to give consumers who express a desire to participate in the survey but whose names do not appear on the lists of survey consumers (i.e., the consumer was not randomly selected). However, to maintain the scientific rigor of the protocol, the survey questionnaire completed by this group of respondents will be tracked separately by the RBHA. Names of individuals who belong to this group will not be added to the survey client list. This group will be tracked in some other ways as described in the succeeding paragraph. If the RBHA elects to collect responses from non-selected consumers, the same protocol will be followed for this group of respondents but the staff administering the survey will ensure that these “voluntary” surveys are kept separate from those on the lists of survey clients (randomly selected consumers). Several control measures will be used. First, consumers that are not randomly selected to participate in the survey (i.e., walk-in respondents) will be given a copy of the survey questionnaire with the pre-filled section of the questionnaire not completed. Second, the survey questionnaire that will be used for non-randomly selected consumers will be color-coded. (Note: Surveys for the random sample will be printed on white paper.) RBHAs will enter the responses on the voluntary surveys into the same database as the results of the selected respondents, using the correct indicator, “C” for control (selected) respondents and “V” for voluntary (non-selected) respondents. 32 2011 Consumer Survey Report Pre-Survey Activities Notification to Consumers about the Survey To encourage greater participation, efforts will be made to inform consumers in advance about the survey. T/RBHAs and providers will be encouraged to utilize all or a combination of any of the following media: flyers, posters, website announcements, or other promotional materials. A staff member or members at each provider site will be assigned to work on the survey. In addition to daily survey administrative duties, a component of the staff member‟s role will include assisting consumers with the survey if necessary. Assistance may include: reading the survey to individuals unable to read, explaining the Likert scale used for scoring answers, emphasizing confidentiality of the survey, or ensuring consumers that participation in the survey is voluntary. Staff will be allowed to provide administrative assistance to the consumer, or provide encouragement to participate in the survey process. However, staff will not be able to explain the meaning of particular questions or provide interpretations on what particular questions mean. Data Management and Reporting Scoring Protocol The scoring protocol recommended by MHSIP will be utilized for evaluating the domain areas within the survey, as follows: Recode ratings of „not applicable‟ as missing values. For each survey, exclude domains more than one-third of the domain questions missing. Calculate the mean of the items for each respondent. Calculate the percent of scores that are less than 2.5 1. 2. 3. 4. Technical assistance ADHS will provide technical assistance to the RBHAs as needed. Response Rate Calculation: The rate will be calculated for each population for each GSA using the formula: Response rate = A / B Where: A= Total number of surveys returned B= 400 to 500 (sample population) Weighting methodology The statewide data will be weighted by GSA client population to compensate for the stratified sample collection. Weights will be applied to the survey data prior to any statewide data analysis. 33 2011 Consumer Survey Report Dissemination of Findings The following methods have been identified to circulate findings: Reporting of survey results in management meetings – Executive Management, Quality Management, Human Rights Committees, Behavioral Health Planning Council, other consumer advocate groups, and other interest groups that may be identified  Dissemination of information by providers to their local communities  Having copies of the survey available at the provider sites  Publishing results of survey on ADHS, RBHA and provider websites  Timeline for tasks to be completed by RBHA and ADHS/DBHS:       34 March 1, 2011: ADHS/DBHS provides each RBHA with: Lists of consumers in survey samples Survey tools, Adult and YSS-F – Survey tools in Adobe format to be copied by T/RBHA (white for selected respondents; light green for voluntary respondents). Header portion of survey tools will be editable for RBHA entry of information called for. Database – for RBHA data entry of survey results. March: RBHAs identify appropriate provider for consumers on lists of survey consumers, copy ADHS/DBHSprovided survey tools, complete top portions specific to each consumer and forward survey tools to providers. April 4 – May 27, 2011: Administer survey April 22, May 6, May 20, June 10, 2010: RBHA submits to ADHS/DBHS: Lists of survey clients – the lists of survey clients with current status regarding which consumers have and have not completed a survey. July 29, 2011: RBHA submits to ADHS/DBHS: Database containing survey results. Completed surveys – this includes surveys completed by selected respondents as well as voluntary respondents. A copy of the surveys is to be retained by the RBHA. December 9, 2011: RBHA submits a written report of survey findings to ADHS/DBHS. This report will provide analysis of 2010 consumer survey results and performance improvement activities planned or implemented to address areas in need of improvement. 2011 Consumer Survey Report ATTACHMENT-C Raw Data 35 2011 Consumer Survey Report Arizona Department of Health Services Division of Behavioral Health Services 2011 Annual Consumer Survey Raw Data Assumptions for interpreting data: 1. Denominator – Number of responses to specific domain or question on completed/analyzed surveys. 2. Numerator – Number of positive responses to specific domain or question on completed/analyzed surveys. 3. T/RBHA – specific numbers and percentages are based on actual valid survey returns. 4. Statewide RBHA numbers and percentages are based on weighted scores. 5. Tribal BHA scores not included in Statewide numbers/percentages due to Tribal BHA convenience sampling methodology. 6. Weighting: As a result, if weighting were not applied for the Statewide performance calculation, a GSA with a relatively smaller population would be over represented and a GSA with a larger population would be under represented. The weight applied to each GSA is determined by the number of clients enrolled in the GSA. Weighting is utilized to correct the sample design when describing statewide data. Each GSA is provided a similar number of surveys to have completed but the GSAs have very different population sizes. The table below details the specific weighting factors used: RBHA Adult Youth 36 Total Surveys RBHA Population RBHA Surveys Total Population Equation Weight 7 1,573 38,541 224 74,312 (1573*38541)/(224*74312) 3.64204 26 1,573 16,294 352 74,312 (1573*16294)/(352*74312) 0.97984 15 1,573 10,293 340 74,312 (1573*10293)/(340*74312) 0.64082 22 1,573 3,816 194 74,312 (1573*3816)/(194*74312) 0.41637 2 1,573 2,559 262 74,312 (1573*2559)/(262*74312) 0.20675 32 1,573 2,809 201 74,312 (1573*2809)/(201*74312) 0.29582 7 1,376 20,342 196 35,853 (1376*20342)/(196*35853) 3.98319 26 1,376 7,152 293 35,853 (1376*7152)/(293*35853) 0.93681 15 1,376 3,962 333 35,853 (1376*3962)/(333*35853) 0.45663 22 1,376 2,140 233 35,853 (1376*2140)/(233*35853) 0.35249 32 1,376 1,274 197 35,853 (1376*1274)/(197*35853) 0.24820 2 1,376 983 124 35,853 (1376*983)/(124*35853) 0.30425 2011 Consumer Survey Report 2011 MHSIP Consumer Survey Numbers Used in Calculation of Performance ADULT SURVEY RESULTS BY T/RBHA & MHSIP DOMAIN Domain: General Satisfaction T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 259 188 187 328 197 325 1,434 25 127 97 29 Numerator 231 161 171 292 164 293 1,238 23 115 91 27 Positive Response Percentage 89% 86% 91% 89% 83% 90% 86% 92% 91% 94% 93% Domain: Service Access T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 262 200 190 345 213 337 1,522 26 127 98 28 Numerator 231 172 169 292 176 287 1,280 22 102 84 27 Positive Response Percentage 88% 86% 89% 85% 83% 85% 84% 85% 80% 86% 96% Domain: Participation in Treatment Planning T/RBHA Denominator Cenpatico 2 246 Cenpatico 3 185 Cenpatico 4 184 CPSA 5 327 Magellan 195 NARBHA 332 Statewide RBHA 1,426 Gila River 24 Navajo Nation 127 Pascua Yaqui 96 White Mountain Apache 28 Numerator 224 169 174 302 160 303 1,242 22 113 86 24 Positive Response Percentage 91% 91% 95% 92% 82% 91% 87% 92% 89% 90% 86% 37 2011 Consumer Survey Report Domain: Service Quality & Appropriateness T/RBHA Denominator Cenpatico 2 260 Cenpatico 3 190 Cenpatico 4 190 CPSA 5 340 Magellan 209 NARBHA 333 Statewide RBHA 1,497 Gila River 25 Navajo Nation 128 Pascua Yaqui 97 Whtie Mountain Apache 28 Numerator 235 168 176 308 176 310 1,313 24 112 90 24 Positive Response Percentage 90% 88% 93% 91% 84% 93% 88% 96% 88% 93% 86% Domain: Outcomes T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Denominator 252 186 190 330 200 327 1,448 25 128 99 29 Numerator 205 139 145 246 136 247 1,039 21 105 81 22 Positive Response Percentage 81% 75% 76% 75% 68% 76% 72% 84% 82% 82% 76% Domain: Improved Functioning T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 257 189 189 329 202 331 1,458 25 129 98 29 Numerator 199 137 132 237 132 244 1,006 21 107 76 25 Positive Response Percentage 77% 72% 70% 72% 65% 74% 69% 84% 83% 78% 86% 38 2011 Consumer Survey Report Domain: Social Connectedness T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 257 193 188 340 207 333 1,489 26 131 100 30 Numerator 212 157 148 258 154 267 1,137 22 112 85 27 Positive Response Percentage 82% 81% 79% 76% 74% 80% 76% 85% 85% 85% 90% STATEWIDE RESULTS FOR EACH QUESTION IN DOMAIN General Satisfaction: Q1 Q2 Q3 Denominator 1,507 1,490 1,491 Numerator 1,336 1,217 1,315 Positive Response Percentage 89% 82% 88% Service Access Q4 Q5 Q6 Q7 Q8 Q9 1,515 1,485 1,444 1,501 1,497 1,429 1,234 1,295 1,145 1,290 1,220 1,060 81% 87% 79% 86% 81% 74% Participation in Treatment Planning Q11 Q17 1,494 1,460 1,327 1,180 89% 81% Service Quality & Appropriateness Q10 Q12 Q13 Q14 Q15 Q16 Q18 Q19 Q20 1,509 1,491 1,503 1,465 1,472 1,461 1,426 1,453 1,423 1,287 1,252 1,372 1,259 1,175 1,323 1,217 1,223 1,151 85% 84% 91% 86% 80% 91% 85% 84% 81% 39 2011 Consumer Survey Report Outcomes Q21 Q22 Q23 Q24 Q25 Q26 Q27 Q28 1,480 1,480 1,479 1,458 1,462 1,146 1,357 1,437 1,178 1,117 1,084 1,029 982 698 835 925 80% 75% 73% 71% 67% 61% 62% 64% Improved Functioning Q28 Q29 Q30 Q31 Q32 1,437 1,468 1,476 1,471 1,465 925 1,062 1,099 1,000 1,041 64% 72% 74% 68% 71% Social Connectedness Q33 Q34 Q35 Q36 1,456 1,458 1,453 1,449 1,079 1,112 900 1,157 74% 76% 62% 80% 40 2011 Consumer Survey Report 2011 MHSIP Consumer Survey Numbers Used in Calculation of Performance YOUTH SERVICES SURVEY FOR FAMILIES RESULTS BY T/RBHA & MHSIP DOMAIN Domain: General Satisfaction T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 124 194 233 288 181 329 1,309 28 45 56 20 Numerator 113 166 195 253 141 295 1,078 23 38 54 20 Positive Response Percentage 91% 86% 84% 88% 78% 90% 82% 82% 84% 96% 100% Domain: Service Access T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mountain Apache Denominator 124 193 230 283 180 330 1,299 28 43 55 20 Numerator 118 174 215 247 148 309 1,117 24 31 55 18 Positive Response Percentage 95% 90% 93% 87% 82% 94% 86% 86% 72% 100% 90% Domain: Participation in Treatment Planning T/RBHA Denominator Cenpatico 2 124 Cenpatico 3 194 Cenpatico 4 233 CPSA 5 283 Magellan 180 NARBHA 327 Statewide RBHA 1,299 Gila River 26 Navajo Nation 45 Pascua Yaqui 56 White Mountain Apache 20 Numerator 119 173 220 264 170 308 1,222 25 39 53 18 Positive Response Percentage 96% 89% 94% 93% 94% 94% 94% 96% 87% 95% 90% 41 2011 Consumer Survey Report Domain: Cultural Sensitivity T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Denominator 123 187 229 270 165 316 1,219 27 45 57 20 Numerator 122 179 223 261 158 307 1,174 27 41 57 20 Positive Response Percentage 99% 96% 97% 97% 96% 97% 96% 100% 91% 100% 100% Domain: Outcomes T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui Whtie Mountain Apache Denominator 122 194 231 280 176 325 1,278 28 44 56 20 Numerator 86 142 161 190 107 223 824 18 32 40 17 Positive Response Percentage 70% 73% 70% 68% 61% 69% 64% 64% 73% 71% 85% Domain: Improved Functioning T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mtn Denominator 122 195 232 280 176 325 1,279 28 44 56 20 Numerator 89 145 165 198 111 231 854 18 33 42 17 Positive Response Percentage 73% 74% 71% 71% 63% 71% 67% 64% 75% 75% 85% 42 2011 Consumer Survey Report Domain: Social Connectedness T/RBHA Cenpatico 2 Cenpatico 3 Cenpatico 4 CPSA 5 Magellan NARBHA Statewide RBHA Gila River Navajo Nation Pascua Yaqui White Mtn Denominator 119 191 232 278 173 326 1,264 27 44 56 20 Numerator 115 167 211 250 152 294 1,125 26 40 53 19 Positive Response Percentage 97% 87% 91% 90% 88% 90% 89% 96% 91% 95% 95% STATEWIDE RESULTS FOR EACH QUESTION IN DOMAIN General Satisfaction: Denominator Numerator Q1 1,324 1,120 Q4 1,273 1,041 Q5 1,230 963 Q7 1,309 1,081 Q10 1,290 1,036 Q11 1,269 936 Positive Response Percentage 85% 82% 78% 83% 80% 74% Service Access Q8 Q9 Participation in Treatment Planning Q2 Q3 Q6 1,309 1,312 1,055 1,084 81% 83% 1,267 1,295 1,306 1,098 1,182 1,227 87% 91% 94% Cultural Sensitivity Q12 Q13 Q14 Q15 1,303 1,179 1,305 1,186 1,236 1,068 1,242 1,091 95% 91% 95% 92% Outcomes Q16 Q17 Q18 Q19 Q20 Q21 Q22 1,291 1,262 1,273 1,262 1,271 1,278 1,282 879 804 866 865 752 769 817 68% 64% 68% 69% 59% 60% 64% 43 2011 Consumer Survey Report Improved Functioning Q16 Q17 Q18 Q19 Q20 Q22 1,291 1,262 1,273 1,262 1,271 1,282 879 804 866 865 752 817 68% 64% 68% 69% 59% 64% Social Connectedness Q23 Q24 Q25 Q26 1,270 1,256 1,263 1,251 1,082 1,111 1,054 1,090 85% 88% 83% 87% CLAS (New in 2011) Q30 Q31 1,247 603 1,204 456 97% 76% 44 2011 Consumer Survey Report ATTACHMENT-D 45 2011 Consumer Survey Report 2011 Adult Demographics 2011 Adult Consumer Survey, Percent of Positive Response by Domain and Subgroup Subgroup General Satisfaction Service Access Service Quality and Appropriateness Participation in Treatment Planning Outcomes Improved Functioning Social Connectedness N % N % N % N % N % N % N % 505 731 88% 85% 559 720 88% 81% 546 765 88% 87% 525 716 90% 85% 470 567 79% 67% 428 576 73% 66% 480 655 79% 74% 82 219 373 490 48 8 87% 85% 84% 88% 92% 100% 89 202 418 495 50 8 90% 75% 86% 85% 98% 100% 89 233 405 507 51 7 91% 86% 86% 88% 100% 100% 88 216 381 488 41 7 98% 82% 87% 88% 80% 100% 81 166 332 404 32 3 86% 65% 73% 71% 67% 43% 72 172 320 377 39 7 81% 66% 70% 66% 75% 100% 71 202 377 415 42 8 76% 77% 79% 72% 81% 100% 897 75 10 44 87% 84% 71% 80% 916 86 14 45 85% 88% 100% 66% 960 81 9 46 90% 84% 64% 69% 907 68 14 45 88% 85% 100% 71% 761 53 10 43 74% 64% 100% 64% 745 50 10 42 71% 60% 91% 64% 819 72 6 46 77% 78% 100% 68% 3 33 100% 64% 3 37 75% 67% 3 41 75% 73% 3 37 75% 77% 2 32 50% 58% 2 30 59% 73% 2 39 100% 70% 308 872 90% 85% 326 893 92% 82% 306 942 88% 88% 296 883 90% 86% 262 721 76% 70% 249 705 73% 67% 276 799 80% 75% 46 79 243 259 575 74% 80% 86% 92% 86% 61 79 240 255 597 94% 76% 83% 87% 84% 54 85 263 263 601 84% 83% 90% 91% 86% 60 77 239 244 572 98% 85% 88% 88% 86% 40 76 198 190 497 70% 79% 70% 70% 72% 38 70 188 185 485 64% 71% 67% 67% 70% 41 89 219 213 534 72% 86% 75% 75% 76% Gender Male Female Age Group 18-21 22-30 31-45 46-65 66-75 76+ Race White only African American only Asian only Am Indian/Al Native only Nat Hawaiian/Pacific Islander only Multiple Race Ethnicity Hispanic or Latino Not Hispanic or Latino Length of Services 0-6 months 7-11 months 1-2 years 3-5 years 5 years + 46 2011 Consumer Survey Report Program SMI Non-SMI 47 466 766 86% 87% 486 789 83% 85% 484 824 83% 90% 455 782 82% 90% 393 641 71% 72% 384 617 68% 70% 415 717 72% 79% 2011 Consumer Survey Report ATTACHMENT-E 48 2011 Consumer Survey Report 2011 Adult Line Item 2011 Adult Consumer Survey, Statewide Percent of Positive Response by Line Item (Line specific numbers are based on actual valid survey returns. Domain percentages are based on weighted scores.) Survey Item General Satisfaction: 1. I like the services that I received here. 2. If I had other choices, I would still get services from this agency. 3. I would recommend this agency to a friend or family member. Service Access: 4. The location of services was convenient (parking, public transportation, distance, etc.) 5 Staff were willing to see me as often as I felt it was necessary. 6. Staff returned my call in 24 hours. 7. Services were available at times that were good for me. 8. I was able to get all the services I thought I needed. 9. I was able to see a psychiatrist when I wanted to. Participation in Treatment Planning 11. I felt comfortable asking questions about my treatment and medication. 17. I, not staff, decided my treatment goals. Service Quality and Appropriateness 10. Staff here believe that I can grow, change and recover. 12. I feel free to complain. 13. I was given information about my rights. 14. Staff encouraged me to take responsibility for how I live my life. 15. Staff helped me to understand what side effects to watch out for. 16 Staff respected my wishes about who is and who is not to be given information about my treatment.. 18. Staff were sensitive to my cultural background (For example: values, traditions, beliefs, race, language, etc.) 19. Staff helped me obtain the information I needed so that I could take charge of managing my illness. 20. I was encouraged to use consumer-run programs (support groups, drop-in centers, crisis phone line, etc.) Outcomes 21. I deal more effectively with daily problems. 22. I am better able to control my life. 23. I am better able to deal with crisis. 24 I am getting along better with my family. 25. I do better in social situations. 26. I do better in school and/or work 27. My housing situation has improved 28. My symptoms are not bothering me as much. 49 Number Percent 1238 1336 1217 1315 1280 86% 89% 82% 88% 84% 1234 1295 1145 1290 1220 1060 1242 1327 1180 1313 1287 1252 1372 1259 1175 81% 87% 79% 86% 82% 74% 87% 89% 81% 88% 85% 84% 91% 86% 80% 1323 91% 1217 86% 1223 84% 1151 1039 1178 1117 1084 1029 982 698 835 925 81% 72% 80% 76% 73% 71% 67% 61% 62% 64% 2011 Consumer Survey Report Improved Functioning 28. My symptoms are not bothering me as much. 29. I do things that are more meaningful to me. 30. I am better able to take care of my needs. 31. I am better able to handle things when they go wrong. 32 I am better able to do things that I want to do. Social Connectedness 33. I am happy with the friendships I have. 34. I have people with whom I can do enjoyable things. 35. I feel I belong in my community. 36. In a crisis, I would have the support I need from family or friends. 50 1006 925 1062 1099 1000 1041 1137 1079 1112 900 1157 69% 64% 72% 75% 68% 71% 69% 74% 76% 62% 80% 2011 Consumer Survey Report ATTACHMENT-F 51 2011 Consumer Survey Report 2011 YSS-F Demographics 2011 YSS-F, Percent of Positive Response by Domain and Subgroup General Satisfaction Subgroup Service Access Cultural Competency Participation in Treatment Planning Outcomes Improved Functioning Social Connectedness N % N % N % N % N % N % N % 677 393 80% 87% 709 404 85% 89% 758 409 96% 97% 789 425 93% 96% 529 287 64% 64% 544 302 66% 68% 732 389 89% 89% 0-4 5-12 13-17 78 612 387 92% 83% 80% 83 633 401 98% 87% 83% 78 662 434 99% 97% 95% 80 706 436 94% 95% 92% 56 492 276 69% 68% 58% 58 502 294 71% 69% 62% 8 639 409 93% 90% 88% Race White only African American only Asian only Am Indian/Al Native only Nat Hawaiian/Pacific Islander only Multiple Race 697 88 7 34 10 72 83% 77% 88% 87% 100% 77% 742 87 7 31 10 82 88% 76% 88% 80% 100% 83% 773 100 7 38 10 74 98% 95% 88% 100% 100% 85% 787 102 8 34 10 92 94% 91% 100% 90% 100% 99% 507 78 6 27 9 54 62% 71% 86% 69% 90% 56% 525 83 6 28 9 58 64% 76% 86% 72% 90% 60% 714 98 7 37 9 86 88% 87% 100% 97% 100% 94% 398 655 87% 80% 395 697 88% 85% 418 731 96% 96% 428 768 94% 94% 315 494 71% 61% 327 512 73% 64% 411 689 91% 88% 92 126 387 272 145 81% 83% 86% 84% 73% 102 132 414 275 151 87% 90% 91% 85% 77% 105 149 411 290 165 96% 100% 97% 96% 92% 107 135 437 312 173 93% 89% 97% 96% 87% 61 98 312 203 115 54% 67% 72% 62% 59% 64 103 322 213 117 57% 71% 74% 65% 60% 92 137 412 283 151 86% 94% 94% 90% 77% 752 251 83% 78% 786 254 88% 78% 833 269 97% 95% 850 299 95% 91% 573 198 65% 62% 593 206 68% 64% 787 268 91% 85% Gender Male Female Age Group Ethnicity Hispanic or Latino Not Hispanic or Latino Length of Services 0-6 months 7-11 months 1-2 years 3-5 years 5 years + CFT Yes No 52 2011 Consumer Survey Report ATTACHMENT-G 53 2011 Consumer Survey Report 2011 YSS-F Line Item 2011 YSSF, Statewide Percent of Positive Response by Line Item (Line specific numbers are based on actual valid survey returns. Domain percentages are based on weighted scores.) Survey Item Number Percent General Satisfaction: 1. Overall, I am satisfied with the services my child received. 4. The people helping my child stuck with us no matter what. 5. I felt my child had someone to talk to when he/she was troubled. 7. The services my child and/or family received were right for us. 10. My family got the help we wanted for my child. 11. My family got as much help as we needed for my child. Service Access 8. The location of services was convenient for us. 9. Services were available at times that were convenient for us. Participation in Treatment Planning 2. I helped to choose my child‟s services. 3. I helped to choose my child‟s treatment goals. 6. I participated in my child‟s treatment. Cultural Sensitivity 12 Staff treated me with respect. 13. Staff respected my family‟s religious/spiritual beliefs. 14. Staff spoke with me in a way that I understood. 15. Staff were sensitive to my cultural/ethnic background. Outcomes 16 My child is better at handling daily life. 17. My child gets along better with family members. 18. My child gets along better with friends and other people. 19. My child is doing better in school and/or work. 20. My child is better able to cope when things go wrong. 21. I am satisfied with our family life right now. 22. My child is better able to do things he or she wants to do. Improved Functioning 16 My child is better at handling daily life. 17. My child gets along better with family members. 18. My child gets along better with friends and other people. 19. My child is doing better in school and/or work. 20. My child is better able to cope when things go wrong. 22. My child is better able to do things he or she wants to do. Social Connectedness 23. I know people who will listen and understand me when I need to talk. 24. I have people that I am comfortable talking with about my child‟s problems. 25. In a crisis, I would have the support I need from family or friends. 26. I have people with whom I can do enjoyable things. 1078 1120 1041 963 1081 1036 936 1117 1055 1084 1222 1098 1182 1227 1174 1236 1068 1242 1091 824 879 804 866 865 752 769 817 854 879 804 866 865 752 817 1125 1082 1111 1054 1090 82% 85% 82% 78% 83% 80% 74% 86% 81% 83% 94% 87% 91% 94% 96% 95% 91% 95% 92% 65% 68% 64% 68% 69% 59% 60% 64% 67% 68% 64% 68% 69% 59% 64% 89% 85% 88% 83% 87% 54 2011 Consumer Survey Report ATTACHMENT-H 55 2011 Consumer Survey Report List of Acronyms 2011 Consumer Survey Report Acronyms Description ADHS/DBHS ASOC CBHS CFT CPSA CSOC CSOCPR CSR GMH GSA MHSIP MMWIA NARBHA NASMHPD NOMs NRI QM RBHA RSS SA SAMSHA SMI TBHA WSDSG YSS-F Arizona Department of Health Services/Division of Behavioral Health Services Adult System of Care Cenpatico Behavioral Health Services Child and Family Team Community Partnership of Southern Arizona Children‟s System of Care Child System of Care Practice Review Consumer Survey Report General Mental Health Geographical Service Area Mental Health Statistics Improvement Program Meet Me Where I Am Northern Arizona Regional Behavioral Health Authority National Association of State Mental Health Program Directors National Outcomes Measures National Research Institute Quality Management Regional Behavioral Health Authority Recovery Support Specialist Substance Abuse Substance Abuse and Mental Health Services Administration Serious Mental Illness Tribal Behavioral Health Authority Western States Decision Support Group Youth Services Survey for Families 56 2011 Consumer Survey Report APPENDIX-1 (See attached) Year 2011 Adult Consumer Survey (English and Spanish) 57 2011 Consumer Survey Report THIS SECTION MUST BE COMPLETED BY T/RBHA OR PROVIDER! Name of Service Agency: ____________________________________________________________________ T/RBHA NAME: _________________________ SMI  Program/Fund Source: Client’s Entitlement Status: Survey completed at: Facility ID: ______________ Non-SMI (GMH or SA)  TXIX  TXXI  Survey ID: ________________________ Home  Clinic  By Phone  Client enrolled in DDD: Yes _____ No _____ YEAR 2011 ADULT CONSUMER SURVEY Please help us improve our program by completing this survey about the services you have received in the last six months. We are interested in your honest opinion. All responses will be treated as confidential. Any personal information will be excluded in the presentation of the survey results. Your current and/or future services will not be affected if you decide not to participate in this survey. If you have already taken this survey during the months of April or May 2011, you do not need to complete it again. After you have completed the survey, please fold and drop it in the survey box before you leave the office today. Thank you. Use Pen or Pencil Please fill in the bubbles completely with your answers to the following questions: Information about the Person Receiving Services: Age: Sex:  Male  Female Ethnicity:  Hispanic or Latino  Not Hispanic or Latino Please check all applicable race categories: Race:  White  Black/African American  Native Hawaiian/Pacific Islander  Asian  American Indian/Alaska Native How long have you been receiving mental health and/or substance abuse services? (from any provider)  0 - 6 months  7 - 11 months  1 - 2 years  3 - 5 years  more than 5 years About the Person Completing This Survey: I am the person directly receiving services.  Yes  No If not, please check your relation to the person who is directly receiving services:  Parent/Guardian  Friend  Relative (Uncle, Aunt, Cousin, Grandparent, etc.) Please continue to answer questions on the next page. 58  Other 2011 Consumer Survey Report Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 1. I like the services that I received here.       2. If I had other choices, I would still get services from this agency.       3. I would recommend this agency to a friend or family member. 4. The location of services was convenient (parking, public transportation, distance, etc.) 5. Staff were willing to see me as often as I felt it was necessary.                   6. Staff returned my call in 24 hours.       7. Services were available at times that were good for me.       8. I was able to get all the services I thought I needed.       9. I was able to see a psychiatrist when I wanted to.       10. Staff here believe that I can grow, change and recover.       11. I felt comfortable asking questions about my treatment and medication.       12. I feel free to complain.       13. I was given information about my rights.       14. Staff encouraged me to take responsibility for how I live my life.       15. Staff helped me to understand what side effects I should watch out for. 16. Staff respected my wishes about who is and who is not to be given information about my treatment. 17. I, not staff, decided my treatment goals. 18. Staff were sensitive to my cultural background. (For example: values, traditions, beliefs, race, language, etc.) 19. Staff helped me obtain the information I needed so that I could take charge of managing my illness. 20. I was encouraged to use consumer-run programs (support groups, drop-in centers, crisis phone line, etc.)                                     21. I deal more effectively with daily problems.       22. I am better able to control my life.       23. I am better able to deal with crisis.       24. I am getting along better with my family.       25. I do better in social situations.       26. I do better in school and/or work.       27. My housing situation has improved.       28. My symptoms are not bothering me as much.       29. I do things that are more meaningful to me.       30. I am better able to take care of my needs.       31. I am better able to handle things when they go wrong.       32. I am better able to do things that I want to do.       In order to provide the best possible behavioral health services, we need to know what you think about the services you received DURING THE LAST 6 MONTHS, the people who provided it, and the results. As a direct result of the services I received: Please continue to answer questions on the next page. 59 2011 Consumer Survey Report Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 33. I am happy with the friendships I have.       34. I have people with whom I can do enjoyable things.       35. I feel I belong in my community.       36. In a crisis, I would have the support I need from family or friends.                         For questions 33-36 please answer for relationships with persons other than your mental health provider(s) State-added Questions: 37. In creating my service plan my cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) 38. Services were provided in a language I understood. 39. In creating my service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) If you do not participate in tribal customs, please mark Not Applicable. Please answer the following questions to let us know how you are doing. 40. How long have you received mental health services from this provider?  Less than a year (less than 12 months) (Continue to Question 41)  1 year or more (at least 12 months) (Skip to Question 44) If you answered “Less than a year (less than 12 months)”, please complete questions 41 - 43. 41. Were you arrested since you began to receive mental health services? 42. Were you arrested during the 12 months prior to that? 43. Since you began to receive mental health services, have your encounters with police…  Yes  No  Yes  No  been reduced (for example, I have not been arrested, hassled by police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (I had no police encounters this year or last year) If you answered “1 year or more (at least 12 months)”, please complete questions 44 - 46. 44. Were you arrested during the last 12 months?  Yes  No  Yes  No 45. Were you arrested during the 12 months prior to that? 46. Over the last year, have your encounters with police…  been reduced (for example, I have not been arrested, hassled by police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (I had no police encounters this year or last year) Please continue to answer questions on the next page. 60 2011 Consumer Survey Report Please feel free to use the space provided below to comment on any of your answers. Also, if there are areas which were not covered by this questionnaire which you feel should have been, please write them in the comments section. Thank you for your time and cooperation in completing this questionnaire. What have been some of the most helpful things about the services you received over the last 6 months? ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ What would improve the services that you receive here? ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Please list any other comments you may want to share: ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Thank you for completing this questionnaire. 61 2011 Consumer Survey Report ¡¡ESTA SECCIÓN DEBE SER COMPLETADA POR LA RBHA O EL PROVEEDOR!! Nombre de Agencia de Servicio ____________________________________________________ NOMBRE de LA RBHA: ___________________________ SMI  Fuente de Programa/Fondo: Encuesta tomada en: Casa  No - SMI (GMH o SA)  TXIX  El Estado de Elegibilidad del Cliente: ID de la agencia o proveedor: ___________ TXXI  Clínica  Por teléfono  ID de la encuesta: _________________ Cliente inscrito en DDD: Si_____ No_____ ENCUESTA DE SERVICIOS PARA ADULTOS QUE RECIBIERON SERVICIOS DURANTE EL AÑO 2011 Por favor ayúdenos a mejorar nuestro programa completando ésta revisión sobre los servicios que usted ha recibido en los últimos seis meses. Estamos interesados en su opinión honesta. Todas las respuestas serán tratadas como confidenciales. Cualquier información personal será excluida en la presentación de los resultados ésta encuesta. Sus servicios actuales y/o futuros no se verán afectados si usted decide no participar en ésta encuesta. Si usted ha tomado ya ésta encuesta en los meses de abril o mayo de 2011, no tiene que completarla otra vez. Después de que usted haya finalizado, por favor doble la hoja y deposítela en la caja de encuestas, antes de usted salga de la oficina hoy. Gracias. Use pluma o lápiz Por favor rellene las burbujas completamente con sus respuestas a las siguientes preguntas: Información acerca de la persona que recibe los servicios: Edad: _______  Masculino Sexo:  Femenino  Hispano o Latino Pertenencia étnica:  No Hispano o Latino Por favor compruebe todas las categorías de raza aplicables Raza:  Blanco  Negro/Africano Americano  Nativo Americano / Nativo de Alaska  Asiático  Nativo de Hawai/Isleño del Pacífico ¿Por cuánto tiempo ha estado recibiendo servicios de salud mental y/o de abuso de sustancias? (de cualquier proveedor )  0 - 6 meses  7 - 11 meses  1 - 2 años  3 - 5 años  más de 5 años Sobre la persona que completa ésta encuesta: Soy la persona que directamente recibe servicios:  Sí  No Si no, por favor marque la relación con la persona que recibe directamente servicios:  Padre/Guardián  Amigo  Pariente (Tío, Tía, Primo, Abuelo, etc.) Por favor siga contestando preguntas en la siguiente página. 62  Otro 2011 Consumer Survey Report Neutro       2. Si yo tuviera otras opciones, yo todavía preferiría los servicios de esta agencia.       3. Yo recomendaría ésta agencia a un miembro de la familia o un amigo.       4. La ubicación de los servicios fue conveniente (estacionamiento, transporte público, distancia, etc.)       5. El personal quiso verme tan a menudo como sentí que era necesario.       6. El personal devolvió mi llamada en 24 horas.       7. Los servicios estaban disponibles a las horas que eran convenientes para mí.       8. Recibí los servicios que pensé que necesitaba.       9. Pude ver a un Psiquiatra cuando así lo solicité.       10. El personal aquí cree que puedo crecer, cambiar y recuperarme.       11. Me sentí cómodo haciendo preguntas acerca de mi tratamiento y medicación.       12. Me siento libre de presentar mis inquietudes sobre los servicios.       13. Me dieron la información sobre mis derechos.       14. El personal me animó a tomar la responsabilidad de como vivo mi vida. 15. El personal me ayudó a entender acerca de los efectos secundarios sobre los que debo estar preparado. 16. El personal respetó mis deseos sobre a quién sí y a quien no deben dar información sobre mi tratamiento.                   17. Yo, y no el personal, decidí las metas de mi tratamiento.       18. El personal fue respetuoso de mis raíces culturales/étnicas. (Por ejemplo: mis valores, tradiciones, creencias, raza, lenguaje, etc.).       19. El personal me ayudó a obtener la información que necesité de modo que yo pudiera hacerme cargo de manejar mi enfermedad. 20. Fui animado a usar programas manejados por consumidores (grupos de apoyo, centros de ayuda informal, línea telefónica de crisis, etc.)             21. Manejo de forma más eficaz los problemas diarios.       22. Soy más capaz de controlar mi vida.       23. Soy más capaz de lidiar con las crisis.       24. Me llevo mejor con los miembros de mi familia.       25. Me desenvuelvo mejor en situaciones sociales.       26. Tengo mejor desempeño en la escuela y/o trabajo.       27. Mi situación de vivienda ha mejorado.       28. Los síntomas no me molestan tanto como antes.       29. Hago cosas que son más significativas para mí.       30. Soy más capaz de ocuparme de mis necesidades.       31. Soy más capaz de manejar las cosas o situaciones cuando no van bien.       32. Soy más capaz de hacer cosas que quiero hacer.       No Aplica De acuerdo 1. Me gustaron los servicios que recibí aquí. A fin de proporcionar un mejor servicio en los cuidados de salud mental y de abuso de sustancias, tenemos que saber lo que usted piensa sobre los servicios que recibió DURANTE los 6 MESES PASADOS, la gente que lo proporcionó, y los resultados. En Desacuerd o Muy en desacuerd o Muy de acuerdo POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE Como un resultado directo de los servicios recibí: Por favor siga contestando preguntas en la siguiente página 63 2011 Consumer Survey Report Muy de acuerdo De acuerdo Neutro En Desacuerdo Muy en desacuerdo No Aplica POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 33. Soy feliz con las amistades que tengo.       34. Tengo a personas con quienes puedo hacer cosas agradables.       35. Siento que pertenezco a mi comunidad.       36. En una crisis, yo tendría el apoyo que necesito de familia o amigos.       Preguntas adicionales del estado: 37. En la creación de mi plan de servicios, mis preferencias culturales fueron incluidas. (Por ejemplo; mis valores, tradiciones, creencias, raza, lenguaje, etc.). 38. Los servicios me fueron proporcionados en un lenguaje que entendí.             39. En la creación de mi plan de servicios las tradiciones y creencias de mi tribu fueron incluidas (Por ejemplo: me preguntaron si quería participar en sesiones de sanación tradicional). Si usted no participa de las tradiciones tribales, por favor marque la opción "No aplica".       Para las preguntas 33-36 por favor responda basado en las relaciones con personas diferentes de su proveedor (es) de salud mental. Por favor conteste las siguientes preguntas para hacernos saber cómo se encuentra. 40. ¿Durante cuánto tiempo ha recibido usted servicios de salud mental de éste proveedor?  Menos de un año (menos de 12 meses) (Continué con la pregunta 41)  1 año o más (al menos 12 meses) (Vaya a la pregunta 44) Si su respuesta es “menos de un año (menos de 12 meses)”, por favor complete las preguntas 41-43. 41. ¿Ha sido usted arrestado desde que comenzó a recibir  Sí  No servicios de salud mental? 42. ¿Ha sido usted arrestado durante los 12 meses  Sí  No anteriores a esto? 43. Desde que usted comenzó a recibir servicios de salud  han disminuido (por ejemplo, no he sido arrestado, mental, ha tenido incidentes con la policía… molestado por la policía, enviado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (yo no he tenido ningún incidente con la policía durante este año o el año pasado) Si su respuesta es “1 año o más (al menos 12 meses)”, por favor complete las preguntas 44 - 46. 44. ¿Ha sido arrestado durante los 12 meses pasados?  Sí  No 45. ¿Ha sido arrestado durante los 12 meses anteriores a esto?  Sí  No 46. Durante el año pasado, sus incidentes con la policía …  han disminuido (por ejemplo, no he sido arrestado, molestado por policía, enviado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (yo no he tenido ningún encuentro con la policía durante este año o el año pasado) Por favor siga contestando preguntas en la siguiente página. 64 2011 Consumer Survey Report Por favor siéntase libre de usar el espacio proporcionado abajo para comentar sobre cualquiera de sus respuestas. También, si hay áreas que no fueron cubiertas por este cuestionario que usted siente que deberían haber sido incluidas, por favor escríbalas en la sección de comentarios. Gracias por su tiempo y cooperación en completar éste cuestionario. ¿Cuáles han sido algunas cosas más provechosas sobre los servicios que usted recibió durante los 6 meses pasados? ¿Qué mejoraría los servicios qué usted recibe aquí? Por favor escriba cualquier otro comentario que usted quiera compartir: Gracias por completar este cuestionario. 65 2011 Consumer Survey Report APPENDIX-2 (See attached) Year 2011 Youth Consumer Survey for Families (English and Spanish) 66 2011 Consumer Survey Report THIS SECTION MUST BE COMPLETED BY T/RBHA OR PROVIDER! Name of Service Agency: ____________________________________________________________________ T/RBHA NAME: ________________________ Client’s Entitlement Status: TXIX  Facility ID:_______________ TXXI  Survey ID: ____________________________ Survey completed at: Home  Clinic  By Phone  Client enrolled in DDD: Yes ______ No ______ YEAR 2011 YOUTH SERVICES SURVEY FOR FAMILIES Please help us improve our program by completing this survey about the services you have received in the last six months. We are interested in your honest opinion. All responses will be treated as confidential. Any personal information will be excluded in the presentation of the survey results. Your child and family’s current and/or future services will not be affected if you decide not to participate in this survey. If you have already taken this survey in the months of April or May 2011, then you do not need to complete it again. After you have completed the survey, please fold and drop it in the survey box before you leave the office today. Thank you. Use Pen or Pencil Please fill in the bubbles completely with your answers to the following questions: Information about the Person Receiving Services: Child’s Age: Child’s Sex:  Male Child’s Ethnicity:  Hispanic or Latino  Female  Not Hispanic or Latino Please check all applicable race categories: Child’s Race:  White  Black/African American  Native Hawaiian/Pacific Islander  Asian  American Indian/Alaska Native How long has your child been receiving mental health and/or substance abuse services? (from this provider)  0 - 6 months  7 - 11 months  1 - 2 years Please check your relationship with the child: Does your family have a Child and Family Team?  3 - 5 years  more than 5 years  Parent/Guardian  Friend  Relative (Uncle, Aunt, Cousin, Grandparent, etc.)  Other  Yes  No Please continue to answer questions on the next page. PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 67 Strongly Agree Agree I am Neutral Disagree Strongly Disagree Not Applicable 2011 Consumer Survey Report 1. Overall, I am satisfied with the services my child received.       2. I helped to choose my child’s services.       3. I helped to choose my child’s treatment goals.       4. The people helping my child stuck with us no matter what.       5. I felt my child had someone to talk to when he/she was troubled.       6 I participated in my child’s treatment.       7. The services my child and/or family received were right for us.       8. The location of services was convenient for us.       9. Services were available at times that were convenient for us.       10. My family got the help we wanted for my child.       11. My family got as much help as we needed for my child.       12. Staff treated me with respect.       13. Staff respected my family’s religious/spiritual beliefs.       14. Staff spoke with me in a way that I understood. 15. Staff were sensitive to my cultural/ethnic background. (For example: values, traditions, beliefs, race, language, etc.) As a result of the services my child and/or family received: 16. My child is better at handling daily life.                   17 My child gets along better with family members.       18. My child gets along better with friends and other people.       19. My child is doing better in school and/or work.       20. My child is better able to cope when things go wrong.       21 I am satisfied with our family life right now.       Please help our agency make services better by answering some questions about the services your child received OVER THE LAST 6 MONTHS.       22. My child is better able to do things he or she wants to do. As a result of the services my child and/or family received: please answer for relationships with persons other than your mental health provider(s).       23 I know people who will listen and understand me when I need to talk. 24. I have people that I am comfortable talking with about my child’s       problems.       25 In a crisis, I would have the support I need from family or friends. 26. I have people with whom I can do enjoyable things.      27. What has been the most helpful thing about the services you and your child received over the last 6 months? ________________________________________________________________________________________________ 28. What would improve the services here? _____________________________________________________________ ________________________________________________________________________________________________ Please continue to answer questions on the next page. 68  2011 Consumer Survey Report Please answer the following questions to let us know how your child is doing. 32. Is your child currently living with you? Agree I am Neutral Disagree Strongly Disagree Not Applicable State-added Questions: 29. In creating my child’s service plan our cultural preferences were included. (For example: values, traditions, beliefs, race, language, etc.) 30. Services were provided in a language my child understood. 31. In creating my child’s service plan our tribal customs and beliefs were included. (For example: being asked to participate in traditional healing practices.) If you do not participate in tribal customs, please mark Not Applicable. Strongly Agree PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY                    Yes  No 33. Has your child lived in any of the following places in the last 6 months? (CHECK ALL THAT APPLY)  With one or both parents  With another family member  Foster home  Therapeutic foster home  Crisis shelter  Homeless shelter  Group home  Residential treatment facility  Hospital  Local jail or detention facility  State correctional facility 34. In the last year, did your child see a medical doctor (or nurse) for a health check up or because he/she was sick? (Check one)  Runaway/homeless/on the streets  Yes, in a clinic or office  Yes, but only in a hospital emergency room  No  Do not remember 35. Is your child on medication for emotional/behavioral problems? 36. If yes, did the doctor or nurse tell you and/or your child what side effects to watch for? 37. Is your child still getting services from this provider? 38. How long did your child receive services from this provider?  Yes  No  Yes  No  Yes  No  Less than 1 month  1-5 Months  6 months to 1 year  More than 1 year (skip to Question 45) 39. Was your child arrested since beginning to receive mental health services? 40. Was your child arrested during the 12 months prior to that? 41. Since your child began to receive mental health services, have their encounters with the police…  Yes  No  Yes  No  been reduced (for example, they have not been arrested, hassled by the police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (They had no police encounters this year or last year) 42. Was your child expelled or suspended from school since beginning services? 43. Was your child expelled or suspended from school during the 12 months prior to that?  Yes  No  Yes  No Please continue to answer questions on the next page. 69 2011 Consumer Survey Report PLEASE MARK YOUR ANSWERS BY FILLING IN THE BUBBLES COMPLETELY 44. Since starting to receive services, the number of days my child was in school is…  greater  about the same  less  does not apply (please select why this does not apply)  child did not have a problem with attendance before starting services  child is too young to be in school  child was expelled from school  child is home schooled  child dropped out of school  Other: 45. Was your child arrested during the last 12 months? 46. Was your child arrested during the 12 months prior to that? 47. Over the past year, have your child’s encounters with the police…  Yes  No  Yes  No  been reduced (for example, they have not been arrested, hassled by the police, taken by police to a shelter or crisis program)  stayed the same  increased  not applicable (They had no police encounters this year or last year) 48. Was your child expelled or suspended from school during the last 12 months? 49. Was your child expelled or suspended from school during the 12 months prior to that? 50. Over the last year, the number of days my child was in school is…  Yes  No  Yes  No  greater  about the same  less  does not apply (please select why this does not apply)  child did not have a problem with attendance before starting services  child is too young to be in school  child was expelled from school  child is home schooled  child dropped out of school  Other: Please continue to answer questions on the next page. 70 2011 Consumer Survey Report Please feel free to use the space provided below to comment on any of your answers. Also, if there are areas which were not covered by this questionnaire which you feel should have been, please write them in the comments section. Thank you for your time and cooperation in completing this questionnaire. Please list any other comments you may want to share: ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ ____________________________________________________________________________________________ Thank you for completing this questionnaire. 71 2011 Consumer Survey Report ¡¡ESTA SECCIÓN DEBE SER COMPLETADA POR LA RBHA O EL PROVEEDOR!! Nombre de Agencia de Servicio __________________________________________ NOMBRE de la RBHA: __________________ TXIX  El Estado de Elegibilidad del Cliente: Encuesta tomada en : Casa  ID de agencia o proveedor: _____________ TXXI  Clínica  Por teléfono  ID de la encuesta: ___________________ Cliente inscrito en DDD: Si____ No_____ ENCUESTA PARA FAMILIAS DE JÓVENES QUE RECIBIERON SERVICIOS DURANTE EL AÑO 2011 Por favor ayúdenos a mejorar nuestro programa completando ésta encuesta sobre los servicios que usted ha recibido en los últimos seis meses. Estamos interesados en su opinión honesta. Todas las respuestas serán tratadas como confidenciales. Cualquier información personal será excluida en la presentación de los resultados de ésta encuesta. Los servicios actuales y/o futuros de su hijo(a) o la familia no se verán afectados si usted decide no participar en esta encuesta. Si usted ha tomado ya esta encuesta en los meses de abril o mayo de 2011, no tiene que completarla otra vez. Después de que usted haya finalizado, por favor doble la hoja y deposítela en la caja de encuestas, antes de usted salga de la oficina hoy. Gracias. Use pluma o lápiz Por favor rellene las burbujas completamente con sus respuestas a las preguntas siguientes: Información acerca de la persona que recibe los servicios: La Edad del Niño: El Sexo del Niño:  Masculino La Pertenencia étnica del Niño  Femenino  Hispano o Latino  No Hispano o Latino Por favor compruebe todas las categorías de raza aplicables: La Raza del Niño:  Blanco  Negro/Africano Americano  Asiático  Nativo Americano / Nativo de Alaska  Nativo de Hawai/Isleño del Pacífico ¿Por cuánto tiempo ha estado su hijo recibiendo servicios de salud mental y/o de abuso de sustancias? (De éste proveedor)  0 - 6 meses  7 - 11 meses  1 - 2 años  3 - 5 años  más de 5 años Por favor indique su relación con el niño: ¿Tiene su familia un Equipo del Niño y la Familia?  Padre/Guardián  Amigo  Pariente (Tío, Tía, Primo, Abuelo, etc.)  Otro  Sí  No Por favor siga contestando preguntas en la siguiente página. 72 2011 Consumer Survey Report POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE Muy De acuerdo De acuerdo Neutro En desacuerdo Muy en desacuerdo No Aplica Por favor ayude a nuestra agencia a mejorar los servicios, contestando algunas preguntas sobre los servicios que su niño recibido DURANTE LOS ÚLTIMOS 6 MESES. 1. En general, estoy satisfecho con los servicios que mi hijo recibió.       2. Ayudé elegir los servicios de mi hijo.       3. Ayudé elegir los objetivos de tratamiento de mi hijo.       4. La gente que ayuda a mi hijo nos apoyó sin falta.       5. Sentí que mi hijo tenía a alguien para dirigirse cuando estaba preocupado.       6. Participé en el tratamiento de mi hijo. 7. Los servicios que mi hijo y/o la familia recibieron fueron los apropiados para nosotros. 8. La ubicación de los servicios fue conveniente para nosotros. 9. Los servicios estuvieron disponibles en momentos en que fue conveniente para nosotros. 10. Mi familia recibió la ayuda que estábamos buscando para mi hijo.                               11. Mi familia recibió tanta ayuda como necesitábamos para mi hijo.       12. El personal me trató con respeto.       13. El personal respetó las creencias religiosas/espirituales de mi familia.       14. El personal habló conmigo de tal forma que entendí. 15. El personal fue respetuoso de mis raíces culturales/étnicas. (Por ejemplo: mis valores, tradiciones, creencias, raza, lenguaje, etc.).             Como resultado de los servicios que mi niño y/o la familia recibieron: 16. Mi hijo maneja de forma más eficaz los problemas diarios.       17. Mi hijo se lleva mejor con los otros miembros de la familia.       18. Mi hijo se lleva mejor con los amigos y otras personas.       19. A mi hijo le va mejor en la escuela y/o el trabajo.       20. Mi hijo está más dispuesto a enfrentar las situaciones difíciles.       21. Estoy satisfecho con nuestra vida familiar actualmente.       22. Mi hijo es más capaz de hacer las cosas que quiere hacer.       Como resultado de los servicios que mi niño y/o la familia recibieron: por favor responda basado en las relaciones con personas diferentes de su proveedor(es) de salud mental. 23. Conozco gente que escuchará y me entenderá cuando necesito       hablar. 24. Tengo personas con las que me siento cómodo hablando acerca del       problema de mi hijo.       25. En una crisis, yo tendría el apoyo que necesito de mi familia o amigos.    26. Cuento con personas con quienes puedo hacer cosas agradables. 27. ¿Cuál ha sido la cosa más provechosa sobre los servicios usted y su hijo recibieron durante los 6 meses 28. ¿Qué mejoraría los servicios aquí? Por favor siga contestando preguntas en la siguiente página. 73    pasados? 2011 Consumer Survey Report Neutro En desacuerdo Muy en desacuerdo No Aplica                Muy De acuerdo De acuerdo POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE Pregunta adicional del estado: 29. En la creación del plan de servicios de mi hijo, nuestras preferencias  culturales fueron incluidas. (Por ejemplo; mis valores, tradiciones, creencias, raza, lenguaje, etc.).  30. Los servicios fueron proporcionados en un lenguaje que mi hijo entendiera. 31. En la creación del plan de servicios de mi hijo nuestras tradiciones y creencias tribales fueron incluidas (Por ejemplo: se nos preguntó si  queríamos participar en sesiones de sanación tradicional). Si usted no participa de las tradiciones tribales, por favor marque la opción "No aplica". Por favor conteste las siguientes preguntas, para informarnos acerca de cómo va su hijo.  Sí  No 32. ¿Vive actualmente su hijo con usted? 33. ¿Ha vivido su hijo en cualquiera de los siguientes sitios en los últimos 6 meses? (MARQUE TODAS LAS QUE APLICAN)  Con uno o ambos padres  Con otro miembro de familia  Familia adoptiva  Familia adoptiva terapéutica  Refugio de crisis  Refugio para los sin hogar  Casa de Grupo  Centro de tratamiento  Hospital  Cárcel local o instalación de detención  Otro (describa)  Instalación correccional  Escapó/o vive en las calles 34. ¿Durante el año pasado, visitó su hijo un médico (o enfermera) para un control de salud o porque él/ella estaba enfermo? (Elija una de las opciones)  Sí, en una clínica o oficina   Sí, pero sólo en un cuarto de emergencia de hospital No  No recuerdo 35. ¿Está su hijo tomando medicamentos para problemas de conducta / emocionales? 36. ¿Si la respuesta es afirmativa, responda si el médico o la enfermera le explicaron con qué clase de efectos secundarios debe estar alerta? 37. ¿Todavía está su hijo recibiendo servicios de éste proveedor? 38. ¿Por cuánto tiempo recibió su hijo servicios de éste proveedor? 39. ¿Ha sido su hijo arrestado desde que comenzó a recibir servicios de salud mental? 40. ¿Estuvo arrestado su hijo durante los 12 meses anteriores a ésto? 41. Desde que su hijo comenzó a recibir servicios de salud mental, los incidentes con la policía...  Sí  No  Sí  No  Sí  No  Menos de 1 mes  1-5 Meses  6 Meses a 1 año  Más de 1 año (vaya a la pregunta 45)  Sí  No  Sí  No  han disminuido (por ejemplo, no ha sido arrestado, molestado por la policía, llevado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable (no tuvo ningún incidente con la policía éste año o el año pasado) 42. ¿Ha sido su hijo expulsado o suspendido de su escuela desde comenzó a recibir los servicios? 74 que  Sí  No 2011 Consumer Survey Report 43. ¿Fue su hijo expulsado o suspendido de su escuela durante meses anteriores a ésto? los 12  Sí Por favor siga contestando preguntas en la siguiente página. 75  No 2011 Consumer Survey Report Por favor siga contestando preguntas en la siguiente página. POR FAVOR MARQUE SUS RESPUESTAS RELLENANDO LAS BURBUJAS COMPLETAMENTE 44. Desde que comenzó a recibir servicios, el número de días que mi hijo estuvo en la escuela fue…  mayor  el mismo  menos  no se aplica (por favor seleccione por qué éste no se aplica)  el niño no tenía un problema con la asistencia antes de los servicios iniciales  el niño es demasiado joven para estar en la escuela  el niño fue expulsado de la escuela  el niño está siendo enseñado en la casa  el niño abandonó la escuela 45. ¿Ha sido su hijo arrestado durante los 12 meses pasados? 46. ¿Estuvo arrestado su hijo durante los 12 meses antes a ésto? 47. Durante el año pasado, los incidentes de su niño con la policía…  Otro  Sí  No  Sí  No  han disminuido (por ejemplo, no ha sido arrestado, molestado por la policía, llevado por la policía a un refugio o programa de crisis)  han sido igual  han aumentado  no aplicable ( no tuvo ningún encuentro con policía éste año o el año pasado) 48. ¿Fue expulsado su niño o suspendido de la escuela durante los 12 meses pasados? 49. ¿Fue expulsado su niño o suspendido de la escuela durante los 12 meses antes de esto? 50. Durante el año pasado, el número de días mi niño fue a la escuela era…  Sí  No  Sí  No  mayor  el mismo  menos  no se aplica (por favor seleccione por qué éste no se aplica)  El niño no tenía un problema con la asistencia antes de servicios iniciales  El niño es demasiado joven para estar en la escuela  El niño fue expulsado de la escuela  El niño está siendo enseñado en la casa  El niño abandonó la escuela  Otro Por favor siga contestando preguntas en la siguiente página. 76 2011 Consumer Survey Report Por favor siéntase libre de usar el espacio proporcionado abajo para comentar sobre cualquiera de sus respuestas. También, si hay áreas que no fueron cubiertas por éste cuestionario que usted siente deberían haber sido incluidas, por favor escríbalas en la sección de comentarios. Gracias por su tiempo y cooperación en completar éste cuestionario. Por favor escriba cualquier otro comentario que usted quiera compartir: Gracias por completar éste cuestionario. 77