Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 State of Arizona Department of Education Office of John Huppenthal Superintendent of Public Instruction Arizona FFY 2011 Annual Performance Report for Special Education Highly Effective Schools Division Exceptional Student Services 1535 West Jefferson, Phoenix, AZ 85007 http://www.azed.gov/ February 15, 2013 Revised May 17, 2013 Part B Arizona Annual Performance Report for FFY 2011 1 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Arizona FFY 2011 Annual Performance Report for Special Education Submitted to the Office of Special Education Programs United States Department of Education Highly Effective Schools Division Exceptional Student Services Angela Denning Deputy Associate Superintendent 602-542-4013 essdesk@azed.gov http://www.azed.gov/special-education/ February 15, 2013 Revised May 17, 2013 Part B Arizona Annual Performance Report for FFY 2011 2 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Table of Contents Overview of the Annual Performance Report Development ......................................................................... 4 Indicator 1: Graduation Rates ....................................................................................................................... 6 Indicator 2: Dropout Rates .......................................................................................................................... 18 Indicator 3: Assessments ............................................................................................................................ 29 Indicator 4A: Suspension and Expulsion .................................................................................................... 43 Indicator 4B: Suspension and Expulsion by Race or Ethnicity ................................................................... 48 Indicator 5: School Age LRE ....................................................................................................................... 54 Indicator 6: Preschool LRE ......................................................................................................................... 60 Indicator 7: Preschool Outcomes ................................................................................................................ 63 Indicator 8: Parent Involvement .................................................................................................................. 69 Indicator 9: Racial / Ethnic Disproportionality ............................................................................................. 74 Indicator 10: Racial / Ethnic Disproportionality by Disability ....................................................................... 81 Indicator 11: Evaluation Timelines .............................................................................................................. 89 Indicator 12: Early Childhood Transition ..................................................................................................... 97 Indicator 13: Secondary Transition ........................................................................................................... 105 Indicator 14: Post School Outcomes ......................................................................................................... 119 Indicator 15: Effective General Supervision .............................................................................................. 137 Indicator 18: Resolution Session Effectiveness ........................................................................................ 157 Indicator 19: Mediation Agreements ......................................................................................................... 160 Indicator 20: State Reported Data ............................................................................................................ 163 Attachments .............................................................................................................................................. 169 Part B Arizona Annual Performance Report for FFY 2011 3 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The Part B Arizona Annual Performance Report for Special Education Federal Fiscal Year 2011 Overview of the Annual Performance Report Development In accordance with 20 U.S.C. § 1416 (b)(2)(C)(ii) and 34 CFR § 300.602, the State of Arizona must report annually to the United States Secretary of Education on Arizona’s performance under its Part B State Performance Plan (SPP). The annual report is the Part B Annual Performance Report (APR). The submission of the Part B APR, due February 15, 2013, reflects those requirements and the State’s progress toward the goals established in the State Performance Plan submitted to the United States Department of Education (USDOE) in December 2005. The February 15, 2013, APR gives actual target data and other responsive information for Indicators 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, and 19. The Annual Performance Report was developed by the staff at the Arizona Department of Education/Exceptional Student Services (ADE/ESS) and the Arizona Department of Education/Early Childhood Special Education (ADE/ECSE). A number of Arizona Department of Education staff members with specialization in different areas examined improvement activities, collected and analyzed the data, and drafted the reports for the 17 indicators. Members of the Special Education Advisory Panel (SEAP) and education personnel from the field reviewed data, annual targets, and improvement activities and offered suggestions. Descriptions of the data, including sources, sampling methodology, and validity and reliability, are located under each indicator. Information is included that replies to the Arizona Part B FFY 2010 SPP/APR Response Table from the Office of Special Education Programs (OSEP). All improvement activities were reviewed during FFY 2011, which led to the revision of some of the activities. Revisions for FFY 2011 were made to Arizona’s FFY 2005–2012 State Performance Plan for Special Education. The document is available on the ADE/ESS Web site at http://www.azed.gov/specialeducation/resources/spp-apr/ under the list titled State Performance Plan. Stakeholder Involvement As data and other communications became available after the close of the 2011–2012 school year, the ADE/ESS staff reported to the Special Education Advisory Panel (SEAP). The SEAP members represent a broad group of stakeholders throughout Arizona. Groups represented on the panel include parents of children with disabilities, individuals with disabilities, teachers, early childhood education, charter schools, school districts, institutions of higher education that prepare special education and related services personnel, secure care facilities, and public agencies. The ADE/ESS responded to questions and comments from the SEAP members and considered the panel’s advice. In addition to reporting on the APR to the SEAP, ESS requested input from special education administrators through meetings of the regional organizations, small workshops, and large conferences. The ADE/ESS data management coordinator trained data managers and administrators on the data requirements and also requested input for improving the State’s data collection and reporting process. ESS program specialists spoke to administrators and teachers specifically about the 0% and 100% compliance indicators during on-site visits, seeking information for the revision of improvement activities to increase compliance. Public Reporting and Dissemination Part B Arizona Annual Performance Report for FFY 2011 4 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Arizona must report annually to the public on: (1) the State’s progress and/or slippage in meeting the measurable and rigorous targets in the SPP and, (2) the performance of each public education agency (PEA) in the State on the SPP targets. The annual performance report (APR) on the State’s progress and/or slippage is available on the ADE/ESS Web site at http://www.azed.gov/special-education/resources/spp-apr/ under the list titled Annual Performance Report, beginning on February 15, 2013. The title of the APR is Arizona FFY 2011 Annual Performance Report. The revised State Performance Plan (SPP) for FFY 2011 is available on the ADE/ESS Web site at http://www.azed.gov/special-education/resources/spp-apr/ under the list titled State Performance Plan, beginning on February 15, 2013. The title of the SPP is Arizona State Performance Plan FFY 2005–2012 Revised FFY 2011. The annual public reports will be available on the ADE/ESS Web site at http://www.azed.gov/specialeducation/resources/ under the list titled Public Reports School Year 2011–2012, within 120 days of the February 15, 2013, submission of the APR. These reports list the performance of each school district and charter school in Arizona on the SPP targets. The SPP and APR are disseminated to the public by hard copy, e-mail, and the ADE/ESS Web site. Each member of SEAP receives a copy of the revised SPP and the APR, as does Arizona’s Parent and Training Information Center. The ESS special education listserv, Parent Information Network specialists (PINS), ESS and ECSE specialists, trainings, and conferences serve as the vehicles to notify parents, the PEAs, and the public of the availability of the SPP and APR. Special Education Monitoring Alerts, memoranda pertaining to specific topics including the SPP/APR, are sent to the field electronically on the ESS listserv and distributed by hard copy through the ESS and PIN specialists. Part B Arizona Annual Performance Report for FFY 2011 5 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 1: Graduation Rates Percent of youth with IEPs graduating from high school with a regular diploma. (20 U.S.C. 1416 (a)(3)(A)) Measurement: States must report using the adjusted cohort graduation rate required under the ESEA. Target Data for FFY 2011 FFY Measurable and Rigorous Target 2011 80% Arizona’s single, statewide target graduation rate is 80%. Actual Target Data for FFY 2011 number of youth with IEPs who graduated in 4 years with a regular high school diploma number of youth with IEPs who entered high school 4 years earlier (adjusting for transfers, home schooled, and deceased youth) 4,867 7,270 Actual Target Data for FFY 2011 67% 4,867  7,270 = 0.6694  100 = 67% Arizona did not meet the target. Data Data Source The graduation data from Arizona’s 2011 cohort were reported by the public education agencies (PEAs) through the Student Accountability Information System (SAIS), a Web-based system for reporting all student-level details to the Arizona Department of Education. Data Description The graduation data were analyzed by the Arizona Department of Education’s Accountability Division/Research and Evaluation Section (ADE/R and E) and the Information Technology Division (IT). The same graduation rate calculation was used, and it is the same data as that reported to the U.S. Department of Education under Title I of the Elementary and Secondary Education Act (ESEA). Part B Arizona Annual Performance Report for FFY 2011 6 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Target Data The target data are the same as the annual graduation rate targets under Title I of the ESEA and explained in Arizona’s Accountability Workbook. Arizona’s single, statewide target graduation rate is 80%. Valid and Reliable Data The graduation data were obtained from the ADE Accountability Division/Research and Evaluation Section (ADE/R and E), which follows internal processes to ensure valid, reliable, and accurate data. Conditions to Graduate Graduation Cohort Arizona uses a four-year cohort. Any student who receives a traditional high school diploma within the first four years of starting high school is considered a four-year graduate. A four-year rate is calculated by dividing the sum of all four-year graduates in a cohort by the sum of those who should have graduated and did not transfer to another qualified educational facility, left to be home schooled, or are deceased. Students who receive a diploma prior to September 1 of the following school year following their fourth year are included as part of the graduation cohort. Conditions to Graduate with Regular Diploma Conditions students without disabilities must meet in order to graduate with a regular high school diploma:  Complete the PEA’s requirements to receive a regular high school diploma (Arizona Revised Statutes § 15-701.01 (C) and Arizona Administrative Code R7-2-302); and  Achieve passing scores on the Arizona’s Instrument to Measure Standards (Arizona Revised Statutes § 15-701.01 (A)). Conditions students with disabilities must meet in order to graduate with a regular high school diploma:  The local governing board of each school district is responsible for developing a course of study and graduation requirements for all students placed in special education programs (Arizona Administrative Code R7-2-302 (6)).  Students with disabilities do not have to achieve passing scores on the Arizona’s Instrument to Measure Standards (AIMS) or Arizona’s Instrument to Measure Standards Alternate (AIMS A) to graduate with a regular high school diploma unless specifically required by the Individualized Education Program (IEP) team (Arizona Revised Statutes (ARS) § 15-701.01 (B)). Explanation of Progress That Occurred for FFY 2011 Arizona’s graduation target (80%) for FFY 2010 is the same as the annual graduation rate target under Title 1 of the Elementary and Secondary Education Act (ESEA). The State did not meet this target but made a gain from FFY 2010 (results rose from 65.8% to 67%). This reflects steady improvement and a cumulative increase over the past six years of 6.6%. Arizona maintains that quality transition planning for students from secondary to postsecondary education, training, and employment settings has positively affected graduation rates for students with disabilities. To this end, the ADE/ESS transition specialists provided training and technical assistance to 1,089 participants at 75 training sessions offered throughout the State; established 13 new teams through Part B Arizona Annual Performance Report for FFY 2011 7 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 the Secondary Transition Mentoring Project (STMP); held a statewide conference dedicated to transition; and developed and disseminated information and materials through webinars and updates to the ADE/ESS secondary transition Web site. The ADE/ESS is committed to offering intensive capacity building grant opportunities for PEAs in need of assistance in transition planning and service provision. In FFY 2011, the Secondary Transition Mentoring Project (STMP), a two-year capacity building grant opportunity, provided funding to 26 PEAs, including 13 new teams (2011–2013 cohort) and 13 returning teams (2010–2012 cohort). The ADE/ESS, in collaboration with training facilitators from the University of Kansas Transition Coalition, developed and provided a training series that included an introductory session and webinars, three two-day face-to-face trainings, an online short course provided by the University of Kansas, and ongoing support and technical assistance. This capacity building training focused on developing strategies and products for use in participating PEAs to ensure 100% compliance on Indicator 13, using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. As part of the FFY 2011 STMP grant, teams reviewed their own PEA’s data in the following areas: graduation rates, dropout rates, post school outcomes data (if available), and Indicator 13 compliance. Throughout the two-year process, the STMP teams focused on identifying PEA needs and priorities and developed action plans to address the identified needs. If applicable, teams developed action plans for increasing graduation rates or reducing dropout rates for students with disabilities that included root cause analyses to determine potential barriers within their schools. Additionally, the STMP team trainings assisted local PEAs in creating sustainable community teams to engage community partners and facilitate positive post school outcomes for students with disabilities. A review of the action plans by University of Kansas/Transition Coalition coaches and ADE/ESS personnel indicated a 100% completion rate on activities outlined in team plans. Additionally, review of training evaluation forms showed that participating PEAs felt more prepared to plan and implement transition services leading to improved post school outcomes. Arizona’s Eleventh Annual Transition Conference offered two breakout sessions specifically targeting improving graduation rates and decreasing dropout rates of students with disabilities. Sessions were led by a nationally recognized expert from the National High School Center. Resources and facilitated discussions aimed at addressing graduation and dropout issues were provided. The ADE/ESS secondary transition Web site also was updated with additional resources and materials on dropout prevention. In addition to PEA training opportunities, internal and external stakeholder collaboration efforts were conducted to enhance the secondary transition planning process. During FFY 2011, four intradepartmental collaboration meetings were held with the ADE sections of Dropout Prevention, Innovative Practices, and Career and Technical Education (including School/Career Counseling). The goal of these discussions was to improve collaboration between key ADE personnel and to assist with preparation and progress toward successful postsecondary outcomes for students with disabilities currently enrolled in secondary education. Further, ADE/ESS transition and program specialists worked closely with PEAs to assist in data review, analysis, and training. Transition specialists analyzed data collected from on-site PEA annual site visits (technical assistance visits conducted by ESS program specialists) and targeted staff development to those PEAs most in need of training with regard to secondary transition requirements. ESS program specialists also reviewed graduation rates with PEAs on an annual basis as part of a larger data review that included performance and compliance Indicator data, dispute resolution data, and annual technical assistance visit information. If a PEA had not met the State target for graduation rate, then the PEA may have been required to conduct a drill down exercise that is designed to explore root causes. Discussion of Improvement Activities Completed for FFY 2011 Part B Arizona Annual Performance Report for FFY 2011 8 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Primary Activity (GOAL) 1) Revise, implement, and evaluate a comprehensive plan for training PEAs to increase compliance with postsecondary requirements related to Indicator 13 Sub-Activities (Objectives or Action Steps) a) On an annual basis, identify PEAs in Years 2 and 3 of the monitoring cycle through collaboration with ESS specialists b) On an annual basis, review, revise (if necessary), and implement the comprehensive training plan, emphasizing the eight required components of Indicator 13 Timeline Complete Activity completed 8/31/11. The FFY 2011 PEA list was completed August 2011. The Annual Site Visit Log (ASVL) from school year 2010-2011 was used to identify PEAs most in need of training and technical assistance (TA) for Indicator 13. Activities completed from 7/1/11 to 6/30/12. Projected 7/1/11– 6/30/13 Resources (Planned) ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists The FFY 2011 review and revision of the Strategic Plan for Statewide Transition Planning was completed July 2011. Implementation of the Transition Strategic Plan was completed from July 2011 to June 2012. The FFY 2011 Transition Strategic Plan includes seven main components: 1. Provide training to targeted PEAs and in response to requests from non-targeted PEAs on Indicator 13; 2. Organize Arizona’s Eleventh Annual Transition Conference focusing on improving post school outcomes for students with disabilities by providing sessions on transition planning and dropout prevention; 3. Provide training to special education directors from across the State at the annual ADE/ESS Directors Institute; 4. Provide capacity building grants to PEAs to facilitate intra/interagency collaboration and build local capacity to improve post school outcomes through local interagency work, as well as provide intensive training and support for PEAs to achieve Part B Arizona Annual Performance Report for FFY 2011 9 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 100% compliance on Indicator 13; 5. Collaborate with national technical assistance centers and organizations including NSTTAC, NPSO, NDPC-SD, and the NASDSE IDEA Partnership Community of Practice on Transition and Transition Coalition; 6. Participate with other Arizona State agencies including Rehabilitation Services of Arizona/Vocational Rehabilitation (RSA/VR), Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN); 7. Collaborate with other ADE sections: High School Renewal and Improvement (AZHSRI), Career and Technical Education (CTE), Dropout Prevention, and School Counselors and ADE/ESS areas: Program Support, Assistive Technology, and Parent Information Network. All components of Arizona’s FFY 2011 Strategic Plan for Statewide Transition Planning were immediately implemented upon revision, from 8/1/11 to 6/30/12. Activities completed:  1,089 participants from 130 targeted and non-targeted PEAs received secondary transition training, including Indicator 13 trainings from ADE/ESS transition specialists at 75 sites statewide. Trainings were delivered in regional or direct school locations.  Arizona’s Eleventh Annual Transition Conference was held in October 2011 and offered sessions focused on improving Part B Arizona Annual Performance Report for FFY 2011 10 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 compliance with the eight components of Indicator 13. 829 participants attended the conference, including education and agency professionals, youth, young adults, family members of youth with disabilities, and vendors/exhibitors.  Four sessions on secondary transition were provided by ADE/ESS transition specialists at the annual ADE/ESS Directors Institute (DI). Of the 660 in attendance at the DI, 78 participants attended sessions related to secondary transition.  26 PEAs participated in Year 1 (14 PEAs in the 2010–2012 cohort) or Year 2 (13 PEAs in the 2010–2012 cohort) team trainings of the Secondary Transition Mentoring Project (STMP) capacity building grant. Through a contract with the University of Kansas Transition Coalition and in collaboration with ADE/ESS, training was provided to achieve and sustain 100% compliance on Indicator 13 using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. Team training included: three two-day face-toface training sessions, webinars, a Web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and attendance at Arizona’s Eleventh Annual Transition Conference.  Collaboration with national technical assistance centers and organizations occurred throughout the year and included: participation in NPSO and NSTTAC Community of Practice Part B Arizona Annual Performance Report for FFY 2011 11 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 calls; use of resources from the NSTTAC Web site; participation in the NASDSE IDEA Partnership, Community of Practice National Meeting; and attendance at the National Secondary Transition Planning Institute where OSEP, NPSO, NSTTAC, and NDPC-SD provided guidance. Additionally, ADE/ESS maintained ongoing collaboration with the University of Kansas Transition Coalition and the Mountain Plains Regional Resource Center.  At the State level, ADE/ESS collaborated with RSA/VR, Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN), and met every other month with these agency stakeholders through the Arizona Community of Practice on Transition (AZCoPT). In FFY 2011, AZCoPT revised a presentation used statewide through RSA/VR and DBHS teleconferencing media to introduce participants to the supports/services available to school-aged and adult individuals with disabilities.  ADE/ESS collaboration meetings with the ADE sections of High School Renewal and Improvement, Career and Technical Education, Dropout Prevention, and School Counselors were scheduled and held approximately every three months and resulted in crosstraining for conferences sponsored by each ADE section on the topic of secondary transition.  Intra-ADE/ESS collaborative efforts included: monthly meetings with PINS (Parent Information Network specialists) as fellow Part B Arizona Annual Performance Report for FFY 2011 12 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 c) On an annual basis, create and disseminate information through a variety of sources: annual statewide conference, monitoring alerts, Web site, and listserv announcements AZCoPT members, as well as involvement with PINS during quarterly Transition Conference Planning Committee meetings; at least quarterly meetings with ESS Program Support to discuss the use of the Annual Site Visit Log (ASVL); review/revision of the secondary transition section of the monitoring manual and needed secondary transition trainings for ESS program specialists and PEAs; and the development of collaborative presentations with the ADE/ESS assistive technology unit. Activities completed from 8/1/11 through 6/30/12. 7/1/11– 6/30/13 ADE/ESS Transition Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists  Four secondary transition presentations were offered at the ADE/ESS Directors Institute in August 2011 for approximately 78 participants.  Ninety-six sessions either directly or indirectly related to Indicator 1 were offered at Arizona’s Eleventh Annual Transition Conference held in October 2011.  Between July 2011 and June 2012, the ADE/ESS Secondary Transition Web site was updated to include additional and/or revised Indicator 13 materials from NSTTAC, IDEA Partnership, and other secondary transition technical assistance centers. The Web address is http://www.azed.gov/specialeducation/specialprojects/secondary-transition/. Activities completed from 7/1/11 to 6/30/12. d) On an annual basis, analyze preand post-training data collected  Analysis of ASVL pre-training through the Annual data was completed in January Site Visit Log (ASVL) 2012 after all ESS program for each PEA to specialists were able to complete determine level of at least one PEA annual site visit compliance on all during fall 2011. The analysis eight required showed a 76.5% average for components of compliance with the eight items of ADE/ESS Program Specialists Part B Arizona Annual Performance Report for FFY 2011 13 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator 13 Indicator 13.  Post-training data analysis of all PEAs that received a pre-training annual site visit and were trained in secondary transition during FFY 2011 showed a 79.1% average for compliance with the eight items of Indicator 13. 2) Provide a twoyear capacity building grant to PEAs to participate in the Secondary Transition Mentoring Project (STMP) Team Training a) On an annual basis, identify PEAs who meet eligibility requirements and extend invitations to them to participate in STMP trainings b) On an annual basis, provide indepth and ongoing professional development on transition requirements and best practices Through pre- and post-training analysis, an increase of 2.6% in compliance for Indicator 13 was demonstrated after training and technical assistance was provided to PEAs by transition and education program specialists. Activities completed from 1/1/11 to 7/30/12. 7/1/11– 6/30/13  Using Annual Site Visit (ASV) data for PEAs currently in Years 1, 2, or 3 of the monitoring cycle, 23 PEAs achieving less than 100% compliance on Indicator 13 were invited to apply for a noncompetitive Secondary Transition Mentoring Project (STMP) capacity building grant.  Fourteen PEAs were accepted for participation in Year 1 of the STMP Team training in July 2011. Activities completed from 7/1/11 through 6/30/12. ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists  STMP participants attended Arizona’s Eleventh Annual Transition Conference, which included a STMP team orientation and designated sessions.  Training materials and activities were designed, created, and disseminated by ADE/ESS in collaboration with the University of Kansas Transition Coalition. Trainings focused on: identifying PEA barriers to meeting transition requirements; developing an action plan to eliminate barriers; creating IEPs that meet transition requirements; implementing training to build intra-PEA Part B Arizona Annual Performance Report for FFY 2011 14 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 capacity to attain 100% compliance on secondary transition requirements; and determining improvements made and targeting areas still in need of improvement.  Team training for each cohort consisted of three two-day faceto-face trainings throughout the year. Additionally, Year 1 teams completed an online short course specially designed for STMP teams and focused on best practices in transition planning. Activity completed from 7/1/11 through 6/30/12. c) On an annual basis, analyze preand post-training data collected during  Measures to determine STMP trainings for effectiveness of STMP training each PEA that included the use of: participated to a) team process checklist; determine level of b) interrater reliability measures compliance on all for IEP file review; eight required c) training session evaluations; components of d) STMP team self-reported Indicator 13 progress toward completion of action plans; and e) feedback provided by ADE/ESS monitoring program specialists on compliance. 7/1/11– 6/30/13 ADE/ESS Transition Specialists Data from the measures are as follows: a) April 2012 trainings for both STMP cohorts included the completion of a 37-item “team process” checklist. Results from the checklist use indicated both Year 1 and Year 2 teams reported an average of 92% for the item “all team members are highly knowledgeable of transition compliance requirements” and an average of 69% for the item “STMP team collaboratively develops and implements an action plan that addresses the prioritized needs.” b) Training for both cohorts included instruction and practice activities using interrater reliability measures. Part B Arizona Annual Performance Report for FFY 2011 15 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 c) All STMP training sessions during FFY 2011 included instruction, activities, and Indicator 13 file reviews. Based on a 5-point Likert scale (1 lowest and 5 highest), evaluation results indicated an average score of 4.4 across all activities for both cohorts. Additionally, all team members completed a pre-/postevaluation of their competency in meeting Indicator 13 requirements that asked the question, “How prepared do you feel?” Based on a 5-point Likert Scale (1 equaled unprepared and 5 equaled prepared) and using a Paired Samples T-Test, all scores indicated a statistically significant increase in knowledge from the beginning to the end of the training for FFY 2011 for both cohorts (Years 1 and 2). The areas reporting the greatest increase in knowledge were transition assessments and documenting transition services and course of study that will support the student in reaching postsecondary goals. d) Review of each STMP team’s action plan indicated all teams set goals to build and sustain systems to ensure 100% compliance on Indicator 13 school- or district-wide. All teams reported progress in meeting these goals. e) During the April 2012 training, Year 2 teams met with ADE/ESS program specialists to receive informal feedback on IEPs written during the second year of the STMP training program. Feedback indicated significant improvement in PEA knowledge and compliant practices. Part B Arizona Annual Performance Report for FFY 2011 16 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response In the FFY 2011 APR, the State must use the same data it used for reporting to the Department under Title I of the ESEA, using the adjusted cohort graduation rate required under the ESEA. In the FFY 2011 APR, Arizona used the same data it used for reporting to the Department under Title I of the ESEA, using the adjusted cohort graduation rate required under the ESEA. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 17 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 2: Dropout Rates Percent of youth with IEPs dropping out of high school. (20 U.S.C. 1416 (a)(3)(A)) Measurement: States must report using the dropout data used in the ESEA graduation rate calculation and follow the timeline established by the Department under the ESEA. Target Data for FFY 2011 FFY Measurable and Rigorous Target 2011 ≤ 5.0% Actual Target Data for FFY 2011 (using 2010–2011 data) number of youths with IEPs dropping out of grades 9 – 12 number of youths with IEPs in grades 9 – 12 1,741 36,926 Actual Target Data for FFY 2011 4.7% 1,741  36,926 = 0.0471  100 = 4.7% Arizona met the target. Data Data Source Arizona used the same data source and measurement that the State used for its FFY 2010 APR that was submitted on February 1, 2012. The dropout data were reported by the public education agencies (PEAs) through the Student Accountability Information System (SAIS), a Web-based system for reporting all student-level details to the Arizona Department of Education. Data Description The 2010–2011 data were analyzed by the Arizona Department of Education’s Accountability Division/Research and Evaluation Section (ADE/R and E). Valid and Reliable Data Part B Arizona Annual Performance Report for FFY 2011 18 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The dropout data were obtained from the ADE Accountability Division/Research and Evaluation Section (ADE/R and E), which follows internal processes to ensure valid, reliable, and accurate data. Definition of Dropout and Methodology For purposes of calculating and reporting a dropout rate, Arizona used the annual event school dropout rate for students leaving a school in a single year determined in accordance with the National Center for Education Statistics (NCES) Common Core of Data. Consistent with this requirement, Arizona used the NCES definition of high school dropout, defined as an individual who: (1) was enrolled in school at some time during the previous school year; and (2) was not enrolled at the beginning of the current school year; and (3) has not graduated from high school or completed a State- or district-approved educational program; and (4) does not meet any of the following exclusionary conditions: (a) transfer to another public school district, private school, or State- or districtapproved educational program (including correctional or health facility programs); (b) temporary absence due to suspension or school-excused illness; or (c) death. The same definition and methodology for dropout rates apply to all students in Arizona. Explanation of Progress That Occurred for FFY 2011 Arizona met the FFY 2011 target of 5.0% and had the same results as FFY 2010 (4.7%). Arizona maintains that quality transition planning for students from secondary to postsecondary education, training, and employment settings can positively affected dropout rates for students with disabilities. To this end, the ADE/ESS transition specialists provided training and technical assistance to 1,089 participants at 75 training sessions offered throughout the State; established 13 new teams through the Secondary Transition Mentoring Project (STMP); held a statewide conference dedicated to transition; and developed and disseminated information and materials through webinars and updates to the ADE/ESS secondary transition Web site. The ADE/ESS is committed to offering intensive capacity building grant opportunities for PEAs in need of assistance in transition planning and service provision. In FFY 2011, the Secondary Transition Mentoring Project (STMP), a two-year capacity building grant opportunity, provided funding to 26 PEAs, including 13 new teams (2011–2013 cohort) and 13 returning teams (2010–2012 cohort). The ADE/ESS, in collaboration with training facilitators from the University of Kansas Transition Coalition, developed and provided a training series that included an introductory session and webinars, three two-day face-to-face trainings, an online short course provided by the University of Kansas, and ongoing support and technical assistance. This capacity building training focused on developing strategies and products for use in participating PEAs to ensure 100% compliance on Indicator 13, using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. As part of the FFY 2011 STMP grant, teams reviewed their own PEA’s data in the following areas: graduation rates, dropout rates, post school outcomes data (if available), and Indicator 13 compliance. Throughout the two-year process, the STMP teams focused on identifying PEA needs and priorities and developed action plans to address the identified needs. If applicable, teams developed action plans for increasing graduation rates or reducing dropout rates for students with disabilities that included root cause analyses to determine potential barriers within their schools. Additionally, the STMP team trainings assisted local PEAs in creating sustainable community teams to engage community partners and facilitate positive post school outcomes for students with disabilities. A review of the action plans by University of Kansas/Transition Coalition coaches and ADE/ESS personnel indicated a 100% completion rate on activities outlined in team plans. Additionally, review of training evaluation forms showed that Part B Arizona Annual Performance Report for FFY 2011 19 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 participating PEAs felt more prepared to plan and implement transition services leading to improved post school outcomes. Arizona’s Eleventh Annual Transition Conference offered two breakout sessions specifically targeting improving graduation rates and decreasing dropout rates of students with disabilities. Sessions were led by a nationally recognized expert from the National High School Center. Resources and facilitated discussions aimed at addressing graduation and dropout issues were provided. The ADE/ESS secondary transition Web site was also updated with additional resources and materials on dropout prevention. In addition to PEA training opportunities, internal and external stakeholder collaboration efforts were conducted to enhance the secondary transition planning process. During FFY 2011, four intradepartmental collaboration meetings were held with the ADE sections of Dropout Prevention, Innovative Practices, and Career and Technical Education (including School/Career Counseling). The goal of these discussions was to improve collaboration between key ADE personnel and to assist with preparation and progress toward successful postsecondary outcomes for students with disabilities currently enrolled in secondary education. Further, ADE/ESS transition and program specialists worked closely with PEAs to assist in data review, analysis, and training. Transition specialists analyzed data collected from on-site PEA annual site visits (technical assistance visits conducted by ESS program specialists) and targeted staff development to those PEAs most in need of training with regard to secondary transition requirements. ESS program specialists also reviewed dropout rates with PEAs on an annual basis as part of a larger data review that included performance and compliance Indicator data, PEA determinations, dispute resolution data, and annual technical assistance visit information. If a PEA had not met the State target for dropout rate, then the PEA may have been required to conduct a drill down exercise that is designed to explore root causes. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Revise, implement, and evaluate a comprehensive plan for training PEAs to increase compliance with postsecondary requirements related to Indicator 13 Sub-Activities (Objectives or Action Steps) a) On an annual basis, identify PEAs in Years 2 and 3 of the monitoring cycle through collaboration with ESS specialists b) On an annual basis, review, revise (if necessary), and implement the comprehensive training plan, emphasizing the eight required components of Indicator 13 Timeline Complete Activity completed 8/31/11. The FFY 2011 PEA list was completed August 2011. The Annual Site Visit Log (ASVL) from school year 2010-2011 was used to identify PEAs most in need of training and technical assistance (TA) for Indicator 13. Activities completed from 7/1/11 to 6/30/12. Projected 7/1/11– 6/30/13 Resources (Planned) ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists The FFY 2011 review and revision of the Strategic Plan for Statewide Transition Planning was completed July 2011. Implementation of the Transition Strategic Plan was completed from July 2011 to June 2012. The FFY 2011 Transition Strategic Plan includes seven Part B Arizona Annual Performance Report for FFY 2011 20 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 main components: 1. Provide training to targeted PEAs and in response to requests from non-targeted PEAs on Indicator 13; 2. Organize Arizona’s Eleventh Annual Transition Conference focusing on improving post school outcomes for students with disabilities by providing sessions on transition planning and dropout prevention; 3. Provide training to special education directors from across the State at the annual ADE/ESS Directors Institute; 4. Provide capacity building grants to PEAs to facilitate intra/interagency collaboration and build local capacity to improve post school outcomes through local interagency work, as well as provide intensive training and support for PEAs to achieve 100% compliance on Indicator 13; 5. Collaborate with national technical assistance centers and organizations including NSTTAC, NPSO, NDPC-SD, and the NASDSE IDEA Partnership Community of Practice on Transition and Transition Coalition; 6. Participate with other Arizona State agencies including Rehabilitation Services of Arizona/Vocational Rehabilitation (RSA/VR), Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN); 7. Collaborate with other ADE sections: High School Renewal and Improvement (AZHSRI), Career and Technical Education Part B Arizona Annual Performance Report for FFY 2011 21 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 (CTE), Dropout Prevention, and School Counselors and ADE/ESS areas: Program Support, Assistive Technology, and Parent Information Network. All components of Arizona’s FFY 2011 Strategic Plan for Statewide Transition Planning were immediately implemented upon revision, from 8/1/11 to 6/30/12. Activities completed:  1,089 participants from 130 targeted and non-targeted PEAs received secondary transition training, including Indicator 13 trainings from ADE/ESS transition specialists at 75 sites statewide. Trainings were delivered in regional or direct school locations.  Arizona’s Eleventh Annual Transition Conference was held in October 2011 and offered sessions focused on improving compliance with the eight components of Indicator 13. 829 participants attended the conference, including education and agency professionals, youth, young adults, family members of youth with disabilities, and vendors/exhibitors.  Four sessions on secondary transition were provided by ADE/ESS transition specialists at the annual ADE/ESS Directors Institute (DI). Of the 660 in attendance at the DI, 78 participants attended sessions related to secondary transition.  26 PEAs participated in Year 1 (14 PEAs in the 2010–2012 cohort) or Year 2 (13 PEAs in the 2010–2012 cohort) team trainings of the Secondary Transition Mentoring Project (STMP) capacity building grant. Through a contract with the University of Part B Arizona Annual Performance Report for FFY 2011 22 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Kansas Transition Coalition and in collaboration with ADE/ESS, training was provided to achieve and sustain 100% compliance on Indicator 13 using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. Team training included: three two-day face-toface training sessions, webinars, a Web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and attendance at Arizona’s Eleventh Annual Transition Conference.  Collaboration with national technical assistance centers and organizations occurred throughout the year and included: participation in NPSO and NSTTAC Community of Practice calls; use of resources from the NSTTAC Web site; participation in the NASDSE IDEA Partnership, Community of Practice National Meeting; and attendance at the National Secondary Transition Planning Institute where OSEP, NPSO, NSTTAC, and NDPC-SD provided guidance. Additionally, ADE/ESS maintained ongoing collaboration with the University of Kansas Transition Coalition and the Mountain Plains Regional Resource Center.  At the State level, ADE/ESS collaborated with RSA/VR, Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN), and met every other month with these agency stakeholders through the Arizona Community of Practice on Part B Arizona Annual Performance Report for FFY 2011 23 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Transition (AZCoPT). In FFY 2011, AZCoPT revised a presentation used statewide through RSA/VR and DBHS teleconferencing media to introduce participants to the supports/services available to school-aged and adult individuals with disabilities.  ADE/ESS collaboration meetings with the ADE sections of High School Renewal and Improvement, Career and Technical Education, Dropout Prevention, and School Counselors were scheduled and held approximately every three months and resulted in crosstraining for conferences sponsored by each ADE section on the topic of secondary transition. c) On an annual basis, create and disseminate information through a variety of sources: annual statewide conference, monitoring alerts, Web site, and listserv  Intra-ADE/ESS collaborative efforts included: monthly meetings with PINS (Parent Information Network specialists) as fellow AZCoPT members, as well as involvement with PINS during quarterly Transition Conference Planning Committee meetings; at least quarterly meetings with ESS Program Support to discuss the use of the Annual Site Visit Log (ASVL); review/revision of the secondary transition section of the monitoring manual and needed secondary transition trainings for ESS program specialists and PEAs; and the development of collaborative presentations with the ADE/ESS assistive technology unit. Activities completed from 8/1/11 through 6/30/12. 7/1/11– 6/30/13 ADE/ESS Transition Specialists  Four secondary transition presentations were offered at the ADE/ESS Directors Institute in August 2011 for approximately 78 participants.  Ninety-six sessions either directly Part B Arizona Annual Performance Report for FFY 2011 24 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 announcements or indirectly related to Indicator 1 were offered at Arizona’s Eleventh Annual Transition Conference held in October 2011.  Between July 2011 and June 2012, the ADE/ESS Secondary Transition Web site was updated to include additional and/or revised Indicator 13 materials from NSTTAC, IDEA Partnership, and other secondary transition technical assistance centers. The Web address is http://www.azed.gov/specialeducation/specialprojects/secondary-transition/. Activities completed from 7/1/11 to 6/30/12. d) On an annual basis, analyze preand post-training data collected  Analysis of ASVL pre-training through the Annual data was completed in January Site Visit Log (ASVL) 2012 after all ESS program for each PEA to specialists were able to complete determine level of at least one PEA annual site visit compliance on all during fall 2011. The analysis eight required showed a 76.5% average for components of compliance with the eight items of Indicator 13 Indicator 13. 7/1/11– 6/30/13 ADE/ESS Transition Specialists ADE/ESS Program Specialists  Post-training data analysis of all PEAs that received a pre-training annual site visit and were trained in secondary transition during FFY 2011 showed a 79.1% average for compliance with the eight items of Indicator 13. 2) Provide a twoyear capacity building grant to PEAs to participate in the Secondary Transition Mentoring Project (STMP) a) On an annual basis, identify PEAs who meet eligibility requirements and extend invitations to them to participate in STMP trainings Through pre- and post-training analysis, an increase of 2.6% in compliance for Indicator 13 was demonstrated after training and technical assistance was provided to PEAs by transition and education program specialists. Activities completed from 1/1/11 to 7/30/12.  Using Annual Site Visit (ASV) data for PEAs currently in Years 1, 2, or 3 of the monitoring cycle, 23 PEAs achieving less than 100% compliance on Indicator 13 were invited to apply for a 7/1/11– 6/30/13 ADE/ESS Transition Specialists ADE/ESS Program Specialists Part B Arizona Annual Performance Report for FFY 2011 25 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Team Training noncompetitive Secondary Transition Mentoring Project (STMP) capacity building grant. b) On an annual basis, provide indepth and ongoing professional development on transition requirements and best practices  Fourteen PEAs were accepted for participation in Year 1 of the STMP Team training in July 2011. Activities completed from 7/1/11 through 6/30/12. 7/1/11– 6/30/13 ADE/ESS Transition Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists  STMP participants attended Arizona’s Eleventh Annual Transition Conference, which included a STMP team orientation and designated sessions.  Training materials and activities were designed, created, and disseminated by ADE/ESS in collaboration with the University of Kansas Transition Coalition. Trainings focused on: identifying PEA barriers to meeting transition requirements; developing an action plan to eliminate barriers; creating IEPs that meet transition requirements; implementing training to build intra-PEA capacity to attain 100% compliance on secondary transition requirements; and determining improvements made and targeting areas still in need of improvement.  Team training for each cohort consisted of three two-day faceto-face trainings throughout the year. Additionally, Year 1 teams completed an online short course specially designed for STMP teams and focused on best practices in transition planning. Activity completed from 7/1/11 through 6/30/12. c) On an annual basis, analyze preand post-training data collected during  Measures to determine STMP trainings for effectiveness of STMP training each PEA that included the use of: participated to a) team process checklist; determine level of b) interrater reliability measures compliance on all for IEP file review; eight required c) training session evaluations; components of d) STMP team self-reported Part B Arizona Annual Performance Report for FFY 2011 26 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator 13 progress toward completion of action plans; and e) feedback provided by ADE/ESS monitoring program specialists on compliance. Data from the measures are as follows: a) April 2012 trainings for both STMP cohorts included the completion of a 37-item “team process” checklist. Results from the checklist use indicated both Year 1 and Year 2 teams reported an average of 92% for the item “all team members are highly knowledgeable of transition compliance requirements” and an average of 69% for the item “STMP team collaboratively develops and implements an action plan that addresses the prioritized needs.” b) Training for both cohorts included instruction and practice activities using interrater reliability measures. c) All STMP training sessions during FFY 2011 included instruction, activities, and Indicator 13 file reviews. Based on a 5-point Likert scale (1 lowest and 5 highest), evaluation results indicated an average score of 4.4 across all activities for both cohorts. Additionally, all team members completed a pre-/postevaluation of their competency in meeting Indicator 13 requirements that asked the question, “How prepared do you feel?” Based on a 5-point Likert Scale (1 equaled unprepared and 5 equaled prepared) and using a Paired Samples T-Test, all scores indicated a statistically significant increase in knowledge from the beginning to the end of the training for FFY 2011 for both cohorts (Years 1 and 2). The areas Part B Arizona Annual Performance Report for FFY 2011 27 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 reporting the greatest increase in knowledge were transition assessments and documenting transition services and course of study that will support the student in reaching postsecondary goals. d) Review of each STMP team’s action plan indicated all teams set goals to build and sustain systems to ensure 100% compliance on Indicator 13 school- or district-wide. All teams reported progress in meeting these goals. e) During the April 2012 training, Year 2 teams met with ADE/ESS program specialists to receive informal feedback on IEPs written during the second year of the STMP training program. Feedback indicated significant improvement in PEA knowledge and compliant practices. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 28 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 3: Assessments Participation and performance of children with IEPs on statewide assessments: A. Percent of the districts with a disability subgroup that meets the State’s minimum “n” size that meet the State’s AYP/AMO targets for the disability subgroup. B. Participation rate for children with IEPs. C. Proficiency rate for children with IEPs against grade level, modified and alternate academic achievement standards. (20 U.S.C. 1416 (a)(3)(A)) Measurement A. (choose either A.1 or A.2) A.1 AYP percent = [(# of districts with a disability subgroup that meets the State’s minimum “n” size that meet the State’s AYP targets for the disability subgroup) divided by the (total # of districts that have a disability subgroup that meets the State’s minimum “n” size)] times 100. A.2 AMO percent = [(# of districts with a disability subgroup that meets the State’s minimum “n” size that meet the State’s AMO targets for the disability subgroup) divided by the (total # of districts that have a disability subgroup that meets the State’s minimum “n” size)] times 100. B. Participation rate percent = [(# of children with IEPs participating in an assessment) divided by the (total # of children with IEPs enrolled during the testing window, calculated separately for reading and math)]. The participation rate is based on all children with IEPs, including both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year. C. Proficiency rate percent = ([(# of children with IEPs scoring at or above proficient against grade level, modified and alternate academic achievement standards) divided by the (total # of children with IEPs who received a valid score and for whom a proficiency level was assigned, and, calculated separately for reading and math)]. The proficiency rate includes both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year. Data Data Source The assessment data were from Arizona’s Instrument to Measure Standards (AIMS) and Arizona’s Instrument to Measure Standards Alternate (AIMS A). Data Description The assessment data were analyzed by the Arizona Department of Education’s Accountability Division/Research and Evaluation section (ADE/R and E) and the Information Technology division (IT). The Annual Measurable Objectives (AMO) data are the same data as the State’s data reported to the United States Department of Education (USDOE) under Title I of the Elementary and Secondary Education Act (ESEA). The participation and proficiency data are the same data as the State’s data reported to the United States Department of Education in the Consolidated State Performance Report (CSPR) submitted December 20, 2012. Part B Arizona Annual Performance Report for FFY 2011 29 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Arizona’s alternate assessment, AIMS A, is based on alternate academic achievement standards. The AIMS A is not based on grade level academic achievement standards and is not based on modified academic achievement standards. The AIMS and AIMS A data were used for determining AMO and for reporting participation and performance. The grades tested for FFY 2011 were grades 3 through 8 and grade 10. The State uses four categories for the proficiency status:  Falls Far Below the Standard (F)  Approaches the Standard (A)  Meets the Standard (M)  Exceeds the Standard (E) Students who met the standard (M) or exceeded the standard (E) were counted as proficient. Valid and Reliable Data The assessment data were obtained from the ADE Accountability Division/Research and Evaluation section and the Information Technology division (IT), which follows internal processes to ensure valid, reliable, and accurate data. The ADE Standards and Assessment division/Assessment section ensures its assessments adhere to the Standards for Educational and Psychological Testing. Elementary and Secondary Education Act (ESEA) Flexibility The USDOE issued a conditional approval to Arizona’s request for Elementary and Secondary Education Act (ESEA) flexibility, also known as an ESEA waiver, on July 19, 2012. Arizona’s goal for the flexibility request is to merge two different accountability systems, State and federal, into one seamless system that positions every student on track to college- and career-readiness. The ESEA waiver eliminates the requirement to determine Adequate Yearly Progress (AYP). Arizona was given the flexibility to redefine proficiency targets, setting new Annual Measurable Objectives (AMOs) in mathematics and reading/language arts to achieve 100% proficiency for all students by school year 20192020. Indicator 3A.2 — Annual Measurable Objectives (AMOs) Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 26% Indicator 3A.2 — Annual Measurable Objectives (AMOs) Actual Target Data for FFY 2011 Year Total Number of PEAs Number of PEAs That Met the Minimum “n” Size Number of PEAs That Met the Minimum “n” Size and Met AMO for FFY 2011 Percent of PEAs FFY 2011 (2011– 2012) 648 69 2 2.90% Part B Arizona Annual Performance Report for FFY 2011 30 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Arizona did not meet the target for Indicator 3A.2 for AMO. Indicator 3B — Target Data for Mathematics and Reading Participation for FFY 2011 FFY 2011 Measurable and Rigorous Targets Mathematics 95% Reading 95% Indicator 3B — Actual Target Data for Mathematics Participation for FFY 2011 Mathematics Assessment Participation for FFY 2011 Total Number of Children with Disabilities Participating Percentage of Children with Disabilities Participating Who Took the Specified Assessment a Children with IEPs enrolled 73,140 b Children with IEPs participating in regular assessment without accommodations 32,317 44.2% c Children with IEPs participating in regular assessment with accommodations 33,363 45.6% d Children with IEPs participating in alternate assessment against grade-level standards 0 0 e Children with IEPs participating in alternate assessment against modified standards 0 0 f Children with IEPs participating in alternate assessment based on alternate achievement standards 6,386 8.7% g Students with IEPs participating (b+c+d+e+f) 72,066 98.5% Children included in a but not included in the other counts above Children with IEPs who were not participants, were absent, or had invalid scores, consistent with Arizona’s Accountability Workbook and requirements. 1,074 1.5% Arizona exceeded the target for Indicator 3B for the mathematics participation rate. Indicator 3B — Actual Target Data for Reading Participation for FFY 2011 Part B Arizona Annual Performance Report for FFY 2011 31 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Reading Assessment Participation for FFY 2011 Total Number of Children with Disabilities Participating Percentage of Children with Disabilities Participating Who Took the Specified Assessment a Children with IEPs enrolled 73,213 b Children with IEPs participating in regular assessment without accommodations 36,839 50.3% c Children with IEPs participating in regular assessment with accommodations 28,948 39.5% d Children with IEPs participating in alternate assessment against grade-level standards 0 0 0 0 6,388 8.7% 72,175 98.6% e f g Children with IEPs participating in alternate assessment against modified standards Children with IEPs participating in alternate assessment based on alternate achievement standards Students with IEPs participating (b+c+d+e+f) Children included in a but not included in the other counts above Children with IEPs who were not participants, were absent, or had invalid scores, consistent with Arizona’s Accountability Workbook and requirements. 1,038 1.4% Arizona exceeded the target for Indicator 3B for the reading participation rate. Mathematics and Reading Participation Data The mathematics and reading participation data are the same as the State’s data used for accountability reporting under Title I of the ESEA and were reported in Arizona’s Consolidated State Performance Report Part I, submitted December 20, 2012. Mathematics and reading participation rates are inclusive of all ESEA grades assessed in Arizona (grades 3 through 8 and grade 10) for students with IEPs and inclusive of all assessments (regular and alternate). The calculation includes all students with IEPs in all the grades assessed, including those enrolled for a full academic year and those not enrolled for a full academic year. The differences between the denominators for mathematics and reading can be attributed to the different assessment dates for the different grades. Mathematics and reading assessments were given to all students in grades 3 through 8 on April 16, 2012. The mathematics assessment was given to students in grade 10 on April 10, 2012. The reading assessment was given to students in grade 10 on February 29, 2012. Indicator 3C — Target Data for Mathematics and Reading Proficiency for FFY 2011 Part B Arizona Annual Performance Report for FFY 2011 32 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Annual Measurable Objectives (AMOs) by Grade and Subject Grades Mathematics Reading 3 69% 77% 4 5 6 7 8 10 66% 64% 61% 63% 56% 63% 76% 80% 82% 83% 73% 79% The mathematics and reading AMOs are the same as the State’s AMOs as given in the State of Arizona ESEA Flexibility Request, dated July 13, 2012, which is considered the current Arizona Accountability Workbook. The mathematics and reading AMOs are the new AMOs established by the ADE that increase in annual equal increments and result in 100% proficiency no later than the end of school year 20192020. Indicator 3C — Actual Target Data for Mathematics Proficiency for FFY 2011 FFY 2011 Grades Total enrolled Total number tested and enrolled for full academic year Total number children with IEPs scoring at or above proficient Total percent children with IEPs scoring at or above proficient Mathematics Assessment Proficiency 3 4 5 6 7 8 10 11275 11501 11366 10885 10072 9854 8187 11129 11379 11254 10764 9930 9689 7921 4538 3957 3262 2629 2313 1846 1537 40.8% 34.7% 28.9% 24.4% 23.3% 19.1% 19.4% Arizona did not meet the target for Indicator 3C for mathematics proficiency in any of the assessed grades 3 through 8 and grade 10. Indicator 3C — Actual Target Data for Reading Proficiency for FFY 2011 FFY 2011 Grades Total enrolled Reading Assessment Proficiency 3 4 5 6 7 8 10 11275 11501 11366 10885 10072 9854 8260 Part B Arizona Annual Performance Report for FFY 2011 33 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Total number tested and enrolled for full academic year Total number children with IEPs scoring at or above proficient Total percent children with IEPs scoring at or above proficient 11131 11379 11249 10763 9939 9691 8023 4516 4719 4482 4399 4395 2889 3121 40.6% 41.5% 39.8% 40.8% 44.2% 29.8% 38.9% Arizona did not meet the target for Indicator 3C for reading proficiency in any of the assessed grades 3 through 8 and grade 10. Mathematics and Reading Proficiency Data The mathematics and reading proficiency data are provided in the same format as the State’s data reported under Title I of the ESEA in Arizona’s Consolidated State Performance Report Part I, submitted December 20, 2012. Mathematics and reading proficiency rates are inclusive of all ESEA grades assessed in Arizona (grades 3 through 8 and grade 10) for students with IEPs and inclusive of all assessments (regular and alternate). The calculation includes all students with IEPs in all the grades assessed, including those enrolled for a full academic year and those not enrolled for a full academic year. Explanation of Progress and Slippage That Occurred for FFY 2011 Although the target was not met for Indicator 3A.2—AMO, Arizona saw improvement from FFY 2010 (1.39%) to FFY 2011 (2.9%), a rise of 1.51 percentage points. Arizona exceeded mathematics and reading targets for Indicator 3B—Participation and made progress in both areas. Mathematics rose from 97.3% participation in FFY 2010 to 98.5% participation in FFY 2011. The gain in reading was the same amount (1.2%), increasing from 97.4% in FFY 2010 to 98.6% in FFY 2011. Arizona did not meet the targets for Indicator 3C—Proficiency in mathematics or reading. Arizona will continue its efforts to increase student achievement in the two subject areas. In FFY 2011, the ADE/ESS supported PEAs in the area of mathematics that targeted a range of grades with sponsorship of three initiatives—Special Education Using Mathematics for School Improvement Project (SUMS), Special Education Achieving Success in Mathematics (SEAS-Math), and Dimensions of Algebra. Passages: Achieving Success in Reading was a reading program implemented in middle schools and high schools. Both the math and reading initiatives provided research-based interventions, strategies and resources, and coaching to the school-based teams. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (Goal) 1) Increase opportunities for training in Sub-Activities (Objectives or Action Steps) a) Conduct mathematics strategy trainings annually at Timeline Complete Activity completed August 2011. Projected 7/1/11– 6/30/13 Resources (Planned) CSPD Staff Part B Arizona Annual Performance Report for FFY 2011 34 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 mathematics strategies to public education agency (PEA) special education personnel and distribute resource information in reading the Directors Institute for special education personnel from school districts and charter schools b) Compile mathematics strategy and resource information 2) Increase opportunities for training in reading strategies to public education agency (PEA) special education personnel and distribute resource information in reading c) Disseminate mathematics strategy and resource information through the Arizona Promising Practices Web site, the ESS listserv, and ESS/CSPD trainings a) Represent ESS at the Response to Intervention (RTI) meetings with other ADE divisions b) Conduct reading strategy trainings annually at the Directors Institute for special education personnel from school districts and charter schools c) Compile reading strategy and resource information d) Disseminate reading strategy and resource information through the Arizona Promising Practices Web site, the ESS The 2011 Directors Institute offered special education teachers three sessions on mathematics strategies for kindergarten through high school students. Activity completed from 7/1/11 to 6/30/12. Mathematics and strategy resource information was collected and is located at http://www.azed.gov/stan dardspractices/academicstandards/math/. Activity not completed during FFY 2011 due to restructuring of personnel and technology. Activity will be reported in the FFY 2012 APR. Activity discontinued due to restructuring of personnel and reorganization of the Division. Activity completed August 2011. The 2011 Directors Institute offered special education teachers sessions on reading strategies for kindergarten through high school students. Activity completed from 7/1/11 to 6/30/12. Reading and strategy resource information was collected and is located at http://www.azed.gov/stan dards-practices/reading/. Activity not completed during FFY 2011 due to restructuring of personnel and technology. Activity will be reported in the FFY 2012 APR. 7/1/11– 6/30/13 CSPD Staff 7/1/11– 6/30/13 CSPD Staff 7/1/10– 6/30/13 CSPD Staff 7/1/11– 6/30/13 CSPD Staff 7/1/11– 6/30/13 CSPD Staff 7/1/11– 6/30/13 CSPD Staff Part B Arizona Annual Performance Report for FFY 2011 35 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 3) By the end of FFY 2012, teams will increase their content knowledge of scientifically based strategies for mathematics instruction for grades K–2 students with IEPs through Special Education Using Mathematics for School Improvement Project (SUMS) listserv, and ESS/CSPD trainings a) Teams will learn and use a cyclical process of screening, content strategies, data analysis, and collaboration of student need and will implement studentspecific strategies. Measurement will be team effectiveness data. (Each team will have a coach during the training sessions to assist with the team processes.) b) Apply the 2010 Arizona mathematics standards using the mathematics processes that enable students with IEPs to become fluent in mathematics as reflected in classroom observation protocol visits and various assessments c) Track and analyze data of students with IEPs over time using AIMS data and other assessment data provided by the schools Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff Each of the 10 teams had coaches during the training sessions. Each team completed a survey rating the support received from the coaches. The five-point scale ranged from strongly disagree to strongly agree. The overall rating for the effectiveness of the coaches was 4.2. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff 100% of the teams received instruction in the Mathematical Processes. The Mathematics Student-Centered Observation Protocol was used to measure if the strategies were being used in the classrooms. The protocol was based on a two-point rating system with two being totally observed and zero not being observed. The schools started with an average of 0.5 and ended with an average of 1.6. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff 100% of the 10 teams received the cyclical instruction process. 100% of the 10 teams completed instructional plans. AIMS data is not available because these students will take AIMS for the first time during school year 2012-2013. Other assessment data Part B Arizona Annual Performance Report for FFY 2011 36 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 d) Analyze teacher pre- and postassessment data to determine continuity of learning and instruction 4) By the end of FFY 2012, teams will increase their content knowledge of scientifically based strategies for mathematics instruction for students with IEPs grades 3–5 through Special Education Achieving Success in Mathematics (SEASMath) a) Teams will learn and use a cyclical process of screening, content strategies, data analysis, and collaboration on student need and will implement studentspecific strategies Measurement will be team effectiveness data. (Each team will have a coach during the training sessions to assist with the team processes.) b) Apply the 2008 Arizona mathematics standards and make connections to the 2010 Arizona mathematics standards using the mathematics processes that enable students with IEPs to become more fluent in mathematics as reflected in classroom observation protocol visits and various used by the group showed student growth ranged from 5% to 42%. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff Team members were given a pre-assessment and a post-assessment. The average score for the pre-assessment was three of 15 possible points. The average score for the postassessment was 11 of 15 possible points. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff 100% of the 10 teams received the cyclical instruction process. 80% of the 10 teams completed instructional plans. Each of the 10 teams had coaches during the training sessions. Each team completed a survey rating the support received from the coaches. The five-point scale ranged from strongly disagree to strongly agree. The overall rating for the effectiveness of the coaches was 4.2. Activity completed from 7/1/11 to 6/30/12. 100% of the teams received instruction in the 2008 standards with connections to the 2010 standards. The Mathematics StudentCentered Observation Protocol was used to determine if the strategies were used in the classrooms. The protocol was based on a two-point Part B Arizona Annual Performance Report for FFY 2011 37 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 assessments c) Track and analyze data of students with IEPs over time using AIMS data and other assessment data provided by the schools d) Analyze teacher pre- and postassessment data to determine continuity of learning and instruction 5) By the end of FFY 2012, teams will increase their content knowledge of scientifically based strategies for mathematics instruction for students with IEPs grades 7–12 through Dimensions of Algebra a) Provide training in algebraic strategies for students with IEPs that will enable them to access mathematics from a concrete model to an abstract model b) Apply data analysis processes from various types of assessments to differentiate instruction for students with IEPs rating system with two being totally observed and zero not being observed. The schools started with an average of 0.7 and ended with an average of 1.2. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff The Year One team started with an average score of 62.7% on the AIMS and ended with an average score of 73.4%, with a gain of 10.7%. The Year Two team started with an average score of 58.4% and ended with an average score of 73.7%, with a gain of 15.3%. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff 100% of the teams completed the preassessment, resulting in an average score of four of 15 possible points. The post-assessment will be administered at the end of year two. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff 100% of the eight teams received scientific-based research strategies. Strategies were taught starting with the concrete stage, then moving to the pictorial stage, and finally to the abstract stage through the use of written and verbal communication of mathematical ideas. Activity completed from 7/1/11 to 6/30/12. 9/1/11– 6/30/13 CSPD Staff 100% of the teams administered algebraic assessments to students with disabilities. The Part B Arizona Annual Performance Report for FFY 2011 38 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 c) Provide a networking format for middle school and high school teachers to meet the transitional needs of students with IEPs as they move from middle school to high school d) Track and analyze data of students with IEPs over time using AIMS data and other assessment data provided by the schools e) Analyze teacher pre- and postassessment data to determine continuity of learning and instruction 6) Middle school teams participating in Passages: Achieving Success in Reading training will receive training that when implemented will result in an increased proficiency rate of eighth-grade students with IEPs a) Conduct Passages training for special education teachers and reading specialists assessments were scored and then student need was identified and instructional plans were written based on the data. This activity was not implemented in year one of the program. The activity will be implemented in year two and will be reported in the FFY 2012 APR. This activity not completed because program implementation did not occur until school year 2012-2013. The activity will be reported in the FFY 2012 APR. Activity completed from 7/1/11 to 6/30/12. 100% of the eight teams completed the preassessment, resulting in an average score of two of five possible points. The post-assessment will be administered at the end of year three. Activity completed from 10/1/11 to 3/31/12. One middle school participated in Passages training. Their team consisted of one special education teacher, one math teacher, one language arts teacher, and one social studies teacher. 9/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff 10/1/11– 6/30/13 CSPD capacity building grants for qualifying schools and CSPD staff CSPD reading specialist Presenter for “Passages” Research on improving reading for students with IEPs in middle Part B Arizona Annual Performance Report for FFY 2011 39 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 school b) Provide training on assessing and diagnosing student needs to guide instruction/intervention c) Provide instructional strategy training in the areas of word study, vocabulary, and comprehension d) Provide site-based technical assistance to participating schools to offer feedback and support in implementation of new strategies e) Collect progress monitoring student data throughout training to determine effectiveness of strategies f) Provide capacity building coaching support to each team CSPD capacity building coaches Same Activity completed from 10/1/11 to 3/31/12. 10/1/11– 6/30/13 The team was trained in the administration and interpretation of three assessments: MAZE, spelling assessment, and a writing prompt assessment to determine the range of students’ abilities and to determine greatest area of need. Activity completed from 10/1/11 to 3/31/12. 10/1/11– 6/30/13 Same The team received training in the areas of oral presentation of words, Greek/Latin roots, identification of morphemes and relationship to vocabulary, and using graphic organizers to support comprehension. Activity completed from 10/1/11 to 5/10/12. 10/1/11– 6/30/13 Same 10/1/11– 6/30/13 Same 10/1/11– 6/30/13 Same The middle school was visited on three occasions for observations in the classrooms and for meetings with the team and site administrator. Activity not completed during first year of program. The team is not prepared to implement new strategies and collect data about the effectiveness of strategies. This activity will be reported in the FFY 2012 APR. Activity completed from 10/1/11 to 3/31/12. Part B Arizona Annual Performance Report for FFY 2011 40 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 7) High school teams participating in Passages: Achieving Success in Reading training will receive training that when implemented will result in an increased proficiency rate of 10th grade students with IEPs to increase team effectiveness during training sessions a) Conduct Passages training for special education teachers and reading specialists The team was provided a capacity building coach at each session. Activity completed from 10/1/11 to 3/31/12. 10/1/11– 6/30/13 Two high schools participated in Passages training. These two teams consisted of two special education teachers, five English/Language Arts teachers, one social studies teacher, and one chemistry teacher. CSPD capacity building grants for qualifying schools and CSPD staff CSPD reading specialist Presenter for “Passages” Research on improving reading for high school students with IEPs b) Provide training on assessing and diagnosing student needs to guide instruction/intervention c) Provide instructional strategy training in the areas of word study, vocabulary, and comprehension Activity completed from 10/1/11 to 3/31/12. 10/1/11– 6/30/13 The teams were trained in the administration and interpretation of three assessments: MAZE, spelling assessment, and a writing prompt assessment to determine the range of students’ abilities and to determine greatest area of need. Activity completed from 10/1/11 to 3/31/12. 10/1/11– 6/30/13 CSPD capacity building coaches Same Same The team received training in the areas of oral presentation of words, Greek/Latin roots, identification of morphemes and relationship to vocabulary, and using Part B Arizona Annual Performance Report for FFY 2011 41 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 d) Provide site-based technical assistance to participating schools to offer feedback and support in implementation of new strategies e) Collect progress monitoring student data throughout training to determine effectiveness of strategies f) Provide capacity building coaching support to each team to increase team effectiveness during training sessions graphic organizers to support comprehension. Activity completed from 10/1/11 to 3/31/12. The high schools were visited on three occasions for observations in the classrooms and for meetings with the team. Activity not completed during first year of program. The team is not prepared to implement new strategies and collect data about the effectiveness of strategies. This activity will be reported in the FFY 2012 APR. Activity completed from 10/1/11 to 3/31/12. 10/1/11– 6/30/13 Same 10/1/11– 6/30/13 Same 10/1/11– 6/30/13 Same Each team was provided a capacity building coach at each session. Public Reporting Information The location (URL) of public reports of assessment results conforming to 34 CFR § 300.160 (f) is http://www.azed.gov/research-evaluation/aims-assessment-results/. The FFY 2011 Annual Performance Report (APR) gives information about the participation of students with IEPs. The APR is located on the ADE/ESS Web site at http://www.azed.gov/specialeducation/resources/spp-apr/ under the list titled Annual Performance Report. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 42 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 4A: Suspension and Expulsion Percent of districts that have a significant discrepancy in the rate of suspensions and expulsions of greater than 10 days in a school year for children with IEPs. (20 U.S.C. 1416 (a)(3)(A); 1412 (a)(22)) Measurement Percent = [(# of districts that have a significant discrepancy in the rates of suspensions and expulsions for greater than 10 days in a school year of children with IEPs) divided by the (# of districts in the State)] times 100. Include State’s definition of “significant discrepancy.” Data Data Source The 2010–2011 data were reported by the PEAs through the Arizona Safety Accountability for Education (Az SAFE) application. The data are the same as the data reported under section 618, Table 5 (Report of Children with Disabilities Subject to Disciplinary Removal) for school year 2010–2011, which was due November 2, 2011. The October 1, 2010, child count data are the same as the State’s data reported under section 618, Table 1, Report of Children with Disabilities Receiving Special Education Under Part B of the Individuals With Disabilities Education Act. Note that the source of this data is from FFY 2010. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the Az SAFE data through the Annual Special Education Data Collection process, which uses internal edit checks. The State requires an assurance from the PEAs through their submission of a signed verification form attesting to the validity of the data. Definition of Significant Discrepancy and Methodology Arizona uses Statistical Analysis Software (SAS) to calculate rates of suspension and expulsion for children with IEPs. Arizona uses the state bar method to determine significant discrepancy. The State rate of suspensions/expulsions greater than 10 days for all students with IEPs is 0.83%. The State bar, 5.83%, is five percentage points greater than the State rate. A district or charter school has significant discrepancy when its suspension/expulsion rate greater than 10 days for students with IEPs is 5.83% or greater. There must be at least 50 students in the denominator of a suspension/expulsion rate for a district or charter school to be flagged as having significant discrepancy. The denominator represents the overall special education enrollment at the district or charter school. Part B Arizona Annual Performance Report for FFY 2011 43 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Using the minimum “n” size of 50 students for overall special education enrollment, Arizona excluded 18 PEAs from the calculation (excluded 18 from 587) and used the total number of PEAs (587) in the State in the denominator. Arizona compares the rates of suspensions and expulsions of greater than 10 days in a school year for students with IEPs among PEAs in the State. Target Data for FFY 2011 (using 2010–2011 data) FFY Measurable and Rigorous Target FFY 2011 (using 2010– 2011 data) 1.25% Indicator 4A — Actual Target Data for FFY 2011 (using 2010–2011 data) 0.0% Arizona met the target. PEAs with Significant Discrepancy in Rates for Suspension and Expulsion Year Total Number of PEAs* Number of PEAs That Have Significant Discrepancies Percent FFY 2011 (using 2010–2011 data) 587 0 0.00% *Arizona included the total number of PEAs in the denominator. Review of Policies, Procedures, and Practices for FFY 2011 (using 2010–2011 data) The State reviewed the PEAs’ suspension/expulsion data and did not identify any PEAs with a significant discrepancy. Arizona did not make any findings of noncompliance with Part B requirements as a result of the review required by 34 CFR § 300.170 (b). Explanation of Progress That Occurred for FFY 2011 The FFY 2011 rate of 0.0% represents progress compared to FFY 2010 (0.34%). Arizona has continued to encourage PEAs to reduce their suspension and expulsion rates. The ADE/ESS offers support to the PEAs by sponsoring the Principals Institute for principals and other administrative personnel in order for them to learn IDEA’s discipline requirements. The ADE’s Dispute Part B Arizona Annual Performance Report for FFY 2011 44 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Resolution director and staff present these workshops in the three major urban centers of the State. The Special Education Handbook for Principals: A Reference for Law Related Issues is updated annually by the Dispute Resolution director, who is an attorney, and is distributed to workshop participants. Specialists in the Program Support and Monitoring unit visit their assigned PEAs throughout the year to provide technical assistance on a number of topics, including discipline. School staff and ESS specialists review discipline data, examine root causes of issues, and formulate improvement plans. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Conduct trainings related to the discipline process for students with disabilities Sub-Activities (Objectives or Action Steps) a) Conduct semiannual Principal Institutes in the three main geographical regions of the State b) Disseminate Special Education Handbook for Principals: A Quick Reference for Law Related Issues to participants at the Principals Institutes Timeline Complete Three Principals Institute: Special Education Legal Issues, were presented in:  Flagstaff on 10/19/11 and 4/3/12  Phoenix on 11/9/11 and 3/1/12  Tucson on 11/3/11 and 3/8/12 The trainings were developed and presented by the Dispute Resolution director and the complaint investigator/corrective action compliance monitor. Activity completed from 7/1/11 to 6/30/12. Participants at the first session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Part I  Principals Institute Power Point Part I  Special Education Acronyms  Glossary of Special Education Terms  Disability Classifications in Arizona Participants at the second session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Projected 7/1/11– 6/30/13 Resources (Planned) Comprehensi ve System of Personnel Development (CSPD) Staff Dispute Resolution Director and Staff 7/1/11– 6/30/13 CSPD Staff Dispute Resolution Director and Staff Part B Arizona Annual Performance Report for FFY 2011 45 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Part II Dispute Resolution publication developed by Exceptional Student Services  Prior Written Notice form  What Circumstance Require a Prior Written Notice? handout  Annual Notification to Parents Regarding Confidentiality of Student Education Records form  Manifestation Determination form Activity completed 2/1/12.  2) Provide support for PEAs that are flagged as at risk for significant discrepancy a) On an annual basis, analyze data to flag PEAs that are at risk for significant discrepancy b) On an annual basis, notify PEAs that are flagged as at risk for significant discrepancy c) Provide assessment tools and resources to PEAs that are flagged as at risk to conduct root cause analyses 3) Provide support for PEAs that are flagged for significant discrepancy a) On an annual basis, notify PEAs that are flagged for significant discrepancy b) Provide technical assistance to PEA staff during the PEA’s review of policies, procedures, and practices 7/1/11– 6/30/13 Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk suspension/expulsion rates. Activity completed from 3/1/12 to 6/30/12. 7/1/11– 6/30/13 ESS specialists reviewed suspension/expulsion data during on-site meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 3/1/12 to 6/30/12. 7/1/11– 6/30/13 ESS specialists gave self assessment tools and resources to PEAs that were identified as at-risk. Activity completed 4/1/12. ESS notified PEAs that were flagged for significant discrepancy. Activity completed from 4/1/12 to 6/30/12. ESS specialists made onsite visits to work with PEA staff during the review of policies, procedures, and 7/1/11– 6/30/13 7/1/11– 6/30/13 Suspension/ Expulsion Data ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists Resource Tools and Lists ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists Resource Tools and Part B Arizona Annual Performance Report for FFY 2011 46 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 practices. Lists Correction of FFY 2010 Findings of Noncompliance Arizona did not make any findings of noncompliance in FFY 2010 as a result of the review of policies, procedures, and practices required by 34 CFR 300.170 (b). Actions Taken if Noncompliance Not Corrected Arizona did not make any findings of noncompliance in FFY 2010. Verification of Correction (either timely or subsequent) Arizona did not make any findings of noncompliance in FFY 2010. Correction of Remaining FFY 2009 Findings of Noncompliance Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 47 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 4B: Suspension and Expulsion by Race or Ethnicity Percent of districts that have: (a) a significant discrepancy, by race or ethnicity, in the rate of suspensions and expulsions of greater than 10 days in a school year for children with IEPs; and (b) policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards. (20 U.S.C. 1416 (a)(3)(A); 1412 (a)(22)) Measurement Percent = [(# of districts that have: (a) a significant discrepancy, by race or ethnicity, in the rates of suspensions and expulsions of greater than 10 days in a school year of children with IEPs; and (b) policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards) divided by the (# of districts in the State)] times 100. Data Data Source The 2010–2011 data were reported by the PEAs through the Arizona Safety Accountability for Education (Az SAFE) application. The data are the same as the data reported under section 618, Table 5 (Report of Children with Disabilities Subject to Disciplinary Removal) for school year 2010–2011, which was due November 2, 2011. The October 1, 2010, child count data are the same as the State’s data reported under section 618, Table 1, Report of Children with Disabilities Receiving Special Education Under Part B of the Individuals With Disabilities Education Act. Note that the source of this data is from FFY 2010. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the Az SAFE data through the Annual Special Education Data Collection process, which uses internal edit checks. The State requires an assurance from the PEAs through their submission of a signed verification form attesting to the validity of the data. Definition of Significant Discrepancy and Methodology Arizona uses Statistical Analysis Software (SAS) to calculate rates of suspension and expulsion by race or ethnicity for children with IEPs. Arizona uses the state bar method to determine significant discrepancy. The State rate of suspensions/expulsions greater than 10 days for all students with IEPs is 0.83%. The State bar, 5.83%, is five percentage points greater than the State rate. Any district or charter school that suspends or expels 5.83% or more of its students with IEPs of a given race/ethnicity for more than 10 days is flagged for significant discrepancy. There must be at least 50 Part B Arizona Annual Performance Report for FFY 2011 48 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 students in the denominator of a suspension/expulsion rate for a district or charter school to be flagged as having significant discrepancy. The denominator represents the special education enrollment at the district or charter school for a given race/ethnicity. Using the minimum “n” size of 50 students for a given race/ethnicity enrollment, Arizona excluded 34 PEAs from the calculation (excluded 34 from 587) and used the total number of PEAs (587) in the State in the denominator. Arizona compares the rates of suspension and expulsion of greater than 10 days in a school year for students with IEPs among PEAs in the State. Target Data for FFY 2011 (using 2010–2011 data) FFY Measurable and Rigorous Target FFY 2011 (using 2010– 2011 data) 0.0% Indicator 4B — Actual Target Data for FFY 2011 (using 2010–2011 data) 0.17% Arizona did not meet the target. 4B (a) PEAs with Significant Discrepancy, by Race or Ethnicity, in Rates of Suspension and Expulsion Year Total Number of PEAs* Number of PEAs That Have Significant Discrepancies by Race or Ethnicity Percent of PEAs FFY 2011 (using 2010–2011 data) 587 3 0.51% *Arizona included the total number of PEAs in the denominator. 4B (b) PEAs with Significant Discrepancy, by Race or Ethnicity, in Rates of Suspensions and Expulsions; and policies, procedures, or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards Part B Arizona Annual Performance Report for FFY 2011 49 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Year Total Number of PEAs* Number of PEAs That Have Significant Discrepancies, by Race or Ethnicity, and policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards FFY 2011 (using 2010–2011 data) 587 1 Percent of PEAs 0.17% *Arizona included the total number of PEAs in the denominator Review of Policies, Procedures, and Practices for FFY 2011 (using 2010–2011 data) The State reviewed the PEAs’ suspension/expulsion data by race or ethnicity and identified three PEAs with a significant discrepancy. These PEAs reviewed their policies, procedures, and practices relating to the development and implementation of IEPs, use of positive behavioral interventions and supports, and procedural safeguards to determine if these contributed to the significant discrepancy. Arizona required these PEAs to have special education policies and procedures in compliance with all regulatory requirements prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. These PEAs were required to resubmit the discipline policies and procedures for review by ESS program specialists to determine if they were in alignment with the requirements of 34 CFR § 300.530 through § 300.536. The practices of these PEAs were reviewed by means of a self assessment. The PEAs conducted an assessment of their discipline practices, which consisted of a series of questions requiring narrative responses and a review of student files using the State’s monitoring forms. ADE/ESS specialists conducted on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews. Upon the completion of this review, Arizona determined that two of the three PEAs were in compliance with IDEA requirements that pertain to the development and implementation of IEPs, use of positive behavioral interventions and supports, and procedural safeguards. One PEA was not in compliance. The school district did not follow discipline requirements for the manifestation determination process and did not use positive behavioral interventions and supports in the development and implementation of IEPs. Arizona made one finding of noncompliance with Part B requirements as a result of the review required by 34 CFR § 300.170(b). Upon receipt of the finding of noncompliance, the PEA revised its discipline policies and procedures and corrected all noncompliance. The ADE/ESS program specialist verified that the PEA corrected all instances of noncompliance, including child specific. Based on subsequent file reviews of updated data, the specialist determined that the PEA implemented sustainable practices to meet the regulatory requirements. Part B Arizona Annual Performance Report for FFY 2011 50 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Although one finding of noncompliance was made, correction of all noncompliance was completed by the time of submission of the FFY 2011 APR. Explanation of Slippage That Occurred for FFY 2011 Minor slippage occurred from FFY 2010 (0.0%) to FFY 2011 (0.17%). This slippage was the result of one school district being found out of compliance with discipline practices. The ADE/ESS program specialist assigned to support the district continued to offer technical assistance throughout the fiscal year, which enabled the PEA to comply with the IDEA requirements pertaining to discipline. Arizona continues to encourage PEAs to reduce their suspension and expulsion rates. The ADE/ESS offers support to the PEAs by sponsoring the Principals Institute for principals and other administrative personnel to learn IDEA’s discipline requirements. The ADE’s Dispute Resolution director and staff present these workshops in the three major urban centers of the State. The Special Education Handbook for Principals: A Reference for Law Related Issues is updated annually by the Dispute Resolution director, who is an attorney, and is distributed to workshop participants. Specialists in the Program Support and Monitoring Unit visit their assigned PEAs throughout the year to provide technical assistance on a number of topics, including discipline. School staff and ESS specialists review discipline data, examine root causes of issues, and formulate improvement plans. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Conduct trainings related to the discipline process for students with disabilities Sub-Activities (Objectives or Action Steps) a) Conduct semiannual Principal Institutes in the three main geographical regions of the State b) Disseminate Special Education Handbook for Principals: A Quick Reference for Law Related Issues to participants at the Principals Institutes Timeline Complete Three Principals Institute: Special Education Legal Issues, were presented in:  Flagstaff on 10/19/11 and 4/3/12  Phoenix on 11/9/11 and 3/1/12  Tucson on 11/3/11 and 3/8/12 The trainings were developed and presented by the Dispute Resolution director and the complaint investigator/corrective action compliance monitor. Activity completed from 7/1/11 to 6/30/12. Participants at the first session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Part I Projected 7/1/11– 6/30/13 Resources (Planned) Comprehensi ve System of Personnel Development (CSPD) Staff Dispute Resolution Director and Staff 7/1/11– 6/30/13 CSPD Staff Dispute Resolution Director and Staff Part B Arizona Annual Performance Report for FFY 2011 51 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011  2) Provide support for PEAs that are flagged as at risk for significant discrepancy a) On an annual basis, analyze data to flag PEAs that are at risk for significant discrepancy b) On an annual basis, notify PEAs that are flagged as at risk for significant discrepancy c) Provide assessment tools and resources to Principals Institute Power Point Part I  Special Education Acronyms  Glossary of Special Education Terms  Disability Classifications in Arizona Participants at the second session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Part II  Dispute Resolution publication developed by Exceptional Student Services  Prior Written Notice form  What Circumstance Require a Prior Written Notice? handout  Annual Notification to Parents Regarding Confidentiality of Student Education Records form  Manifestation Determination form Activity completed 2/1/12. 7/1/11– 6/30/13 Suspension/ Expulsion Data Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk suspension/expulsion rates by race/ethnicity. Activity completed from 3/1/12 to 6/30/12. 7/1/11– 6/30/13 ESS specialists reviewed suspension/expulsion data by race/ethnicity during onsite meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 3/1/12 to 6/30/12. ADE/ESS Directors and Program Specialists 7/1/11– 6/30/13 ADE/ESS Directors and Program ADE/ESS Directors and Program Specialists Part B Arizona Annual Performance Report for FFY 2011 52 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 3) Provide support for PEAs that are flagged for significant discrepancy PEAs that are flagged as at risk to conduct root cause analyses ESS specialists gave self assessment tools and resources to PEAs that were identified as at risk. a) On an annual basis, notify PEAs that are flagged for significant discrepancy Activity completed 4/1/12. b) Provide technical assistance to PEA staff during their review of policies, procedures, and practices ESS notified PEAs that were flagged for significant discrepancy by race/ethnicity. Activity completed from 4/1/12 to 6/30/12. ESS specialists made onsite visits to work with PEA staff during the review of policies, procedures, and practices. Specialists 7/1/11– 6/30/13 7/1/11– 6/30/13 Resource Tools and Lists ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists Resource Tools and Lists Correction of FFY 2010 Findings of Noncompliance Arizona did not make any findings of noncompliance in FFY 2010 as a result of the review of policies, procedures, and practices required by 34 CFR 300.170 (b). Actions Taken if Noncompliance Not Corrected Arizona did not make any findings of noncompliance in FFY 2010. Verification of Correction (either timely or subsequent) Arizona did not make any findings of noncompliance in FFY 2010. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 53 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 5: School Age LRE Percent of children with IEPs aged 6 through 21 served: A. Inside the regular class 80% or more of the day; B. Inside the regular class less than 40% of the day; and C. In separate schools, residential facilities, or homebound/hospital placements. (20 U.S.C. 1416 (a)(3)(A)) Measurement A. Percent = [(# of children with IEPs served inside the regular class 80% or more of the day) divided by the (total # of students aged 6 through 21 with IEPs)] times 100. B. Percent = [(# of children with IEPs served inside the regular class less than 40% of the day) divided by the (total # of students aged 6 through 21 with IEPs)] times 100. C. Percent = [(# of children with IEPs served in separate schools, residential facilities, or homebound/hospital placements) divided by the (total # of students aged 6 through 21 with IEPs)] times 100. Indicator 5A, 5B, and 5C — Target Data for FFY 2011 FFY 2011 Measurable and Rigorous Targets Indicator 5A ≥ 80% Indicator 5B < 40% Indicator 5C Separate 55% 14% 1.5% Indicator 5A, 5B, and 5C — Actual Target Data for FFY 2011 Indicator 5A ≥ 80% Indicator 5B < 40% Indicator 5C Separate number of children 67,607 16,437 3,145 percentage of children 60.4% 14.68% 2.8% number of students aged 6–21 with IEPs 111,963 Arizona exceeded the target for Indicator 5A. Arizona did not meet the target for Indicator 5B. Arizona did not meet the target for Indicator 5C. Data Part B Arizona Annual Performance Report for FFY 2011 54 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Data Source The data were collected through the October 1, 2011, Child Count report and are the same as the State’s data reported under section 618, Table 3, Part B, Individuals with Disabilities Education Act Implementation of FAPE Requirements. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because ESS collected, maintained, and reported the October 1, 2011, child count data and the placement data using internal edit checks. The State requires PEAs to assure their data is accurate and reliable by having them submit signed verification letters. Explanation of Progress and Slippage That Occurred for FFY 2011 Arizona exceeded the target for Indicator 5A, but did not meet the targets for Indicators 5B and 5C. Slight progress was made from FFY 2010 on Indicators 5A, from 60% to 60.4%, and 5B, from 14.8% to 14.68%. Slight slippage occurred on Indicator 5C, from 2.6% to 2.8%. The ESS/Comprehensive System of Personnel Development (CSPD) unit sponsors the Autism Spectrum Disorders project, which is designed to expand the number of Arizona educators who are skilled in meeting the needs of students with autism. The two-year training program builds skills in evaluation, instructional planning and strategies, and environmental design to address the academic, communication, behavioral, and social needs of students. The program, when implemented at school sites, improves the decision-making abilities of IEP team members regarding the least restrictive environment (LRE) for students with autism. The ADE/ESS program specialists review least restrictive environment data with school administrators at each PEA in the State on an annual basis. If a PEA’s data does not meet State targets for LRE, then the concern is discussed with the administrator. If a PEA is in Year 4 of the monitoring cycle, then a self assessment in this area may be one of the monitoring activities. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Conduct interviews with special education directors and site administrators about available service delivery models and LRE data as components of all on-site monitorings (documentation requires an individualized Sub-Activities (Objectives or Action Steps) a) Gather data from interview responses and supporting documentation of placement decisions and service delivery models for students with IEPs b) Revise interview questions and documentation requirements based on results related to Timeline Complete Activity completed 6/1/11. Interviews were conducted with special education and site administrators during all on-site monitorings to discuss service delivery models. Results of the interviews and the supporting documentation will be analyzed. Activity completed from 7/1/11 to 6/30/12. Projected Resources (Planned) 10/1/10– 6/30/11 ADE/ESS Directors and Specialists 7/1/11– 6/30/12 ADE/ESS Directors and Specialists Interview question results and documentation of decision- Part B Arizona Annual Performance Report for FFY 2011 55 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 decision-making process for placement) 2) Provide school teams with autism training designed to improve teams’ ability to make LRE decisions that will maintain or increase the amount of time students with autism are served in general education settings LRE targets c) Conduct revised interviews and gather supporting documentation a) Collect baseline data (FFY 2009 service codes) for students with autism enrolled in schools that participate in autism training b) Provide training that, if components are implemented, can increase teams’ decision-making abilities to maintain or increase the amount of time students with autism are served in general education settings by giving teams the:  knowledge of traits and characteristics of students with autism  ability to apply instructional supports and practices consistent with the needs of students with autism  skills to decrease ineffective behaviors of students with autism making process were analyzed. Outcomes demonstrated a high level of both compliance and understanding related to LRE. No revisions were made. Activity completed 12/1/11. Thirteen teams participated in the ASD Year 1 2011–2013 cohort, representing 197 students receiving services under the category of autism. Within this group, approximately 23% received services in the A service code, 10% in the B service code, and 67% in the C service code. Activity completed from 7/1/11 to 6/30/12. The training was completed for the ASD Year 1 cohort (13 teams). An overview of autism was provided to the teams during the first training session. Teams were provided with training materials that allowed them to make informed decisions. Additional supports include supplying teams with the Ziggurat Model (a pyramid construct) that is a tangible reference to address many of the characteristics of students with autism as well as provide a model to assist teams in “seeing” the autism in students. A team task was to complete the Ziggurat worksheet; 11 of 13 teams completed this task. Supports for completing the Ziggurat worksheet and understanding the model were provided with at least one face-to-face visit per team from the project 7/1/12– 6/30/13 ADE/ESS Directors and Specialists 7/1/11– 6/30/13 PEAs’ FFY 2009 service code data 7/1/11– 6/30/13 CSPD Staff Team Training Materials School Administrator General Education Teacher(s) Special Education Teacher(s) Part B Arizona Annual Performance Report for FFY 2011 56 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 c) Provide capacity building coaching support to each team to increase team effectiveness during training d) Provide training that, if implemented, would give teams the necessary processes to plan, execute, and evaluate the effectiveness of their activities coordinator, written feedback on completed products, and for two teams, additional telephone conference support. Of the teams, eight of 13 submitted a completed Reinforcer Survey, and of those, all used that data along with their Ziggurat information to develop an IEP goal, chose an instructional approach, and determine a reinforcement and reinforcement schedule. Five teams submitted data indicating that the instructional approach was achieving the desired results. Activity completed from 7/1/11 to 6/30/12. 7/1/11– 6/30/13 Coaching was provided to the ASD Year 1 2011–2013 cohort and the ASD Year 2 2012–2014 cohort. The CSPD coaching coordinator provided training, forms, and feedback to the coaches. Twelve of the 13 coaches attended a Coaching Refresher Training. Feedback provided by the teams for all the team trainings indicated that 81% of teams agreed or strongly agreed with the statement that “the team improves their ability to work collaboratively, to think deeper, to plan, and to reflect based on interactions with the coach.” Activity completed from 7/1/11 to 6/30/12. Of the 13 ASD Year 1 cohort, 84% completed all required implementation activities. CSPD Coaching Coordinator Coaching Refresher Training Coaching Presenter Coaching Materials 7/1/11– 6/30/13 CSPD Staff Implementati on Portfolios Training was provided to the ASD Year 2 cohort (seven teams) as follows: Social Skills/Student-level Part B Arizona Annual Performance Report for FFY 2011 57 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 activities: When presented with training on strategies as well as an evidence-based tool to measure social skills for students with autism, six of the teams used the evaluation process to develop IEP goals that were supported by the data collected. Of those teams that developed goals, five teams provided data on the effectiveness of the strategy selected to teach the goals to the target student(s). Supports for this effort included site visits by the project coordinator (4 per year), phone consultations, and review of implementation activities as well as feedback to the teams on their progress. Sensory Integration/Schoolwide activities: After being presented with training on strategies and guidance regarding assessment of sensory needs for students with autism, six of the teams developed a Sensory Strategy Action Plan that targeted their team’s local needs for addressing sensory issues within their schoolbased system. Four of those teams chose to develop visually based supports for their school. Professional development (PD) plans were developed for dissemination of the planned supports. Of those PD plans, three teams provided data in regard to the effectiveness of the professional development (using the PD tools provided). Supports for this effort included site visits by the project coordinator (four per year), phone consultations, and review of implementation activities, as well as feedback Part B Arizona Annual Performance Report for FFY 2011 58 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 to the teams on their progress. e) Collect FFY 2010 and FFY 2011 census data for students with autism enrolled in schools that participate in autism training f) Aggregate and disaggregate service code data for students with autism enrolled in schools that participate in autism training Activity completed 10/1/11. All teams submitted the initial 10/1/11 census data, including the LRE code for students with autism. The teams will submit the 10/1/12 census and LRE data for students with autism; this will be reported in the FFY 2012 APR. Teams will meet in January 2013 to complete this activity. The activity will be reported in the FFY 2012 APR. 7/1/11– 6/30/13 CSPD staff PEAs’ FFY 2010 and FFY 2011 census data 7/1/11– 6/30/13 CSPD Staff PEA Staff Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 59 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 6: Preschool LRE Percent of children aged 3 through 5 with IEPs attending a: A. Regular early childhood program and receiving the majority of special education and related services in the regular early childhood program; and B. Separate special education class, separate school or residential facility. (20 U.S.C. 1416 (a)(3)(A)) Measurement A. Percent = [(# of children aged 3 through 5 with IEPs attending a regular early childhood program and receiving the majority of special education and related services in the regular early childhood program) divided by the (total # of children aged 3 through 5 with IEPs)] times 100. B. Percent = [(# of children aged 3 through 5 with IEPs attending a separate special education class, separate school or residential facility) divided by the (total # of children aged 3 through 5 with IEPs)] times 100. Overview of Issue / Description of System or Process The Arizona Department of Education/Early Childhood Education (ADE/ECE) is responsible for the administration of the early childhood special education program (Part B, Section 619) for children aged 3– 5 who are not yet eligible for kindergarten. The ADE/ECE collects preschool least restrictive environment (LRE) data for children aged 3–5 receiving special education services through the Student Accountability Information System (SAIS) based on each year’s October 1 public education agency (PEA) child count. Partnerships with other early childhood programs, such as First Things First, Head Start, Career and Technical Education, Title I, and Community Child Care, are promoted through the Early Childhood Quality Improvement Process (ECQUIP), a continuous improvement process to increase quality among PEAs’ early childhood programs that receive early childhood State funds. Data Data Source The data were collected through the October 1, 2011, Child Count report and are the same as the State’s data reported under section 618, Table 3, Part B, Individuals with Disabilities Education Act Implementation of FAPE Requirements. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because ESS collected, maintained, and reported the October 1, 2011, child count data and the placement data using internal edit checks. The State requires PEAs to assure their data is accurate and reliable by having them submit signed verification letters. Part B Arizona Annual Performance Report for FFY 2011 60 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Baseline Data for FFY 2011 The table below displays the baseline data for Indicators 6A and 6B and the total number of children aged 3–5 (including five year olds in kindergarten) reported on the October 1, 2011, child count. FFY 2011 (2011–2012) Indicator 6 Baseline Data 6A 6B Number of children aged 3–5 reported for 10/1/11 Child Count 15,235 15,235 Number of children 7,315 7026 48.01% 46.11% Percentage of children Discussion of Baseline Data for FFY 2011 The baseline data show that 48.01% of children aged 3–5 were attending a regular early childhood program and receiving the majority of their special education and related services in the regular early childhood or kindergarten program. The baseline data also show that 46.11% of children aged 3–5 were attending a separate special education class, separate school, or residential facility. Proposed Targets for FFY 2012 Targets were set based on the FFY 2011 baseline data and input from the stakeholder groups. Measurable and Rigorous Targets FFY Indicator 6A Indicator 6B 48.5% 45.5% 2012 Proposed Improvement Activities for FFY 2012 The following are improvement activities for FFY 2012 to improve results for Indicator 6, Preschool LRE. Primary Activity (GOAL) 1) Provide statewide baseline data to school districts Sub-Activities (Objectives or Action Steps) a) Provide school districts access to individual baseline data for Early Childhood Quality Improvement Process (ECQUIP) teams b) Provide school districts baseline data with 5-yearold kindergarten student data extrapolated to view preschool LRE data Timeline Complete Projected Resources (Planned) 7/1/12– 6/30/13 ADE/Early Childhood Special Education (ECSE) 7/1/12– 6/30/13 ADE/ECSE Part B Arizona Annual Performance Report for FFY 2011 61 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 2) Continue to promote Head Start programs as least restrictive environment options 3) Promote First Things First (FTF)– funded programs as a least restrictive environment option 4) Provide professional development opportunities that promote the least restrictive environment for preschool students with IEPs 5) Provide information to PEAs about preschool least restrictive environment data collection and reporting a) Provide resources through newsletters to continue the collaboration with Head Start and Arizona Early Intervention Program (AzEIP) to promote LRE and highlight Head Start’s mandate for 10% enrollment of children with disabilities a) Provide information to school district programs that receive FTF scholarship grants through FTF/Early Childhood Education Quality Mentors a) Continue collaboration with Arizona Council of Exceptional Children/Division of Early Childhood (CEC/DEC) for spring conference and presentations at the Exceptional Student Services (ESS) Director’s Institute a) Post OSEP letter regarding preschool LRE on Web site (previously sent to all early childhood special education programs) b) Conduct Webinars and professional development regarding preschool LRE data c) Include LRE information in new Help for Early Learning Professionals (HELP) manual and on Web site 7/01/12– 6/30/13 ADE/ECSE Head Start State Collaboration Office Arizona Head Start Association 7/01/12– 6/30/13 7/1/12– 6/30/13 ADE/ECSE FTF/Early Childhood Education Quality Mentors ADE/ECSE CEC/DEC 7/1/12– 6/30/13 ADE/ECSE 7/1/12– 6/30/13 ADE/ECSE, ESS Data Management Specialist ADE/ECSE 7/1/12– 6/30/13 Part B Arizona Annual Performance Report for FFY 2011 62 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 7: Preschool Outcomes Percent of preschool children aged 3 through 5 with IEPs who demonstrate improved: A. Positive social-emotional skills (including social relationships); B. Acquisition and use of knowledge and skills (including early language/communication and early literacy); and C. Use of appropriate behaviors to meet their needs. (20 U.S.C. 1416 (a)(3)(A)) Measurement Outcomes A. Positive social-emotional skills (including social relationships); B. Acquisition and use of knowledge and skills (including early language/communication and early literacy); and C. Use of appropriate behaviors to meet their needs. Progress Categories for A, B, and C a. Percent of preschool children who did not improve functioning = [(# of preschool children who did not improve functioning) divided by (# of preschool children with IEPs assessed)] times 100. b. Percent of preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers = [(# of preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers) divided by (# of preschool children with IEPs assessed)] times 100. c. Percent of preschool children who improved functioning to a level nearer to same-aged peers but did not reach it = [(# of preschool children who improved functioning to a level nearer to sameaged peers but did not reach it) divided by (# of preschool children with IEPs assessed)] times 100. d. Percent of preschool children who improved functioning to reach a level comparable to sameaged peers = [(# of preschool children who improved functioning to reach a level comparable to same-aged peers) divided by (# of preschool children with IEPs assessed)] times 100. e. Percent of preschool children who maintained functioning at a level comparable to same-aged peers = [(# of preschool children who maintained functioning at a level comparable to same-aged peers) divided by (# of preschool children with IEPs assessed)] times 100. Summary Statements for Each of the Three Outcomes Summary Statement 1: Of those preschool children who entered or exited the preschool program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 6 years of age or exited the program. Measurement for Summary Statement 1: Percent = # of preschool children reported in progress category (c) plus # of preschool children reported in category (d) divided by [# of preschool children reported in progress category (a) plus # of preschool children reported in progress category (b) plus # of preschool children reported in progress category (c) plus # of preschool children reported in progress category (d)] times 100. Summary Statement 2: The percent of preschool children who were functioning within age expectations Part B Arizona Annual Performance Report for FFY 2011 63 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 in each Outcome by the time they turned 6 years of age or exited the program. Measurement for Summary Statement 2: Percent = # of preschool children reported in progress category (d) plus [# of preschool children reported in progress category (e) divided by the total # of preschool children reported in progress categories (a) + (b) + (c) + (d) + (e)] times 100. Overview of Issue / Description of System or Process The Arizona State Board of Education approved the use of a single, Web-based assessment, Teaching Strategies GOLD, in August 2010. All public education agencies (PEAs) serving preschool children with IEPs are required to use this assessment. GOLD was implemented statewide during school year 2011– 2012 and replaced the four assessments previously used to report preschool outcomes. Use of a single, Web-based assessment that is designed for children birth through kindergarten has enabled Arizona to:  capture the progress of preschool children at all developmental stages;  provide programs/districts with real-time access to meaningful data that will guide instructional and programmatic decisions;  easily communicate progress to parents, related service providers, and other school staff;  retain data on children as they transfer to other programs and districts in the State;  consolidate professional development and technical assistance;  increase collaboration between Head Start, private child care, general education preschool, and special education preschool programs. Primary responsibility for data collection and submission belongs to the professional with the greatest degree of interaction with the child. Families and all relevant professionals supporting each child contribute to the body of information used to determine progress ratings within GOLD. Public education agencies are required to submit data 3 times per school year using a secure Web-based system hosted by Teaching Strategies, which is accessible by the Arizona Department of Education Early Childhood Education staff. GOLD provides the ECSE unit with full, real-time access to district-level data, allowing targeted technical assistance to schools. An Early Childhood Data Collaborative (ECDC) stakeholders group meets regularly to address issues and to help the ECSE unit develop appropriate technical assistance. The change to Teaching Strategies GOLD as the single assessment instrument in Arizona is expected to result in an improvement in data reliability as well as improved outcomes for preschool children. As a result of this change to GOLD, new baseline data has been established and is reported in the FFY 2011 APR. Data Data Source PEAs reported FFY 2011 preschool assessment data using Teaching Strategies GOLD, a Web-based data collection system. Sampling was not used for this Indicator as all preschool children with disabilities have their entry status and exit status assessed. Additionally, this assessment system is statewide and inclusive of all children in state-funded preschool programs, as well as all Head Start programs and one Tribal Head Start program. Instruments Part B Arizona Annual Performance Report for FFY 2011 64 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 All early childhood programs must administer the one assessment tool approved by the Arizona State Board of Education for ongoing progress monitoring assessment. Teaching Strategies is the publisher of the GOLD assessment. Trainings were provided to Arizona educators and administrators by Teaching Strategies staff in 2011 and 2012. These trainings continue to take place throughout the State for new users. Online training modules, including interrater reliability certification courses, are available through the GOLD subscriptions. Ongoing technical assistance has been provided by the Arizona Department of Education/Early Childhood Education (ADE/ECE) staff. ADE/ECE staff also train PEAs on a continual basis regarding the use of progress monitoring tools to drive instruction and program improvement. Further guidance is provided through Arizona’s Early Childhood Assessment System for Ongoing Progress Monitoring (http://www.azed.gov/early-childhood/files/2012/02/arizonas-early-childhood-assessment-system-for-ongoing-progress-monitoring-2012-13.pdf) and monthly GOLD Nugget publications (http://www.azed.gov/early-childhood/2011/11/14/teaching-strategies-gold/). Data Analysis The preschool outcome data was analyzed by Teaching Strategies. Valid and Reliable Data The Arizona Department of Education/Early Childhood Special Education (ADE/ECSE) assures the validity and reliability of the early childhood outcomes (ECO) data by offering professional development on recording quality observations of children. PEAs have online access to an interrater reliability certification course provided by Teaching Strategies. Training is provided to administrators on accessing the Documentation Status Report that provides information on the number of observations used to assess the child and assign ratings. The Early Childhood Quality Improvement Process (ECQUIP) also incorporates the early childhood outcomes assessment process in the districts’ self-assessment rubric. The ECQUIP Manual is available at http://www.azed.gov/early-childhood/files/2011/11/ecquip-binder-revision-8.29.pdf. Baseline Data for FFY 2011 The table below displays the number and percentage of preschool children who exited in FFY 2011 in each progress category and the results of the summary statement calculations. FFY 2011 Positive SocialEmotional Skills # of children a. Children who did not improve functioning b. Children who improved functioning but not sufficient to move nearer to functioning comparable to same age peers c. Children who improved functioning to a level nearer to same-aged peers but did not reach it d. Children who improved functioning to reach a level comparable to same-aged peers e. Children who maintained functioning at a level comparable to same-aged peers % of children Acquiring and Using Knowledge and Skills # of children % of children Taking Appropriate Action to Meet Needs # of children % of children 205 5.8 174 4.9 272 7.7 528 15.0 522 14.8 433 12.3 738 20.9 805 22.8 610 17.3 1,121 31.8 1,227 34.8 1,148 32.6 933 26.5 797 22.6 1,062 30.1 Part B Arizona Annual Performance Report for FFY 2011 65 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 3,525 Total 100.0% 3,525 100.0% 3,525 100.0% Summary Statements 1. Of those children who entered or exited the program below age expectations, the percent who substantially increased their rate of growth by the time they exited. 71.7 74.5 71.4 2. Percent of children who were functioning at a level comparable to same-aged peers by the time they exited. 58.3 57.4 62.7 Discussion of Baseline Data for FFY 2011 Baseline data indicate that 71.7%, 74.5% and 71.4% of children who entered the program below age expectations substantially increased their rate of growth by the time they exited the program in positive social-emotional skills, acquiring and using knowledge and skills, and taking appropriate action to meet their needs, respectively. Data also indicate that, by the time they exited, 58.3%, 57.4% and 62.7% of children were functioning at a level comparable to same-aged peers in positive social-emotional skills, acquiring and using knowledge and skills and taking appropriate action to meet their needs, respectively. Proposed Targets for FFY 2012 Targets were set based on the FFY 2011 baseline data and input from the stakeholder groups. Measurable and Rigorous Targets Positive SocialEmotional Skills Acquiring and Using Knowledge and Skills Taking Appropriate Action to Meet Needs 1. Of those children who entered or exited the program below age expectations, the percent who substantially increased their rate of growth by the time they exited. 72.2 75.0 71.9 2. Percent of children who were functioning at a level comparable to same-aged peers by the time they exited. 58.8 57.9 63.2 FFY 2012 Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Implement new preschool assessment (Teaching Strategies GOLD) statewide Sub-Activities (Objectives or Action Steps) a) Identify and implement ADE infrastructure modifications Timeline Complete Activity completed from 7/1/11 to 6/30/12. ADE Information Technology worked with Early Childhood Special Education and Teaching Strategies to determine data requirements and processes for importing Projected 7/1/10– 6/30/12 Resources (Planned) ADE/ECSE Director and Specialist ADE IT Part B Arizona Annual Performance Report for FFY 2011 66 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) Provide regional trainings on the use of Teaching Strategies GOLD 2) Provide professional development activities around quality assessment practices 3) Increase the percentage of PEAs that collect and report timely preschool assessment data a) Provide professional development “How to Improve the Quality of your Ongoing Progress Monitoring Data” within areas of need as identified through the Early Childhood Quality Improvement Practices (ECQUIP) process and upon request of districts a) Crosscheck child count data with PEA preschool assessment data b) Notify PEAs if preschool assessment data are not submitted on time 4) Establish an Early Childhood Data Collaborative (ECDC) a) Obtain stakeholder input at quarterly meetings during transition to Teaching Strategies GOLD b) Use stakeholder input to guide development of Statelevel policies and procedures related to implementation of Teaching Strategies GOLD State-level data to ADE Student Accountability and Information System (SAIS). Activity completed from 7/1/11 to 6/30/12. Nine two-day trainings on Teaching Strategies GOLD were held throughout Arizona and were attended by 256 educators. Activity completed from 7/1/11 to 6/30/12. 1/1/11– 6/30/12 ADE/ECSE Teaching Strategies GOLD 7/1/10– 6/30/13 ADE/ECSE Director and Specialist Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 Child count data were crosschecked with PEA preschool assessment data. The number of PEAs not reporting on time decreased from 34 in FFY 2010 to 11 in FFY 2011. Activity completed from 7/1/11 to 6/30/12. ADE/ECSE Director and Specialist 7/1/10– 6/30/13 Eleven PEAs received notification about submitting data within timelines. Activity completed from 7/1/11 to 6/30/12. ADE/ECSE Director and Specialist 7/1/11– 6/30/13 ADE/ECSE Fifteen “How to Improve the Quality of your Ongoing Progress Monitoring Data” trainings were held throughout Arizona and were attended by 319 educators. Four meetings were held with ECDC members representing PEAs and Head Start programs. Activity completed from 7/1/11 to 6/30/12. Input from ECDC members created guidance in the use of GOLD. This guidance was communicated to PEAs through the Arizona’s Early Childhood Assessment EC Data Collaborati ve 7/1/11– 6/30/13 ADE/ECSE EC Data Collaborati ve Part B Arizona Annual Performance Report for FFY 2011 67 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 5) Develop and disseminate publications statewide to serve as ongoing guidance and communication a) Develop Arizona’s Early Childhood Assessment System manual (guidance document) and disseminate statewide b) Develop monthly bulletins to alert PEAs about updates and new guidance for preschool assessment instrument System manual and a monthly electronic publication, GOLD Nuggets. Both documents were disseminated via email and Web site posting. Activity completed from 7/1/11 to 6/30/12. 7/1/11– 6/30/13 ADE/ECSE The Arizona’s Early Childhood Assessment System was created, posted on the ADE Web site, and disseminated to PEAs via email. Activity completed from 7/1/11 to 6/30/12. 7/1/11– 6/30/13 ADE/ECSE Eight issues of GOLD Nuggets were disseminated to PEAs via email and Web site posting. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table The State must report progress data and actual target data for FFY 2011 with the FFY 2011 APR. State’s Response In the FFY 2011 APR, Arizona is reporting baseline data for FFY 2011 and target data for FFY 2012, based on first-year data using the Teaching Strategies GOLD assessment. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 The targets for FFY 2012 were revised based on the new baseline data (see above). Part B Arizona Annual Performance Report for FFY 2011 68 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: FAPE in the LRE Indicator 8: Parent Involvement Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities. (20 U.S.C. 1416 (a)(3)(A)) Measurement Percent = [(# of respondent parents who report schools facilitated parent involvement as a means of improving services and results for children with disabilities) divided by the (total # of respondent parents of children with disabilities)] times 100. Target Data for FFY 2011 FFY Measurable and Rigorous Target 2011 60% Actual Target Data for FFY 2011 number of respondent parents who report schools facilitated parent involvement as a means of improving services and results for children with disabilities total number of respondent parents of children with disabilities 4,164 6,898 Actual Target Data for FFY 2011 60.4% 4,164  6,898 = 0.6036  100 = 60.4% Arizona met the target. Data Data Source The data are taken from the Arizona Parent Involvement Survey. Arizona uses a 25-question parent survey developed by the National Center for Special Education Accountability Monitoring (NCSEAM). The survey is the same survey as the one that has been used for past years and has not been revised. Data Description Part B Arizona Annual Performance Report for FFY 2011 69 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The Arizona Parent Involvement Survey uses a Web-based data collection system to collect confidential demographic information and parental responses to the 25-question NCSEAM rating scale. A paper version of the survey is available in English and Spanish, and in large font in both languages. Parents complete the demographic data and 25 survey items. The data are analyzed using WINSTEPS statistical software. Following NCSEAM guidelines, a threshold score of 600 has been established for a positive response to the item “The school explains what options parents have if they disagree with a decision of the school.” The instrument measure implies that agreement with this threshold item indicates high likelihood of agreement with items located “under” it on the scale. A score of 600 is required for any parent’s survey response to be considered positive. Sampling Procedures Each school year a new cohort of PEAs is selected to administer the survey. The cohort is composed of PEAs: a) in the assigned year of the ESS monitoring cycle, or b) with a student population of 50,000 or greater. Every parent within these PEAs who has a child with an individualized education program (IEP) is given an opportunity to complete the survey via either the Web-based data collection system or mail. Thus, within the cohort, a census of parents is given the opportunity to complete the survey. The ADE/ESS ensures all newly opened PEAs (typically, charter schools) are included in a cohort and administer the parent survey. The use of these procedures will allow the State to meet the requirement to report on each PEA at least once during the SPP cycle. The response data are representative of Arizona’s demographics by race/ethnicity, as shown in Table 8.1, and by child age group, as shown in Table 8.2 (below). The data set includes charter schools, unified districts, union high school districts, and elementary districts. In addition, urban and rural schools that are distributed throughout the geographical regions of the State are represented. Valid and Reliable Data Arizona ensures the data are valid and reliable by offering extensive ongoing technical assistance to PEAs. Initial survey instructions detail the steps that PEAs must follow to distribute survey instructions and confidential user codes/passwords to all parents who have a child with a disability. PEAs are given surplus user codes/passwords to have ready for transfer students. PEAs also receive guidance on how to maximize their parental response and involvement rates as demonstrated in the improvement activities. Table 8.1 Comparison of Parent Responses by Race / Ethnicity to State Special Education Population Percentage of Special Education Population (Child Count) Race/Ethnicity of Child of Parent Respondent Number of Responses Percentage of Responses Number of Special Education Population (Child Count) Hispanic/Latino of Any Race 2,763 40.06% 52,234 41.07% American Indian or Alaska Native 353 5.12% 8,428 6.63% Asian 98 1.42% 1,782 1.40% Part B Arizona Annual Performance Report for FFY 2011 70 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Black or AfricanAmerican 426 6.18% 8,163 6.42% Native Hawaiian or Other Pacific Islander 45 0.65% 227 0.18% White 2,684 38.91% 53,543 42.09% Two or More Races 397 5.76% 2,821 2.22% Total 6,898 127,198 Note: 132 respondents did not indicate the race/ethnicity of their child. Table 8.1 shows that the response rate by race/ethnicity is in alignment with the race/ethnicity of children in special education in Arizona for all racial/ethnic populations. It should be noted that the percentage of respondents who selected the two or more races is significantly higher than the State race/ethnicity statistics for that group. Table 8.2 Comparison of Parent Responses by Child Age Group to State Special Education Population Percentage of Special Education Population (Child Count) Child Age Group Number of Responses Percentage of Responses Number of Special Education Population (Child Count) Ages 3–5 1,176 17.05% 15,235 11.98% Ages 6–13 3,808 55.20% 73,617 57.88% Ages 14–22 1,690 24.50% 38,346 30.15% Total 6,898 127,198 Note: 224 respondents did not indicate the age of their child. Table 8.2 shows the response rate is in alignment with the age group statistics for parents of children ages 6–13. The response rate is higher than the age group statistics for parents of children aged 3–5. Conversely, the response rate is slightly lower than the age group statistics for parents of children aged 14–22. The significant gain for the group aged 14–22 that was achieved in FFY 2010 was maintained in FFY 2011 through action steps to target parents of those students. Explanation of Progress That Occurred for FFY 2011 Arizona made a gain of three percentage points from FFY 2010 (57%) to FFY 2011 (60%). This change may be attributed to frequent contact by the ADE/ESS Arizona Find Coordinator and ESS specialists with participating PEAs and families. ESS staff offered consultation, training, print and electronic special education resources, and toll-free assistance to stakeholders throughout Arizona. The technical Part B Arizona Annual Performance Report for FFY 2011 71 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 assistance, founded on principles of effective parent involvement that align with survey questions, may yield a higher measurement of satisfactory parental involvement when put into action by PEAs. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Evaluate PEA’s feedback of the parent involvement survey process as a means of improving distribution to families and use of results to enhance parent involvement 2) Increase opportunities for PEAs and parents to gain knowledge about the parent involvement survey and related family involvement projects Sub-Activities (Objectives or Action Steps) a) Develop and administer a survey to PEAs that conducted the parent involvement survey during the 20052011 SPP b) Examine PEA survey results to improve the parent involvement survey process and to advise PEAs on strategies for using the parent survey results to improve family involvement a) Plan and develop a new ESS parent involvement survey Web site combining current links with access to researchbased family involvement literature and effective projects b) Test, revise, and launch the new ESS parent involvement survey Web site Timeline Completed Activity completed from 7/1/11 to 6/30/12. Survey methodologies were researched and questions were drafted. The survey will be administered during the first half of FFY 2012 to solicit special education directors’ feedback about the survey process and parent involvement. Analysis and action plans will be completed during the second half of FFY 2012. Activity completed from 7/1/11 to 6/30/12. Research-based documents were collected and effective projects were identified. Discussions with other ADE units that have parent involvement components were initiated; shared knowledge of applicable family engagement strategies will be incorporated in the Web site. The Web site will be launched in FFY 2012. Projected 7/1/10– 6/30/13 Resources (Planned) Arizona Find Coordinator ADE/ESS PIN Specialists ADE/ESS Program Specialists 7/1/11– 6/30/13 Arizona Find Coordinator ADE/ESS PIN Specialists 7/1/10– 6/30/13 ADE/ESS Program Specialists Arizona Find Coordinator ADE/ESS PIN Specialists 7/1/11– 6/30/13 Arizona Find Coordinator ADE/ESS PIN Specialists Part B Arizona Annual Performance Report for FFY 2011 72 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 73 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Disproportionality Indicator 9: Racial / Ethnic Disproportionality Percent of districts with disproportionate representation of racial and ethnic groups in special education and related services that is the result of inappropriate identification. (20 U.S.C. 1416 (a)(3)(C)) Measurement Percent = [(# of districts with disproportionate representation of racial and ethnic groups in special education and related services that is the result of inappropriate identification) divided by the (# of districts in the State)] times 100. Data Data Source The ADE/ESS collected the data from the PEAs through the October 1, 2011, child count report. The data are the same as the data collected and reported on Table 1, Report of Children with Disabilities Receiving Special Education under Part B of the Individuals with Disabilities Education Act, As Amended, for all children with disabilities aged 6–21 served under IDEA. Note that the source of this data is from FFY 2011. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because it collected, maintained, and reported the October 1, 2011, child count data using internal edit checks. In addition, the State requires the PEAs to assure data accuracy and reliability by submitting a signed verification letter. Definition of Disproportionate Representation Disproportionate Representation Over representation Weighted Risk Ratio Minimum n Size Target Racial/Ethnic Group Minimum n Size Racial / Ethnic Groups in Special Education and Related Services ≥ 3.00 30 30 Methodology The data were analyzed using Statistical Analysis Software (SAS) to produce a weighted risk ratio (WRR) that identified all racial/ethnic groups for all PEAs in the State. Data for over representation were examined. PEAs with a cell size of 30 or more students in the target racial/ethnic group and in the other racial/ethnic groups and that met the weighted risk ratio criteria for over representation were flagged for a Part B Arizona Annual Performance Report for FFY 2011 74 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 review of policies, procedures, and practices by the State. PEAs with a lower cell size in the target groups were not flagged because false positives were identified as a function of the small number rather than as a result of noncompliant policies, procedures, and practices. Arizona included the total number of PEAs in the State (599) in the denominator. Of the 599 PEAs, 15 were eliminated from the analyses because a weighted risk ratio could not be calculated for any racial/ethnic group. Arizona’s Procedures to Determine if Disproportionate Representation Is the Result of Inappropriate Identification Arizona ensures that PEAs’ policies, procedures, and practices are reviewed as required by 34 CFR §§ 300.173, 300.600(d)(3), and 300.602(a). The data are analyzed annually and PEAs may be flagged each year for over representation, according to the State’s definition. When a PEA is flagged, then the policies, procedures, and practices of the PEA are reviewed annually to determine if the disproportionate representation is the result of inappropriate identification. Arizona’s Review of PEAs’ Policies and Procedures On an annual basis, Arizona requires all PEAs to have special education policies and procedures in compliance with the requirements of 34 CFR § 300.111, § 300.201, and § 300.301 through § 300.311 prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. Each year, if the PEA makes any changes to the policies and procedures, the PEA must resubmit them to the State for review and acceptance. Each year, if the PEA does not make any changes to the policies and procedures, the PEA must submit a Statement of Assurance that says: “The PEA has not altered or modified the policies and procedures implementing the State and Federal requirements for services to children with disabilities previously submitted to and accepted by the Arizona Department of Education, Exceptional Student Services. If the PEA proposes to alter or modify the policies and procedures previously submitted to the Exceptional Student Services, the PEA must resubmit the policies and procedures to the Exceptional Student Services for review and acceptance.” In addition, the PEAs that are flagged for disproportionate representation must submit their policies and procedures related to child find, evaluation, and eligibility to an ADE/ESS specialist for review. Arizona’s Review of PEAs’ Practices On an annual basis, Arizona calculates the WRR for each PEA and uses the data as a trigger to flag PEAs with disproportionate representation. If a PEA is flagged, then an investigation of the practices is required to determine whether the disproportionate representation is a result of inappropriate identification. Review of practices when a PEA is flagged for over representation the first year:  The ESS specialist reviews current monitoring data, if applicable.  The PEA conducts a self assessment of the agency’s child find, evaluation, and eligibility practices to determine whether the disproportionate representation is a result of inappropriate identification. The self assessment consists of a series of questions requiring narrative responses and a review of student files using the State’s monitoring forms. The ADE/ESS specialists conduct on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews.  Upon completion of the self assessments, the PEAs have the option to begin immediately revising their policies, procedures, and practices related to child find, evaluation, and eligibility and to correct any noncompliance. No more than 60 days after completion of the self assessment, the ESS specialists then interview the special education administrators and review Part B Arizona Annual Performance Report for FFY 2011 75 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 student files via on-site visits and/or desk audits to verify correction of instances of any noncompliance, including child specific, and to ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. Review of practices when a PEA is flagged for over representation for two or more consecutive years:  If the PEA did not have disproportionate representation as a result of inappropriate identification the first year, then the ESS program specialist: o Reviews current monitoring data, if applicable, and; o Validates the prior year’s self assessment by reviewing a sample of student files.  If the PEA had disproportionate representation as a result of inappropriate identification the first year, then the PEA is required to: o Review current monitoring data, if applicable; o Review the prior year’s self assessment and describe the issues identified; o Describe the steps taken to resolve those issues; o Describe any current concerns regarding possible inappropriate identification; o Describe the resources and technical assistance used to help address the issues related to disproportionate representation within the agency; and o Review individual student files using the State’s monitoring forms:  The ADE/ESS specialists conduct on-site visits and/or desk audits during the file reviews to validate the decisions made by the PEAs.  The ESS specialists verify correction of instances of any noncompliance, including child specific, through on-site visits and/or desk audits.  The ESS specialists ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. When Arizona makes findings of noncompliance as a result of the review of policies, practices and procedures, the PEA has one year from the date of written notification from the State to correct the noncompliance. Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 0% Actual Target Data for FFY 2011 0% Arizona met the target. PEAs with Disproportionate Representation of Racial and Ethnic Groups That Was the Result of Inappropriate Identification Year Total Number of PEAs* Number of PEAs with Disproportionate Number of PEAs with Disproportionate Representation of Racial and Ethnic Groups That Percent of PEAs Part B Arizona Annual Performance Report for FFY 2011 76 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Representation FFY 2011 (2011– 2012) 599 Was the Result of Inappropriate Identification 0 0 0.00% *Arizona included the total number of PEAs in the denominator. PEAs with Disproportionate Representation of Racial and Ethnic Groups Arizona identified zero PEAs with disproportionate representation of racial and ethnic groups in special education and related services (0.0%). PEAs with Disproportionate Representation of Racial and Ethnic Groups That Was the Result of Inappropriate Identification Arizona identified zero PEAs with disproportionate representation of racial and ethnic groups in special education and related services that was the result of inappropriate identification (0.0%). Table 9.1 PEAs with Over Representation by Racial / Ethnic Group Indicator 9 – Over Representation # of PEAs flagged for over representation # of PEAs found to have disproportionate representation (over representation) as a result of inappropriate identification Hispanic/Latino of Any Race American Indian or Alaska Native Asian Black or African American Native Hawaiian or Other Pacific Islander White Two or More Races As shown in Table 9.1, no PEAs were flagged for over representation. Arizona made no findings of noncompliance as a result of the review of child find, evaluation, and eligibility policies, procedures, and practices. Part B Arizona Annual Performance Report for FFY 2011 77 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Explanation of Progress That Occurred for FFY 2011 The FFY 2011 results (0.0%) are the same as those for FFY 2010 (0.0%). The Exceptional Student Services division has placed importance on improving the child find and evaluation process in a number of ways. The ESS directors and specialists review the PEAs’ data related to the SPP/APR Indicators annually. When a district or charter is identified as at risk for over representation, the ESS specialist notifies the PEA and offers information and support during a self-guided drill down. The monitoring process also helps the public schools to identify weaknesses with the child find and evaluation areas and to strengthen them through a comprehensive corrective action plan. Additionally, the Parent Information Network specialists (PINS) disseminate information to both parents and PEAs about child find requirements. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Provide support for PEAs that are flagged as at risk for disproportionate representation with a WRR ≤ 0.40 for under representation and ≥ 2.5 for over representation Sub-Activities (Objectives or Action Steps) a) Analyze data on an annual basis to flag PEAs that are at risk for disproportionate representation b) On an annual basis, notify PEAs that are flagged as at risk for disproportionate representation c) On an annual basis, provide assessment tools and resources to PEAs that are flagged as at risk to conduct a root cause analysis 2) Provide support for PEAs that are flagged for disproportionate representation with a WRR ≤ 0.30 for under representation and a WRR ≥ 3.0 for over representation a) On an annual basis, notify PEAs that are flagged for disproportionate representation b) Provide technical assistance to PEA staff during their review of policies, procedures, and Timeline Complete Activity completed 2/1/12. Projected 7/1/10– 6/30/13 Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk WRRs for disproportionality. Resources (Planned) ADE/ESS Directors and Program Specialists ADE Research and Evaluation ADE/ESS Directors and Program Specialists Activity completed from 3/1/12 to 6/30/12. 7/1/10– 6/30/13 ESS specialists reviewed disproportionality data during on-site meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 3/1/12 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ESS specialists gave self assessment tools and resources to PEAs that were identified as at risk. Activity completed 4/1/12. ESS notified PEAs that were flagged for disproportionality. Activity completed from 4/1/12 to 6/30/12. ESS specialists made onsite visits to work with PEA 7/1/10– 6/30/13 Part B Arizona Annual Performance Report for FFY 2011 78 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 practices 3) Investigate strategies to assist PEAs that are flagged with disproportionate representation a) Investigate resources from the regional Equity Center, NCCRESt, and ADE/OELAS (Office of English Language Acquisition Services) b) Obtain input from stakeholders via regional groups and Special Education Advisory Panel c) Develop new strategies to assist PEAs that are flagged with disproportionate representation staff during the review of policies, procedures, and practices related to child find, eligibility, and evaluation. Activity completed 7/1/11 to 9/1/11. 7/1/11– 12/31/11 ADE/ESS Directors 7/1/11– 12/31/11 ADE/ESS Deputy Associate Superinten dent and Directors 1/1/12– 6/30/12 ADE/ESS Directors and Program Specialists 7/1/12– 6/30/13 ADE/ESS Directors and Program Specialists ESS directors researched resources offered by Equity Centers and ADE/OELAS. Activity completed 9/30/11. Presented resources to Special Education Advisory Panel and received comments from panel members. Activity completed 1/1/12 to 4/1/12. ESS reviewed available resources for PEAs and added new strategies. d) Implement new strategies to assist PEAs that are flagged with disproportionate representation Correction of FFY 2010 Findings of Noncompliance (if State did not report 0%) Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 0% Arizona made no findings of noncompliance in FFY 2010. Actions Taken if Noncompliance Not Corrected Not applicable. Arizona made no findings of noncompliance in FFY 2010. Verification of Correction (either timely or subsequent) Not applicable. Arizona made no findings of noncompliance in FFY 2010. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 Not applicable. Arizona made no findings of noncompliance in FFY 2010. Part B Arizona Annual Performance Report for FFY 2011 79 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Correction of Remaining FFY 2009 Findings of Noncompliance Not applicable. Verification of Correction of Remaining FFY 2009 Findings Not applicable. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2009 Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 80 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Disproportionality Indicator 10: Racial / Ethnic Disproportionality by Disability Percent of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification. (20 U.S.C. 1416 (a)(3)(C)) Measurement Percent = [(# of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification) divided by the (# of districts in the State)] times 100. Data Data Source The ADE/ESS collected the data from the PEAs through the October 1, 2011, child count report. The data are the same as the data collected and reported on Table 1, Report of Children with Disabilities Receiving Special Education under Part B of the Individuals with Disabilities Education Act, As Amended, for all children with disabilities aged 6–21 served under IDEA. Note that the source of this data is from FFY 2011. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because it collected, maintained, and reported the October 1, 2011, child count data using internal edit checks. In addition, the State requires the PEAs to assure data accuracy and reliability by submitting a signed verification letter. Definition of Disproportionate Representation Disproportionate Representation Over representation Weighted Risk Ratio Minimum n Size Target Racial / Ethnic Group Minimum n Size Racial / Ethnic Groups in Special Education and Related Services ≥ 3.00 30 30 Methodology The data were analyzed using Statistical Analysis Software (SAS) to produce a weighted risk ratio (WRR) that identified all racial/ethnic groups and six disability categories for all PEAs in the State. Data for over representation were examined. PEAs with a cell size of 30 or more students in the target racial/ethnic group and in the other racial/ethnic groups and meeting the weighted risk ratio criteria for over Part B Arizona Annual Performance Report for FFY 2011 81 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 representation were flagged for a review of policies, procedures, and practices by the State. PEAs with a lower cell size in the target groups were not flagged because false positives were identified as a function of the small number rather than as a result of noncompliant policies, procedures, and practices. Arizona included the total number of PEAs in the State (599) in the denominator. Of the 599 PEAs, 15 were eliminated from the analyses because a weighted risk ratio could not be calculated for any racial/ethnic group. Arizona’s Procedures to Determine if Disproportionate Representation Is the Result of Inappropriate Identification Arizona ensures that PEAs’ policies, procedures, and practices are reviewed, as required by 34 CFR §§ 300.173, 300.600(d)(3), and 300.602(a). The data are analyzed annually and PEAs may be flagged each year for over representation, according to the State’s definition. When a PEA is flagged, then the policies, procedures, and practices of the PEA are reviewed annually to determine if the disproportionate representation is the result of inappropriate identification. Arizona’s Review of PEAs’ Policies and Procedures On an annual basis, Arizona requires all PEAs to have special education policies and procedures in compliance with the requirements of 34 CFR § 300.111, § 300.201, and § 300.301 through § 300.311 prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. Each year, if the PEA makes any changes to the policies and procedures, the PEA must resubmit them to the State for review and acceptance. Each year, if the PEA does not make any changes to the policies and procedures, the PEA must submit a Statement of Assurance that says: “The PEA has not altered or modified the policies and procedures implementing the State and Federal requirements for services to children with disabilities previously submitted to and accepted by the Arizona Department of Education, Exceptional Student Services. If the PEA proposes to alter or modify the policies and procedures previously submitted to the Exceptional Student Services, the PEA must resubmit the policies and procedures to the Exceptional Student Services for review and acceptance.” In addition, the PEAs that are flagged for disproportionate representation must submit their policies and procedures related to child find, evaluation, and eligibility to an ADE/ESS specialist for review. Arizona’s Review of PEAs’ Practices On an annual basis, Arizona calculates the WRR for each PEA and uses the data as a trigger to flag PEAs with disproportionate representation. If a PEA is flagged, then an investigation of the practices is required to determine whether the disproportionate representation is a result of inappropriate identification. Review of practices when a PEA is flagged for over representation the first year:  The ESS specialist reviews current monitoring data, if applicable.  The PEA conducts a self assessment of the agency’s child find, evaluation, and eligibility practices to determine whether the disproportionate representation is a result of inappropriate identification. The self assessment consists of a series of questions requiring narrative responses and a review of student files using the State’s monitoring forms. The ADE/ESS specialists conduct on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews.  Upon completion of the self assessments, the PEAs have the option to begin immediately revising their policies, procedures, and practices related to child find, evaluation, and eligibility and to correct any noncompliance. No more than 60 days after completion of the self Part B Arizona Annual Performance Report for FFY 2011 82 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 assessment, the ESS specialists then interview the special education administrators and review student files via on-site visits and/or desk audits to verify correction of instances of any noncompliance, including child specific, and to ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. Review of practices when a PEA is flagged for over representation for two or more consecutive years:  If the PEA did not have disproportionate representation as a result of inappropriate identification the first year, then the ESS program specialist: o Reviews current monitoring data, if applicable, and; o Validates the prior year’s self assessment by reviewing a sample of student files.  If the PEA had disproportionate representation as a result of inappropriate identification the first year, then the PEA is required to: o Review current monitoring data, if applicable; o Review the prior year’s self assessment and describe the issues identified; o Describe the steps taken to resolve those issues; o Describe any current concerns regarding possible inappropriate identification; o Describe the resources and technical assistance used to help address the issues related to disproportionate representation within the agency; and o Review individual student files using the State’s monitoring forms:  The ADE/ESS specialists conduct on-site visits and/or desk audits during the file reviews to validate the decisions made by the PEAs.  The ESS specialists verify correction of instances of any noncompliance, including child specific, through on-site visits and/or desk audits.  The ESS specialists ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. When Arizona makes findings of noncompliance as a result of the review of policies, practices and procedures, the PEA has one year from the date of written notification from the State to correct the noncompliance. Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 0% Actual Target Data for FFY 2011 0% Arizona met the target. PEAs with Disproportionate Representation of Racial and Ethnic Groups in Specific Disability Categories That Was the Result of Inappropriate Identification Part B Arizona Annual Performance Report for FFY 2011 83 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Year Total Number of PEAs* FFY 2011 (2011– 2012) Number of PEAs with Disproportionate Representation of Racial and Ethnic Groups in Specific Disability Categories That Was the Result of Inappropriate Identification Number of PEAs with Disproportionate Representation 599 2 0 Percent of PEAs 0.00% *Arizona included the total number of PEAs in the denominator. PEAs with Disproportionate Representation of Racial and Ethnic Groups Arizona identified two PEAs with disproportionate representation of racial and ethnic groups in specific disability categories. PEAs with Disproportionate Representation of Racial and Ethnic Groups That Was the Result of Inappropriate Identification Arizona identified zero PEAs with disproportionate representation of racial and ethnic groups in specific disability categories that was the result of inappropriate identification (0.0%). Table 10.1 PEAs, and Cases, with Over Representation by Racial / Ethnic Group and Disability Note: The cases give a duplicated count of PEAs. Cases of Over Representation Autism Emotional Disturbance 2 1 Intellectual Disabilities Other Health Impairments Specific Learning Disability Speech or Language Impairment Hispanic/Latino of Any Race American Indian or Alaska Native Asian Black or African American Native Hawaiian or Other Pacific Islander White Two or More Races Part B Arizona Annual Performance Report for FFY 2011 84 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 # of PEAs flagged for over representation 2 # of PEAs found to have disproportionate representation (over representation) as a result of inappropriate identification 0 The following is a breakdown of Table 10.1:    One PEA was flagged for over representation due to a WRR of 3.0 or above for a total of two cases, one for autism and one for emotional disturbance. One PEA was flagged for over representation due to a WRR of 3.0 or above for one case for autism. No PEAs were found to have disproportionate representation as a result of inappropriate identification. The following describes the investigation of the policies, procedures, and practices of the two PEAs:  The two PEAs flagged for over representation submitted special education policies and procedures that were in compliance with the requirements of 34 CFR § 300.111, § 300.201, and § 300.301 through § 300.311 prior to having Part B-IDEA Basic Entitlement Grant funds approved. The ADE/ESS specialists reviewed the child find, evaluation, and eligibility policies and procedures during the PEA’s self assessment and found them to be in compliance.  The two PEAs were flagged for more than one consecutive year, and it was determined that the agencies did not have disproportionate representation as a result of inappropriate identification the prior year. In order to confirm this conclusion, the ADE/ESS specialists reviewed individual student files from FFY 2011 and the first few months of FFY 2012 and current monitoring data, where available. The practices of the PEAs were found to be consistent with 34 CFR § 300.173 and § 300.600(d)(3). It was determined that the PEAs did not have disproportionate representation as a result of inappropriate identification. Explanation of Progress That Occurred for FFY 2011 The FFY 2011 results (0.0%) are the same as those for FFY 2010 (0.0%). The Exceptional Student Services division has placed importance on improving the child find and evaluation process in a number of ways. The ESS directors and specialists review the PEAs’ data related to the SPP/APR Indicators annually. When a district or charter is identified as at risk for over representation, the ESS specialist notifies the PEA and offers information and support during a self-guided drill down. The monitoring process also helps the public schools to identify weaknesses with the child find and evaluation areas and to strengthen them through a comprehensive corrective action plan. Additionally, the Parent Information Network specialists (PINS) disseminate information to both parents and PEAs about child find requirements. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) Sub-Activities (Objectives or Action Steps) Timeline Complete Projected Resources (Planned) Part B Arizona Annual Performance Report for FFY 2011 85 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 1) Provide support for PEAs that are flagged as at risk for disproportionate representation with a WRR ≤ 0.40 for under representation and ≥ 2.5 for over representation a) Analyze data on an annual basis to flag PEAs that are at risk for disproportionate representation b) On an annual basis, notify PEAs that are flagged as at risk for disproportionate representation c) On an annual basis, provide assessment tools and resources to PEAs that are flagged as at risk to conduct a root cause analysis 2) Provide support for PEAs that are flagged for disproportionate representation with a WRR ≤ 0.30 for under representation and a WRR ≥ 3.0 for over representation a) On an annual basis, notify PEAs that are flagged for disproportionate representation b) Provide technical assistance to PEA staff during their review of policies, procedures, and practices 3) Investigate strategies to assist PEAs that are flagged with disproportionate representation a) Investigate resources from the regional Equity Center, NCCRESt, and ADE/OELAS (Office of English Language Acquisition Services) b) Obtain input from stakeholders via regional groups and Special Education Advisory Panel Activity completed 2/1/12. 7/1/10– 6/30/13 Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk WRRs for disproportionality. ADE/ESS Directors and Program Specialists ADE Research and Evaluation ADE/ESS Directors and Program Specialists Activity completed from 3/1/12 to 6/30/12. 7/1/10– 6/30/13 ESS specialists reviewed disproportionality data during on-site meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 3/1/12 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ESS specialists gave self assessment tools and resources to PEAs that were identified as at risk. Activity completed 4/1/12. ESS notified PEAs that were flagged for disproportionality. Activity completed from 4/1/12 to 6/30/12. 7/1/10– 6/30/13 ESS specialists made onsite visits to work with PEA staff during the review of policies, procedures, and practices related to child find, eligibility, and evaluation. Activity completed 7/1/11 to 9/1/11. 7/1/11– 12/31/11 ADE/ESS Directors 7/1/11– 12/31/11 ADE/ESS Deputy Associate Superinten dent and ESS directors researched resources offered by Equity Centers and ADE/OELAS. Activity completed 9/30/11. Presented resources to Special Education Advisory Panel and received Part B Arizona Annual Performance Report for FFY 2011 86 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 comments from panel members. c) Develop new strategies to assist PEAs that are flagged with disproportionate representation Activity completed 1/1/12 to 4/1/12. Directors 1/1/12– 6/30/12 ADE/ESS Directors and Program Specialists 7/1/12– 6/30/13 ADE/ESS Directors and Program Specialists ESS reviewed available resources for PEAs and added new strategies. d) Implement new strategies to assist PEAs that are flagged with disproportionate representation Correction of FFY 2010 Findings of Noncompliance (if State reported more than 0% compliance) Level of compliance (actual target data) State reported for FFY 2010 for this Indicator: 0% Arizona made no findings of noncompliance in FFY 2010. Actions Taken if Noncompliance Not Corrected Not applicable. Arizona made no findings of noncompliance in FFY 2010. Verification of Correction (either timely or subsequent) Not applicable. Arizona made no findings of noncompliance in FFY 2010. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 Not applicable. Arizona made no findings of noncompliance in FFY 2010. Correction of Remaining FFY 2009 Findings of Noncompliance Not applicable. Verification of Correction of Remaining FFY 2009 Findings Not applicable. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2009 Not applicable. Part B Arizona Annual Performance Report for FFY 2011 87 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Correction of Any Remaining Findings of Noncompliance from FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 88 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / Child Find Indicator 11: Evaluation Timelines Percent of children who were evaluated within 60 days of receiving parental consent for initial evaluation or, if the State establishes a timeframe within which the evaluation must be conducted, within that timeframe. (20 U.S.C. 1416 (a)(3)(B)) Measurement a. # of children for whom parental consent to evaluate was received. b. # of children whose evaluations were completed within 60 days (or State-established timeline). Account for children included in a but not included in b. Indicate the range of days beyond the timeline when the evaluation was completed and any reasons for the delays. Percent = [(b) divided by (a)] times 100. Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 100% Actual Target Data for FFY 2011 97% Arizona did not meet the target. Data Data Source The data for Indicator 11 are from the Arizona monitoring system. A public education agency (PEA) is selected for monitoring each fiscal year based on the results of a review of the agency’s data, including data from the SPP/APR, dispute resolution results, audit findings, and annual determinations. While Arizona has maintained a six-year monitoring cycle with assigned activities always occurring in Year 4, a PEA’s monitoring schedule can be adjusted and Year 4 monitoring activities can occur when the data reviews indicate systemic issues. Data Collection Data are collected from the PEAs during one of three types of monitorings: Part B Arizona Annual Performance Report for FFY 2011 89 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011  Data Review — PEAs review student files focusing on Indicator 11. The ADE/ESS specialist validates the compliance calls. The student file forms are submitted to ESS for data entry.  Self-Assessment — PEAs review student files and collect data for Indicator 11. The PEAs also focus on identified SPP/APR Indicators in which agency results have not met the State target. The ADE/ESS specialist validates the compliance calls. The student file forms are submitted to ESS for data entry.  On-Site — PEAs and the ADE/ESS team review student files, collect data through surveys and interviews, and collect data for Indicator 11. The ADE/ESS staff input data. The data that Arizona collects and reports for this Indicator include all children whose permissions to evaluate were received during FFY 2011 and for whom initial evaluations including eligibility determinations were completed during either FFY 2011 or FFY 2012. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data as it is collected, maintained, and reported through the State monitoring system. Training is provided to all ESS program specialists who monitor to ensure interrater reliability on compliance calls that are based on regulatory requirements. The ADE/ESS staff conduct trainings for PEA staff who will participate in monitorings. The ESS specialists validate and verify the data through on-site visits or desk audits. Evaluation Timeline Arizona has established a 60-day timeline for initial evaluations. The Arizona Administrative Code (A.A.C.) R7-2-401 (E)(3) states that the initial evaluation shall not exceed 60 calendar days from receipt of informed written consent. However, the 60-day evaluation period may be extended for an additional 30 days if it is in the best interests of the child and the parents and the public education agency agree in writing to do so (A.A.C. R7-2-401 (E)(4)). Definition of Finding for Monitoring for FFY 2011 During FFY 2011, a finding for Indicator 11 was issued when the line item for the evaluation timeline was found to be noncompliant. The finding was a written notification to the PEA by the State that the line item was noncompliant, and the finding included a description of a Federal or State statute or regulation. The source of information on which to base a finding of noncompliance was an individual student file. Children Evaluated Within 60 Days (or State-established timeline) a. Number of children for whom parental consent to evaluate was received 915 b. Number of children whose evaluations were completed within 60 days (or Stateestablished timeline) 883 Percent of children with parental consent to evaluate, who were evaluated within 60 days (or State-established timeline) (Percent = [(b) divided by (a)] times 100) 97% Children Included in a (above) and Not Included in b (above) Part B Arizona Annual Performance Report for FFY 2011 90 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 FFY 2011 Noncompliance number of findings by incidence of noncompliance number of findings by incidence corrected prior to one-year timeline as of 1/15/13 32 18 Arizona made 32 findings of noncompliance in FFY 2011. Although the PEAs have one year to correct the noncompliance, 18 findings have been corrected as of January 15, 2013. Correction of the remaining noncompliance (14 findings) will be reported in the FFY 2012 APR. Range of Days Beyond the Timeline and Reasons for the Delays Table 11.1 Range of Days Beyond Timeline Range of days 1–120 Mean 26 Median 15 Mode 11 The 120 days beyond the 60-day timeline occurred at a school district that, at the time of the parent request for the evaluation, had one psychologist for four high schools. The evaluation was completed, although it was past the timeline. The ADE/ESS specialist verified the correction of the child specific noncompliance and verified that a new strategy was developed to ensure compliance with the 60-day evaluation timelines. During follow-up visits to ensure sustainability, the ESS specialist verified that new evaluations are being done within the timeline. Table 11.2 Reasons Given for Delays Unavailability of required personnel (parent, general education teacher, etc.) 9 Miscalculation of 60-day timeline 6 Lack of an adequate timeline tracking system 5 Shortage of evaluators 4 Lack of understanding of evaluation process 4 Interruption in school calendar 3 Inability of student to complete requested assessments 1 Part B Arizona Annual Performance Report for FFY 2011 91 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Explanation of Progress That Occurred for FFY 2011 Although the FFY 2011 results are the same as those for FFY 2010 (97%), there were 189 more initial evaluations reviewed during FFY 2011 (915 initial evaluations in FFY 2011 as compared to 726 in FFY 2010). The monitoring system entails an annual comprehensive review of data elements, including Indicator 11. Each February, the ESS directors and program specialists examine the APR data and other information for all the PEAs in the State. Following these daylong meetings, the specialists visit their assigned PEAs to discuss each agency’s results, the SPP/APR targets, and strategies to achieve compliance and improve results. The specialists explain the 60-day timeline requirement, disseminate a flyer with information and relevant citations, provide a tracking spreadsheet for agency use, and distribute a laminated 60-day calendar chart. The specialists provide ongoing technical assistance throughout the year during on-site visits to review files and to teach school personnel how to review and analyze their own student files. These site visits and follow-up by the ADE/ESS specialists continue to focus on Indicator 11. The first site visit of the school year must be completed by the end of the second quarter of the school year for each PEA, and the specialists must enter the information into the tracking log within seven calendar days of the visit. This site visit includes informing the staff about the requirements of IDEA regarding evaluation timelines. The school staff is asked to analyze the system for tracking timelines and examining files. When a PEA does not have systems in place, a second site visit is conducted by the beginning of the fourth quarter of the school year. The PEAs that continue to have difficulty with following evaluation timelines receive more technical assistance from their assigned specialist. Further analyses of the reasons are completed, and systemic changes are suggested by the ESS specialists to improve adherence to the timeline. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Develop and conduct Webinars pertaining to the requirements for compliant evaluations and IEPs Sub-Activities (Objectives or Action Steps) a) Develop Webinar trainings for evaluation and IEP requirements b) Conduct statewide Webinars for evaluation and IEP requirements c) Collect and analyze training feedback from participants Timeline Complete Activity completed 12/1/10 to 6/30/11. Monitoring director and monitoring team created Webinars to train PEAs on the evaluation process and required timelines. Webinars were reviewed and revised and will be presented statewide beginning in November 2012. An update on this improvement activity will be given in the FFY 2012 APR. This activity will be conducted beginning in December 2012 and an update given in the FFY Projected Resources (Planned) 12/1/10– 6/30/11 ADE/ESS Directors and Specialists 7/1/11– 12/31/11 ADE/ESS Directors and Specialists 1/1/12– 4/30/12 ADE/ESS Directors and Specialists Part B Arizona Annual Performance Report for FFY 2011 92 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 2012 APR. 2) Review the Arizona Technical Assistance System (AZTAS) Evaluation and Eligibility document and revise, as necessary d) Collect corrective action close-out (timeline) data for evaluation and IEP monitoring line items a) Review the AZTAS Evaluation and Eligibility document to determine if it is current with statutes and regulations b) Revise the AZTAS Evaluation and Eligibility document, if appropriate This activity will be conducted beginning in June 2013 and an update given in the FFY 2012 APR. Activity completed 7/1/11 to 6/30/12. c) Disseminate revised AZTAS Evaluation and Eligibility document via ESS Web site and ESS specialists Activity completed by 6/30/12. ESS reviewed AZTAS document for current requirements. Activity completed by 6/30/12. 5/1/12– 6/30/12 ADE/ESS Directors and Specialists 7/1/11– 12/31/11 ADE/ESS Deputy Associate Superintende nt and Directors ADE/ESS Deputy Associate Superintende nt, Directors, Specialists ADE/ESS Directors and Specialists 1/1/12– 6/30/12 ESS made revisions to the AZTAS document. 7/1/12– 6/30/13 ESS disseminated the revised AZTAS document by posting on the ADE Web site and distributing it via ESS specialists. This activity completed. Correction of FFY 2010 Findings of Noncompliance (if State reported less than 100% compliance) Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 97% 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) 25 2. Number of FFY 2010 findings the State verified as timely corrected (corrected within one year from the date of notification to the LEA of the finding) 23 3. Number of FFY 2010 findings not verified as corrected within one year [(1) minus (2)] 2 Correction of FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 2 5. Number of FFY 2010 findings the State has verified as corrected beyond the one- 2 Part B Arizona Annual Performance Report for FFY 2011 93 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 year timeline (“subsequent correction”) 6. Number of FFY 2010 findings not verified as corrected [(4) minus (5)] 0 The ADE/ESS specialists reviewed the child specific files from the monitorings to determine that the PEAs completed the evaluation for any child whose initial evaluation was not timely, unless the child was no longer within the PEA. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine that the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to the evaluation process in conformity with 34 CFR § 300.301 (c) (1). Actions Taken if Noncompliance Not Corrected All FFY 2010 noncompliance has been corrected, and Arizona has verified correction for all FFY 2010 findings. When findings of noncompliance are made, Arizona works collaboratively with the PEAs as the agencies identify the root causes of their continuing noncompliance through drill downs in the specific focus area that affects this Indicator. However, when noncompliance is not corrected within one year, the ADE/ESS uses a variety of methods to ensure that all public agencies meet the requirements of State and Federal statutes related to special education. The enforcement actions taken by ESS for the PEAs that are unable to demonstrate compliance within one year from the date of written notification are as follows:  Enforcement of corrective action plan (CAP) activities as outlined in the current agency CAP, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Review and revision of the current CAP to develop targeted activities that address the continued noncompliance, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Assignment of a special monitor.  Interruption of IDEA payments until adequate compliance is achieved. For charter schools not receiving IDEA funds, a request to begin withholding 10% of State funds.  For charter schools, a request to the appropriate board for a notice of intent to revoke the charter.  With Arizona State Board of Education approval, interruption of Group B weighted State aid or redirection of funds pursuant to 34 CFR § 300.227 (a).  Referral to the Arizona Attorney General for legal action. Verification of Correction of FFY 2010 Noncompliance (either timely or subsequent) FFY 2010 Verification of Correction from Monitoring As specified in OSEP’s June 2011 Arizona Part B FFY 2009 SPP/APR Response Table, Arizona verified that each PEA with noncompliance reflected in the data: 1) is correctly implementing 34 CFR § 300.301 (c) (1) (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through on-site monitoring; and Part B Arizona Annual Performance Report for FFY 2011 94 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 2) has completed the evaluation, although late, for any child whose initial evaluation was not timely, unless the child is no longer within the jurisdiction of the LEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data:  ADE/ESS specialists conducted follow-up on-site visits and/or desk audits after the monitoring to verify correction of all instances of noncompliance, including those that were child specific. The specialists reviewed the child specific files to determine that the evaluation was completed within 60 calendar days from the date of written notification of noncompliance.  ADE/ESS specialists reviewed updated data from subsequent files and/or conducted interviews with the special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including those that were child specific, were corrected and to ensure ongoing sustainability of the implementation of the regulatory requirements regarding initial evaluations. Correction of Remaining FFY 2009 Findings of Noncompliance Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2010, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2011 APR for Indicator 11 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2010 data). In reporting on correction of findings of noncompliance in the FFY 2011 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2010 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2011 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2011 APR for Indicator 11 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2010 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. Part B Arizona Annual Performance Report for FFY 2011 95 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 If the State does not report 100% compliance in the FFY 2011 APR, the State must review its improvement activities and revise them, if necessary. Arizona reviewed its improvement activities for Indicator 11. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 96 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / Effective Transition Indicator 12: Early Childhood Transition Percent of children referred by Part C prior to age 3, who are found eligible for Part B, and who have an IEP developed and implemented by their third birthdays. (20 U.S.C. 1416 (a)(3)(B)) Measurement a. # of children who have been served in Part C and referred to Part for Part B eligibility determination. b. # of those referred determined to be NOT eligible and whose eligibilities were determined prior to their third birthdays. c. # of those found eligible who have an IEP developed and implemented by their third birthdays. d. # of children for whom parent refusal to provide consent caused delays in evaluation or initial services or to whom exceptions under 34 CFR § 300.301(d) applied. e. # of children who were referred to Part C less than 90 days before their third birthdays. Account for children included in a but not included in b, c, d, or e. Indicate the range of days beyond the third birthday when eligibility was determined and the IEP developed and the reasons for the delays. Percent = [(c) divided by (a – b – d – e)] times 100. Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 100% Actual Target Data for FFY 2011 99% Arizona did not meet the target. Data Data Source The data for Indicator 12 are reported annually by all PEAs in Arizona that have children who transition from Part C to Part B. Data are included for the entire reporting year, from July 1, 2011, through June 30, 2012. Data Collection Part B Arizona Annual Performance Report for FFY 2011 97 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The data are collected through the Annual Special Education Data Collection, an ADE Web-based data collection system. Valid and Reliable Data The Arizona Department of Education (ADE)/Early Childhood Special Education (ECSE) unit assures the validity and reliability of the data as it is collected, maintained, and reported through internal edit checks. Training is provided to school personnel by the Exceptional Student Services (ESS) Data Management unit regarding the operation of the data system and interpretation of the questions that are components of the measurement. The State requires an assurance from the PEAs through the submission of a signed form attesting to the validity of the data. Random verification checks require that a selected district submit copies of the front pages of IEPs that show the dates of the IEP and the children’s birthdays for children who transitioned from early intervention services or a copy of the prior written notices (PWNs) for children who were found ineligible by the children’s third birthdays. Definition of Finding A finding of noncompliance for Indicator 12 is defined as the number of PEAs with noncompliance. The finding of noncompliance is a written notification to the PEA by the State that the PEA is noncompliant. Actual State Data (Numbers) for FFY 2011 a. # of children who have been served in Part C and referred to Part B for Part B eligibility determination 2,453 b. # of those referred determined to be NOT eligible and whose eligibility was determined prior to third birthday 333 c. # of those found eligible who have an IEP developed and implemented by their third birthdays 1,978 d. # for whom parent refusals to provide consent caused delays in evaluation or initial services or to whom exceptions under 34 CFR § 300.301(d) applied 97 e. # of children who were referred to Part C less than 90 days before their third birthdays 26 # in a but not in b, c, d, or e 19 Percent of children referred by Part C prior to age 3 who are found eligible for Part B, and who have an IEP developed and implemented by their third birthdays Percent = [(c) / (a-b-d-e)] * 100 99% FFY 2011 Noncompliance Part B Arizona Annual Performance Report for FFY 2011 98 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 number of findings of noncompliance number of findings corrected prior to one-year timeline as of 1/15/13 7 7 Arizona made seven findings of noncompliance in FFY 2011. Although the PEAs have one year to correct the noncompliance, all seven findings have been corrected as of January 15, 2013. Account for Children Included in a, but not in b, c, d, or e — Reasons for Delays Late referrals from Part C 12 Failed hearing or vision screening 7 Total 19 Each year since FFY 2009 the number of children not transitioned on time due to late referrals from Part C has decreased. Currently, in FFY 2011, 12 children were not transitioned on time due to late referrals from Arizona Early Intervention Program (AzEIP), Arizona’s Part C referral system, as compared with 21 in FFY 2010. School districts are asked to submit an alert to the ADE/ECSE any time they receive a late referral from AzEIP that could not be classified in category d (parent refusals to provide consent caused delays in evaluation or initial services) or in category e (children who were referred to Part C less than 90 days before their third birthdays). Each late referral from AzEIP to a district is reported to the State AzEIP office. The State AzEIP office provides technical assistance and follow-up to the local service-providing agency. Similarly, if a local service-providing agency under the authority of AzEIP is experiencing difficulty with a school district, the local agency issues an alert to the State AzEIP office. Then the ADE/ECSE provides technical assistance and follow-up to the school district. The ADE/ECSE and AzEIP maintain a shared database to track the resolution of the difficulties. Challenges with the completion of hearing and vision screenings and with the needed follow-ups are an inherent part of evaluating young children; these challenges at times causes delay in transitioning these children to preschool services. Arizona has worked diligently to provide resources and to facilitate collaborative efforts between Head Start, school districts, and Part C. These efforts have helped Part C service coordinators encourage families to have regular hearing screenings. Range of Days Beyond Third Birthday Range of days 1–140 The140 days’ delay beyond the child’s third birthday was due to a delay in obtaining hearing screening results. The school district followed up with medical referrals that delayed the determination of the child’s hearing status. Technical assistance has been provided regarding finding eligibility in other areas of development until hearing status can be confirmed. The ADE/ECSE verified the correction of the child specific noncompliance and that FAPE was provided to the child. Follow-up desk audits by the ECSE and ESS specialists verified that a process was established to mitigate future delays of a similar nature. Part B Arizona Annual Performance Report for FFY 2011 99 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Explanation of Progress That Occurred for FFY 2011 Arizona maintained the same results (99%) from FFY 2010 to FFY 2011. The ADE/ECSE unit worked with the Part C lead agency, Arizona Early Intervention Program (AzEIP), to implement changes based on the new Part C regulations, and used the alert system to improve the transition process for families. Each instance of late referral from AzEIP was followed up by the ADE/ECSE together with AzEIP and systemic issues were identified and corrected. The continued collaboration between AzEIP and ECSE also contributed to continued positive outcomes. Additionally, the ADE/ESS specialists reviewed files during their annual site visits, provided technical assistance, and alerted ECSE of any problems during the year. Discussion of Improvement Activities Completed for FFY 2011 1) Implement new series of transition trainings regarding 2010 Intergovernmental Agreement (IGA) a) Provide joint professional development activities with service coordinators and school district personnel that emphasize 100% compliance and building of relationships 2) Post training materials to ADE/ECSE Web site a) Provide access to professional development training materials for AzEIP and school district personnel on ADE/ECSE Web site 3) Promote and support “I’m Turning 3: What’s Next for Me?” parent trainings a) Post “I’m Turning 3: What’s Next for Me?” on ECSE Web site Activity completed from 7/1/11 to 6/30/12. A series of jointly facilitated trainings were held by ADE and AzEIP. Early intervention service providers, Head Start, and school district staff members participated in the sessions designed to enhance collaboration and reinforce the importance of each step in transition and the required timelines for compliance. Activity completed from 7/1/11 to 6/30/12. Professional development documents were developed jointly with AzEIP and posted on the ADE Early Childhood website at http://www.azed.gov/earlychildhood/preschool/preschoolprograms/ecse/earlyintervention-transitions-part-c-topart-b/. Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ECSE Director and Specialist AzEIP Staff 7/1/10– 6/30/13 ADE/ECSE Director and Specialist AzEIP Staff 7/1/10– 6/30/13 ADE/ECSE AzEIP Staff The parent transition booklet, “I’m Turning 3: What’s Next for Me” was posted on the ECSE Web site and was distributed during parent trainings. The completion of a Spanish version was placed on hold due to anticipated changes in Part C regulations. Arizona’s Parent Training and Information Center PEA Staff Parent Information Network Specialists Raising Special Part B Arizona Annual Performance Report for FFY 2011 100 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 4) Maintain Alert System between Part C and Part B to examine and resolve systemic and situational issues a) Respond to individual alerts at local level to resolve issues b) Maintain database to track number of alerts reported to ADE/ECSE and AzEIP 5) Conduct targeted technical assistance to PEAs found to be noncompliant a) Provide phone and email consultation to PEAs found to be noncompliant b) Review noncompliant PEAs’ policies, procedures, and practices via desk audits and monthly review of data 6) Revise the Intergovernmental Agreement (IGA) with the Arizona Early Intervention Program (AzEIP) a) Review and revise the 2010 IGA with AzEIP to align with the Part C regulations (Raising Special Kids/Pilot Parents of Southern Arizona) and ADE/ESS Parent Information Network specialists (PINS) provided trainings to parents statewide. Activity completed from 7/1/11 to 6/30/12. ECSE and AzEIP staff responded to all alerts from the field. School districts and AzEIP service coordinators were assisted with resolving issues. Activity completed from 7/1/11 to 6/30/12. Kids 7/1/10– 6/30/13 AzEIP Staff 7/1/10– 6/30/13 Database was maintained to track the number and sources of alerts in order to drive professional development and technical assistance decisions. 38 alerts were filed (26 from districts and 12 from AzEIP). Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 Ongoing consultation was provided by ECSE staff in collaboration with AzEIP staff. Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 Noncompliant PEAs’ written policies and procedures were reviewed, along with their routine practices. Technical assistance was provided. File reviews were completed to ensure compliance with corrective action plans. This activity will be reported in the FFY 2012 APR based on the implementation of Part C regulations. ADE/ECSE Director and Specialist ADE/ECSE Director and Specialist AzEIP Staff ADE/ECSE Director and Specialist AzEIP Staff ADE/ECSE Director and Specialist AzEIP Staff 7/1/11– 6/30/13 ADE/ECSE Director AzEIP Staff Correction of FFY 2010 Findings of Noncompliance (if State reported less than 100% compliance in its FFY 2010 APR) Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 99% Part B Arizona Annual Performance Report for FFY 2011 101 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) 8 2. Number of FFY 2010 findings the State verified as timely corrected (corrected within one year from the date of notification to the LEA of the finding) 8 3. Number of FFY 2010 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2010 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2010 findings not verified as corrected [(4) minus (5)] 0 The eight PEAs identified as being out of compliance submitted to ECSE the policies and procedures for early intervention transitions that were mutually agreed upon with the AzEIP service coordinators. The ADE/ECSE specialists reviewed the child specific files from the PEAs to determine if the IEPs were developed and implemented, although late, unless the child was no longer within the jurisdiction of the PEA. The ADE/ESS specialists conducted follow-up on-site visits and/or desk audits and reviewed updated data based on subsequent student files to verify that each PEA was correctly implementing the regulatory requirements (i.e., achieved 100% compliance) in conformity with 34 CFR § 300.124 (b). Actions Taken if Noncompliance Not Corrected All FFY 2010 noncompliance has been corrected, and Arizona has verified correction for all FFY 2010 findings. When findings of noncompliance are made, Arizona works collaboratively with school districts and AzEIP to identify the root causes of continuing noncompliance through individual verification and development of written processes and procedures between districts, AzEIP service coordinators, Head Start Programs, and other programs within the district of residence boundaries. When noncompliance is not corrected within one year, the ADE/ECSE interrupts 619 funds until full compliance is demonstrated. Verification of Correction (either timely or subsequent) As specified in OSEP’s June 2012 FFY 2010 SPP/APR Response Table, Arizona verified that each PEA with noncompliance reflected in the data: 1) is correctly implementing 34 CFR § 300.124 (b) (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through on-site monitoring; and 2) has developed and implemented the IEP, although late, for any child for whom implementation of the IEP was not timely, unless the child is no longer within the jurisdiction of the LEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Part B Arizona Annual Performance Report for FFY 2011 102 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data:  The ADE/ECSE specialists reviewed the written process and procedures for the PEAs’ early intervention transitions.  The ADE/ESS specialists and/or ADE/ECSE specialists reviewed student files during subsequent on-site visits and/or desk audits of updated data to determine if the PEAs corrected all instances of noncompliance, including child specific, and to ensure ongoing sustainability with the implementation of the regulatory requirements. Correction of Remaining FFY 2009 Findings of Noncompliance Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2010, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2011 APR for Indicator 12 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2010 data). In reporting on correction of findings of noncompliance in the FFY 2011 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2010 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2011 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2011 APR for Indicator 12 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2010 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. If the State does not report 100% compliance in the FFY 2011 APR, the State must review its improvement activities and revise them, if necessary. Arizona reviewed its improvement activities for Indicator 12. Part B Arizona Annual Performance Report for FFY 2011 103 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 104 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / Effective Transition Indicator 13: Secondary Transition Percent of youth with IEPs aged 16 and above with an IEP that includes appropriate measurable postsecondary goals that are annually updated and based upon an age appropriate transition assessment, transition services, including courses of study, that will reasonably enable the student to meet those postsecondary goals, and annual IEP goals related to the student’s transition services needs. There also must be evidence that the student was invited to the IEP Team meeting where transition services are to be discussed and evidence that, if appropriate, a representative of any participating agency was invited to the IEP Team meeting with the prior consent of the parent or student who has reached the age of majority. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent = [(# of youth with IEPs aged 16 and above with an IEP that includes appropriate measurable postsecondary goals that are annually updated and based upon an age appropriate transition assessment, transition services, including courses of study, that will reasonably enable the student to meet those postsecondary goals, and annual IEP goals related to the student’s transition services needs. There also must be evidence that the student was invited to the IEP Team meeting where transition services are to be discussed and evidence that, if appropriate, a representative of any participating agency was invited to the IEP Team meeting with the prior consent of the parent or student who has reached the age of majority) divided by the (# of youth with an IEP age 16 and above)] times 100. Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 100% Actual Target Data for FFY 2011 78% Arizona did not meet the target. Year Total number of youth aged 16 and above with an IEP Total number of youth aged 16 and above with an IEP that meets the requirements Percent of youth aged 16 and above with an IEP that meets the requirements FFY 2011 451 352 78% Part B Arizona Annual Performance Report for FFY 2011 105 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 (2011– 2012) FFY 2011 Findings of Noncompliance number of findings by incidence of noncompliance number of findings by incidence corrected prior to one-year timeline as of 1/15/13 99 46 Arizona made 99 findings of noncompliance in FFY 2011. Although the PEAs have one year to correct the noncompliance, 46 findings have been corrected as of January 15, 2013. Correction of the remaining noncompliance (53 findings) will be reported in the FFY 2012 APR. Data Revision to Data Collection The actual number of youth aged 16 and above with an IEP in FFY 2011 (451) is significantly less than the actual number of youth aged 16 and above with an IEP in FFY 2010 (1,513). This is because ESS changed the data collection and reporting system in FFY 2011 to consolidate all the Indicator 13 components into one line item in the monitoring system. Prior to FFY 2011, the components were split into two line items and were summed to report the total number of findings. Data Source The data for Indicator 13 are from the Arizona monitoring system. A public education agency (PEA) is selected for monitoring each fiscal year based on the results of a review of the agency’s data, including data from the SPP/APR, dispute resolution results, audit findings, and annual determinations. While Arizona has maintained a six-year monitoring cycle with assigned activities always occurring in Year 4, a PEA’s monitoring schedule can be adjusted and Year 4 monitoring activities can occur when the data reviews indicate systemic issues. The National Secondary Transition Technical Assistance Center (NSTTAC) Indicator 13 Checklist was used as a guide for the eight components that comprise the monitoring line item from which the data are pulled. The eight components are:         Measurable post-secondary goals Postsecondary goals updated annually Postsecondary goals based upon age-appropriate transition assessments Transition services Courses of study Annual IEP goals related to transition service needs Student invited to IEP meeting Representative of participating agency invited to IEP meeting with prior consent of parent or student who has reached the age of majority Data Collection Part B Arizona Annual Performance Report for FFY 2011 106 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Data are collected from the PEAs during one of three types of monitorings:  Data Review — PEAs review student files focusing on Indicator 13. The ADE/ESS specialists validate the compliance calls. The student file forms are submitted to ESS for data entry.  Self-Assessment — PEAs review student files and collect data for Indicator 13. The PEAs also focus on reviewing files for other identified SPP/APR Indicators on which their agencies have not met the State targets. The ADE/ESS specialists validate the compliance calls. The student file forms are submitted to ESS for data entry.  On-Site — PEAs and the ADE/ESS teams review student files, collect data through surveys and interviews, and collect data for Indicator 13. The ADE/ESS staff inputs data. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data as it is collected, maintained, and reported through the State monitoring system. Training is provided to all ESS program specialists who monitor to ensure interrater reliability for compliance calls according to regulatory requirements. The ADE/ESS staff conducts trainings for PEA staff who will participate in monitorings. The ESS specialists validate and verify the data through on-site visits or desk audits. Definition of Finding for Monitoring for FFY 2011 During FFY 2011, a finding for Indicator 13 was issued when the line item for secondary transition was found to be noncompliant. The finding was a written notification to the PEA by the State that the line item was noncompliant, and the finding included a description of a Federal or State statute or regulation. The source of information on which to base a finding of noncompliance is an individual student file. Explanation of Slippage That Occurred for FFY 2011 The slippage of 11.2 percentage points from FFY 2010 can be contributed to a number of factors. The source of the data for this Indicator is the monitoring database and the cohorts vary from year to year. There were 115 PEAs monitored during FFY 2011. The majority of the 115 PEAs participated in a self assessment monitoring in which they focused on specific results or on compliance indicators as identified by a review of their Indicator data. Many of the PEAs monitored through self assessment were large school districts. The data related to secondary transition for these large PEAs were 100% compliant based on annual site visit reviews; therefore, secondary transition was not a focus area for monitoring. As a result, very few student files reviewed included the line item for secondary transition. The remaining PEAs that were monitored, specifically those selected for on-site monitorings, were PEAs with system-wide issues, including secondary transition. A large file sample is pulled for on-site monitorings. The bulk of the data for this Indicator came from files reviewed for these PEAs in on-site monitorings. The FFY 2011 cohort for on-site monitorings included multiple PEAs with compliance issues related to this Indicator, thus increasing the number of findings of noncompliance. Although there is slippage, the ADE/ESS specialists who monitor provide ongoing technical assistance throughout the year during on-site visits to review files and to teach the school personnel how to review and analyze their own student files. These site visits and follow-ups by the ADE/ESS specialists continue to focus on Indicator 13. Part B Arizona Annual Performance Report for FFY 2011 107 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 On an annual basis, the ESS secondary transition specialists update and implement a comprehensive strategic plan that includes analysis of Indicator 13 data; staff development for PEAs; provision of capacity building grants; collaboration with national technical assistance centers and with other State agencies; and organization of a statewide transition conference (see the discussion of improvement activities below). Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Revise, implement, and evaluate a comprehensive plan for training PEAs to increase compliance with postsecondary requirements related to Indicator 13 Sub-Activities (Objectives or Action Steps) a) On an annual basis, identify PEAs in Years 2 and 3 of the monitoring cycle through collaboration with ESS specialists b) On an annual basis, review, revise (if necessary), and implement the comprehensive training plan, emphasizing the eight required components of Indicator 13 Timeline Complete Activity completed 8/31/11. The FFY 2011 PEA list was completed August 2011. The Annual Site Visit Log (ASVL) from school year 2010-2011 was used to identify PEAs most in need of training and technical assistance (TA) for Indicator 13. Activities completed from 7/1/11 to 6/30/12. Projected 7/1/11– 6/30/13 Resources (Planned) ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists The FFY 2011 review and revision of the Strategic Plan for Statewide Transition Planning was completed July 2011. Implementation of the Transition Strategic Plan was completed from July 2011 to June 2012. The FFY 2011 Transition Strategic Plan includes seven main components: 1. Provide training to targeted PEAs and in response to requests from non-targeted PEAs on Indicator 13; 2. Organize Arizona’s Eleventh Annual Transition Conference focusing on improving post school outcomes for students with disabilities by providing sessions on transition planning and dropout prevention; 3. Provide training to special education directors from across the State at the annual ADE/ESS Directors Institute; Part B Arizona Annual Performance Report for FFY 2011 108 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 4. Provide capacity building grants to PEAs to facilitate intra/interagency collaboration and build local capacity to improve post school outcomes through local interagency work, as well as provide intensive training and support for PEAs to achieve 100% compliance on Indicator 13; 5. Collaborate with national technical assistance centers and organizations including NSTTAC, NPSO, NDPC-SD, and the NASDSE IDEA Partnership Community of Practice on Transition and Transition Coalition; 6. Participate with other Arizona State agencies including Rehabilitation Services of Arizona/Vocational Rehabilitation (RSA/VR), Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN); 7. Collaborate with other ADE sections: High School Renewal and Improvement (AZHSRI), Career and Technical Education (CTE), Dropout Prevention, and School Counselors and ADE/ESS areas: Program Support, Assistive Technology, and Parent Information Network. All components of Arizona’s FFY 2011 Strategic Plan for Statewide Transition Planning were immediately implemented upon revision, from 8/1/11 to 6/30/12. Activities completed:  1,089 participants from 130 targeted and non-targeted PEAs received secondary transition training, including Indicator 13 trainings from ADE/ESS transition Part B Arizona Annual Performance Report for FFY 2011 109 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 specialists at 75 sites statewide. Trainings were delivered in regional or direct school locations.  Arizona’s Eleventh Annual Transition Conference was held in October 2011 and offered sessions focused on improving compliance with the eight components of Indicator 13. 829 participants attended the conference, including education and agency professionals, youth, young adults, family members of youth with disabilities, and vendors/exhibitors.  Four sessions on secondary transition were provided by ADE/ESS transition specialists at the annual ADE/ESS Directors Institute (DI). Of the 660 in attendance at the DI, 78 participants attended sessions related to secondary transition.  26 PEAs participated in Year 1 (14 PEAs in the 2010–2012 cohort) or Year 2 (13 PEAs in the 2010–2012 cohort) team trainings of the Secondary Transition Mentoring Project (STMP) capacity building grant. Through a contract with the University of Kansas Transition Coalition and in collaboration with ADE/ESS, training was provided to achieve and sustain 100% compliance on Indicator 13 using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. Team training included: three two-day face-toface training sessions, webinars, a Web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and attendance at Arizona’s Eleventh Annual Part B Arizona Annual Performance Report for FFY 2011 110 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Transition Conference.  Collaboration with national technical assistance centers and organizations occurred throughout the year and included: participation in NPSO and NSTTAC Community of Practice calls; use of resources from the NSTTAC Web site; participation in the NASDSE IDEA Partnership, Community of Practice National Meeting; and attendance at the National Secondary Transition Planning Institute where OSEP, NPSO, NSTTAC, and NDPC-SD provided guidance. Additionally, ADE/ESS maintained ongoing collaboration with the University of Kansas Transition Coalition and the Mountain Plains Regional Resource Center.  At the State level, ADE/ESS collaborated with RSA/VR, Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN), and met every other month with these agency stakeholders through the Arizona Community of Practice on Transition (AZCoPT). In FFY 2011, AZCoPT revised a presentation used statewide through RSA/VR and DBHS teleconferencing media to introduce participants to the supports/services available to school-aged and adult individuals with disabilities.  ADE/ESS collaboration meetings with the ADE sections of High School Renewal and Improvement, Career and Technical Education, Dropout Prevention, and School Counselors were scheduled and held approximately every three months and resulted in crosstraining for conferences Part B Arizona Annual Performance Report for FFY 2011 111 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 sponsored by each ADE section on the topic of secondary transition. c) On an annual basis, create and disseminate information through a variety of sources: annual statewide conference, monitoring alerts, Web site, and listserv announcements d) On an annual basis, analyze preand post-training  Intra-ADE/ESS collaborative efforts included: monthly meetings with PINS (Parent Information Network specialists) as fellow AZCoPT members, as well as involvement with PINS during quarterly Transition Conference Planning Committee meetings; at least quarterly meetings with ESS Program Support to discuss the use of the Annual Site Visit Log (ASVL); review/revision of the secondary transition section of the monitoring manual and needed secondary transition trainings for ESS program specialists and PEAs; and the development of collaborative presentations with the ADE/ESS assistive technology unit. Activities completed from 8/1/11 through 6/30/12. 7/1/11– 6/30/13 ADE/ESS Transition Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists  Four secondary transition presentations were offered at the ADE/ESS Directors Institute in August 2011 for approximately 78 participants.  Ninety-six sessions either directly or indirectly related to Indicator 1 were offered at Arizona’s Eleventh Annual Transition Conference held in October 2011.  Between July 2011 and June 2012, the ADE/ESS Secondary Transition Web site was updated to include additional and/or revised Indicator 13 materials from NSTTAC, IDEA Partnership, and other secondary transition technical assistance centers. The Web address is http://www.azed.gov/specialeducation/specialprojects/secondary-transition/. Activities completed from 7/1/11 to 6/30/12. Part B Arizona Annual Performance Report for FFY 2011 112 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 data collected  Analysis of ASVL pre-training through the Annual data was completed in January Site Visit Log (ASVL) 2012 after all ESS program for each PEA to specialists were able to complete determine level of at least one PEA annual site visit compliance on all during fall 2011. The analysis eight required showed a 76.5% average for components of compliance with the eight items of Indicator 13 Indicator 13. ADE/ESS Program Specialists  Post-training data analysis of all PEAs that received a pre-training annual site visit and were trained in secondary transition during FFY 2011 showed a 79.1% average for compliance with the eight items of Indicator 13. 2) Provide a twoyear capacity building grant to PEAs to participate in the Secondary Transition Mentoring Project (STMP) Team Training a) On an annual basis, identify PEAs who meet eligibility requirements and extend invitations to them to participate in STMP trainings b) On an annual basis, provide indepth and ongoing professional development on transition requirements and best practices Through pre- and post-training analysis, an increase of 2.6% in compliance for Indicator 13 was demonstrated after training and technical assistance was provided to PEAs by transition and education program specialists. Activities completed from 1/1/11 to 7/30/12. 7/1/11– 6/30/13  Using Annual Site Visit (ASV) data for PEAs currently in Years 1, 2, or 3 of the monitoring cycle, 23 PEAs achieving less than 100% compliance on Indicator 13 were invited to apply for a noncompetitive Secondary Transition Mentoring Project (STMP) capacity building grant.  Fourteen PEAs were accepted for participation in Year 1 of the STMP Team training in July 2011. Activities completed from 7/1/11 through 6/30/12. ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/11– 6/30/13 ADE/ESS Transition Specialists  STMP participants attended Arizona’s Eleventh Annual Transition Conference, which included a STMP team orientation and designated sessions.  Training materials and activities were designed, created, and disseminated by ADE/ESS in collaboration with the University of Part B Arizona Annual Performance Report for FFY 2011 113 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Kansas Transition Coalition. Trainings focused on: identifying PEA barriers to meeting transition requirements; developing an action plan to eliminate barriers; creating IEPs that meet transition requirements; implementing training to build intra-PEA capacity to attain 100% compliance on secondary transition requirements; and determining improvements made and targeting areas still in need of improvement.  Team training for each cohort consisted of three two-day faceto-face trainings throughout the year. Additionally, Year 1 teams completed an online short course specially designed for STMP teams and focused on best practices in transition planning. Activity completed from 7/1/11 through 6/30/12. c) On an annual basis, analyze preand post-training data collected during  Measures to determine STMP trainings for effectiveness of STMP training each PEA that included the use of: participated to a) team process checklist; determine level of b) interrater reliability measures compliance on all for IEP file review; eight required c) training session evaluations; components of d) STMP team self-reported Indicator 13 progress toward completion of action plans; and e) feedback provided by ADE/ESS monitoring program specialists on compliance. 7/1/11– 6/30/13 ADE/ESS Transition Specialists Data from the measures are as follows: a) April 2012 trainings for both STMP cohorts included the completion of a 37-item “team process” checklist. Results from the checklist use indicated both Year 1 and Year 2 teams reported an average of 92% for the item “all team members are highly knowledgeable of transition compliance requirements” and an average of 69% for the item Part B Arizona Annual Performance Report for FFY 2011 114 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) c) d) e) “STMP team collaboratively develops and implements an action plan that addresses the prioritized needs.” Training for both cohorts included instruction and practice activities using interrater reliability measures. All STMP training sessions during FFY 2011 included instruction, activities, and Indicator 13 file reviews. Based on a 5-point Likert scale (1 lowest and 5 highest), evaluation results indicated an average score of 4.4 across all activities for both cohorts. Additionally, all team members completed a pre-/postevaluation of their competency in meeting Indicator 13 requirements that asked the question, “How prepared do you feel?” Based on a 5-point Likert Scale (1 equaled unprepared and 5 equaled prepared) and using a Paired Samples T-Test, all scores indicated a statistically significant increase in knowledge from the beginning to the end of the training for FFY 2011 for both cohorts (Years 1 and 2). The areas reporting the greatest increase in knowledge were transition assessments and documenting transition services and course of study that will support the student in reaching postsecondary goals. Review of each STMP team’s action plan indicated all teams set goals to build and sustain systems to ensure 100% compliance on Indicator 13 school- or district-wide. All teams reported progress in meeting these goals. During the April 2012 training, Year 2 teams met with ADE/ESS program specialists to receive informal feedback on IEPs written during the Part B Arizona Annual Performance Report for FFY 2011 115 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 second year of the STMP training program. Feedback indicated significant improvement in PEA knowledge and compliant practices. Correction of FFY 2010 Findings of Noncompliance (if State reported less than 100% compliance) Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 89.2% 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) 163 2. Number of FFY 2010 findings the State verified as timely corrected (corrected within one year from the date of notification to the LEA of the finding) 137 3. Number of FFY 2010 findings not verified as corrected within one year [(1) minus (2)] 26 Correction of FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 26 5. Number of FFY 2010 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 26 6. Number of FFY 2010 findings not verified as corrected [(4) minus (5)] 0 The ADE/ESS specialists reviewed the child specific files from the monitoring to determine that the PEA implemented the eight components of the secondary transition requirements for the children, unless they were no longer within the jurisdiction of the PEA. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine that the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to secondary transition in conformity with 34 CFR §§ 300.320 (b) and 300.321 (b). Actions Taken if Noncompliance Not Corrected All FFY 2010 noncompliance has been corrected, and Arizona has verified correction for all FFY 2010 findings. When findings of noncompliance are made, Arizona works collaboratively with the PEAs as the agencies identify the root causes of their continuing noncompliance through drill downs in the specific focus area that affects this Indicator. However, when noncompliance is not corrected within one year, the ADE/ESS uses a variety of methods to ensure that all public agencies meet the requirements of State and Federal statutes related to special education. The enforcement actions taken by ESS for the PEAs that are unable to demonstrate compliance within one year from the date of written notification are as follows: Part B Arizona Annual Performance Report for FFY 2011 116 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011  Enforcement of corrective action plan (CAP) activities as outlined in the current agency CAP, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Review and revision of the current CAP to develop targeted activities that address the continued noncompliance, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Assignment of a special monitor.  Interruption of IDEA payments until adequate compliance is achieved. For charter schools not receiving IDEA funds, a request to begin withholding 10% of State funds.  For charter schools, a request to the appropriate board for a notice of intent to revoke the charter.  With Arizona State Board of Education approval, interruption of Group B weighted State aid or redirection of funds pursuant to 34 CFR § 300.227 (a).  Referral to the Arizona Attorney General for legal action. Verification of Correction of FFY 2010 Noncompliance (either timely or subsequent) FFY 2010 Verification of Correction from Monitoring As specified in OSEP’s June 2012 Arizona Part B FFY 2010 SPP/APR Response Table, Arizona verified that each PEA with noncompliance reflected in the data: 1) is correctly implementing 34 CFR §§ 300.320 (b) and 300.321 (b) (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through onsite monitoring; and 2) has completed the evaluation, although late, for any child whose initial evaluation was not timely, unless the child is no longer within the jurisdiction of the LEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data:  ADE/ESS specialists conducted follow-up on-site visits and/or desk audits after the monitoring to verify correction of all instances of noncompliance, including those that were child specific. The specialists reviewed the child specific files to determine that the evaluation was completed within 60 calendar days from the date of written notification of noncompliance.  ADE/ESS specialists reviewed updated data from subsequent files and/or conducted interviews with the special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including those that were child specific, were corrected and to ensure ongoing sustainability of the implementation of the regulatory requirements regarding initial evaluations. Correction of Remaining Findings of Noncompliance from FFY 2009 or Earlier Part B Arizona Annual Performance Report for FFY 2011 117 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2010, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2011 APR for Indicator 13 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2010 data). In reporting on correction of findings of noncompliance in the FFY 2011 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2010 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2011 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2011 APR for Indicator 13 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2010 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. If the State does not report 100% compliance in the FFY 2011 APR, the State must review its improvement activities and revise them, if necessary. Arizona reviewed its improvement activities for Indicator 13. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 118 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / Effective Transition Indicator 14: Post School Outcomes Percent of youth who are no longer in secondary school, had IEPs in effect at the time they left school, and were: A. Enrolled in higher education within one year of leaving high school. B. Enrolled in higher education or competitively employed within one year of leaving high school. C. Enrolled in higher education or in some other postsecondary education or training program; or competitively employed or in some other employment within one year of leaving high school. (20 U.S.C. 1416 (a)(3)(B)) Measurement A. Percent enrolled in higher education = [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education within one year of leaving high school) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100. B. Percent enrolled in higher education or competitively employed within one year of leaving high school = [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education or competitively employed within one year of leaving high school) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100. C. Percent enrolled in higher education, or in some other postsecondary education or training program; or competitively employed or in some other employment = [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education, or in some other postsecondary education or training program; or competitively employed or in some other employment) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100. Overview of Issue / Description of System or Process In the State Performance Plan (SPP) for FFY 2009, Arizona established a baseline for each of the three measures A, B, and C consistent with the language of the revised measurement table (May 2010), developed new targets for measures A, B, and C, and identified improvement activities through FFY 2012. In the APR development for FFY 2011, anomalies in Indicator I4A and 14B were observed. The ADE/ESS drilled into the data and discovered a computer coding error that affected the hierarchical distribution of outcome categories. In an effort to assure valid and reliable data were reported, ADE/ESS corrected the coding error. Simultaneously, the ADE Information Technology (IT) division reviewed the National PostSchool Outcomes Center’s (NPSO’s) “Calculations for Indicator 14 Using Post-School Outcome Survey” contained in the updated June 14, 2010, Post-School Outcome Data Collection Protocol, to assure there were no other IT coding inconsistencies. During this process, the IT division discovered three survey question response options that were not included in the NPSO guidance document. The ADE/ESS requested clarification on the missing response options, which resulted in the NPSO revising their calculation table. Part B Arizona Annual Performance Report for FFY 2011 119 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The NPSO calculation table, dated October 26, 2012, was then used to revise and update the ADE/ESS PSO Survey online application and calculations to align fully with NPSO’s revised Post-School Outcome Data Collection Protocol. When the hierarchy computer coding error was corrected and the new calculations applied, Arizona’s measures A, B, and C changed, necessitating a reset of the FFY 2009 baseline data. The FFY 2005–2012 State Performance Plan, FFY 2011 Revision, includes the new FFY 2011 baseline data and new FFY 2012 targets for measures A, B, and C. To summarize:  The FFY 2005–2012 State Performance Plan, FFY 2009 Revision, in which Arizona established a baseline for measures A, B, and C consistent with the language of the revised measurement table (May 2010), developed new targets for measures A, B, and C, and identified improvement activities through FFY 2013, has been revised for FFY 2011.  The FFY 2011 data, discussion of outcomes, and FFY 2012 targets are based on December 2012 corrected data.  The FFY 2005–2012 State Performance Plan, FFY 2011 Revision, reports new baseline data for measures A, B, and C, and new targets for FFY 2012. Data Data Collection Methods The ADE/ESS used a sampling procedure to collect Post School Outcome (PSO) data. Over the course of the State Performance Plan (SPP), each PEA serving students 16 years old and older is asked to collect and report post school outcomes data during the second year of the six-year monitoring cycle. The monitoring cycle is a representative sample of Arizona’s districts and charter schools and the representative sample is based on the categories of disability, race, and gender. The ADE/ESS sampling plan was approved by OSEP. FFY 2011 marked the beginning of cycle two of Arizona’s PSO Data Collection requirement, in which all eligible PEAs are assigned to a collection year for inclusion in the SPP. Of Arizona’s eligible PEAs, 54 were identified to participate in the PSO Survey requirement for FFY 2011. Of this number, 13 PEAs did not have any leavers who met the criteria (youth with a current IEP who aged out, graduated, or dropped out) to be eligible to participate in the PSO Survey during the 2010–2011 school year. This resulted in 41 PEAs that had eligible leavers and these PEAs were required to conduct the PSO Survey during FFY 2011. It should be noted that of these 41 PEAs, two were charter schools that closed during the 2010– 2011 school year and no longer had staff available to complete the PSO Survey requirement for students who exited. In order to participate in the PSO Survey, PEAs gather contact information on student leavers and either input the data into the online PSO data collection system or maintain contact information locally. The PSO data collection system uses a secure application as part of the ADE Common Logon. The application includes an auto-population of student demographic information and exit reason imported directly from the Student Accountability Information System (SAIS), a Web-based system for reporting all student-level details to the ADE. PEAs designate district or charter school personnel to contact student leavers or designated family members (i.e., parent, grandparent, or guardian), conduct phone interviews, and input survey data into the online PSO data collection system. Youth or family members were contacted Part B Arizona Annual Performance Report for FFY 2011 120 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 between July 1 and September 30, 2012, after being out of school for at least one year. Arizona’s two PEAs with an average daily membership exceeding 50,000 are included in the data collection each year. Definitions The following definitions are used by the ADE/ESS in the data collection and reporting for Indicator 14: Higher Education includes youth who have been enrolled on a full- or part-time basis in a community college (two-year program) or a college/university (four- or more year program) for at least one complete term, at any time in the year since leaving high school. Competitive Employment includes youth who have worked for pay at or above the minimum wage in a setting with others who are nondisabled for a period of 20 hours a week for at least 90 days at any time in the year since leaving high school. This includes military employment. Other Postsecondary Education or Training includes youth enrolled on a full- or part-time basis for at least one complete term at any time in the year since leaving high school in an education or training program (e.g., Job Corps, adult education, workforce development program, or vocational technical school that is less than a two-year program). Some Other Employment includes youth who have worked for pay or been self-employed for a period of at least 90 days at any time in the year since leaving high school. This includes working in a family business (e.g., farm, store, ranching, catering services, etc.). Respondents are youth, young adults, or designated family members who answer the PSO Survey. Leavers are youth or young adults who left school by graduating, aging out, or leaving school early (i.e., dropped out) or who were expected to return to school and did not. Response Rate and Representativeness The response rate was 70%. Table 14.1 shows that Arizona’s FFY 2011 sample included 2,032 youth who were eligible to take the survey. Interviews were conducted with 1,423 youth, young adults, or their family members. The FFY 2011 70% response rate represents a 9% increase in response rate over FFY 2010. Table 14.1 Response Rate Calculation Number of leavers in the sample 2,149 Subtract the number of youth ineligible (those who had returned to school, or were deceased, or whose data were uploaded by the PEA to the SAIS system in error) 80 + 1 + 36 = 117 Number of youth eligible to contact 2,032 Number of completed surveys 1,423 Response rate (1,423 / 2,032) * 100 70% The ADE/ESS used the NPSO Response Calculator to calculate representativeness of the respondent group on the characteristics of (a) disability type, (b) ethnicity, (c) gender, and (d) exit status (e.g., dropout) to determine whether the youth who responded to the interviews were similar to, or different from, the total population of youth with an IEP who exited school during 2010–2011. According to the NPSO Response Calculator, differences between the Respondent Group and the Target Leaver Group Part B Arizona Annual Performance Report for FFY 2011 121 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 +/− 3% are important. Negative differences indicate an under representativeness of the group and positive differences indicate over representativeness. In Table 14.2, bolded text is used to indicate a difference exceeding a +/− 3% interval. As shown in Table 14.2, respondents were slightly overrepresented in the category of all other disabilities (AO) and underrepresented in the category of dropout. All other categories are deemed representative of 2010–2011 target leavers. Although the dropout category continues to be underrepresented, the difference for FFY 2011 was significantly less than in FFY 2010 and FFY 2009 (−4.86% vs. −6.77% vs. −6.81%), respectively. Each year (FFY 2010 and FFY 2011), the number of youth responding to the PSO Survey from the dropout category has increased. The ADE/ESS is encouraged by the increase in representativeness in the dropout category. The ADE/ESS will continue its efforts to increase response rates, especially among youth who drop out. Table 14.2 Representativeness of Responders to Leavers FFY 2011 Representativeness Overall Target Leaver Totals Response Totals Target Leaver Representation Respondent Representation Difference LD ED MR AO Female Minority ELL Dropout 2,032 1,254 208 187 383 731 363 52 303 1,423 858 123 128 314 522 247 34 143 61.71% 10.24% 9.20% 18.85% 35.97% 17.86% 2.56% 14.91% 60.30% 8.64% 9.00% 22.07% 36.68% 17.36% 2.39% 10.05% −1.42% −1.59% −0.21% 3.22% 0.71% −0.51% −0.17% −4.86% Missing Data Arizona’s PSO response rate for FFY 2011 was 70% (2,032 youth eligible to contact and 1,423 respondents). The FFY 2011 PSO Survey is missing data on 30% or 609 former students. An analysis of missing data indicated that the largest segments of missing data were the result of either schools’ not being able to contact leavers after three attempts (303) or their not having correct contact information (237). In FFY 2010, approximately 150 surveys could not be conducted because contact information was not collected by the PEA. A marked improvement in this category was made in FFY 2011, with only 13 surveys identified as contacts not collected. The ADE/ESS attributes this improvement to increased efforts during trainings to prepare PEAs to participate in the PSO Survey and providing a sample form on the ADE/ESS Web site for collecting contact information. Selection Bias The category of all other disabilities (AO) was slightly over represented by 0.22%. This result is evidence that Arizona PEAs are successfully reaching exiters for them to complete the survey. The under representativeness of youth and young adults in the category of dropout could be attributed to the fact that, in general, this group of youth is a difficult population to reach. Although the dropout category continues to be under represented, the difference between the number of youth and young adults who responded to the survey and were eligible for the survey in FFY 2011 was significantly less than in FFY 2010 and FFY 2009 (−4.86% vs. −6.77% vs. −6.81%). The ADE/ESS is encouraged by the increase in representativeness in the dropout category. The State will continue to work with NPSO to identify strategies to encourage survey responses from youth in the dropout category. Part B Arizona Annual Performance Report for FFY 2011 122 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Baseline Data for FFY 2011 FFY 2011 (2011–2012) Number of respondent leavers Percentage of respondent leavers Total respondents Indicator 14 Baseline Data 14A 14B 14C 371 849 1048 26.1% 59.7% 73.6% 1423 Discussion of Baseline Data for FFY 2011 December 2012 Corrected Data There were 1,423 total respondents to the PSO Survey. Each leaver was counted once in the highest category: 1) 371 respondent leavers were enrolled in “higher education” 2) 478 respondent leavers were engaged in “competitive employment” (and not counted in number 1 above) 3) 114 respondent leavers were enrolled in “some other postsecondary education or training” (and not counted in numbers 1 or 2 above) 4) 85 respondent leavers were engaged in “some other employment” (and not counted in numbers 1, 2, or 3 above) Thus: 14A = 371 (#1) divided by 1,423 (total respondents) = 26.1% 14B = 371 (#1) + 478 (#2) divided by 1,423 (total respondents) = 59.7% 14C = 371 (#1) + 478 (#2) + 114 (#3) + 85 (#4) divided by 1,423 (total respondents) = 73.6% Figure 14.1, Arizona PSO Survey FFY 2011 Cohort, 2010–2011 School Year Exiters, shows the outcome categories, including the not engaged category, the number of leavers in each category, and the percentage of leavers in each outcome category. The table below the chart shows the percentages for each measure A, B, and C. As shown in Figure 14.1, the largest percentage of leavers was competitive employment with 33.6% (n = 478) of leavers counted in this category. The second largest percentage of leavers was not engaged with 26.4% (n = 375). The remaining categories, from largest percentage to smallest, were enrolled in higher education, 26.1% (n = 371); enrolled in other postsecondary education or training, 8% (n = 114); and some other employment, 6% (n = 85). Part B Arizona Annual Performance Report for FFY 2011 123 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Baseline Data for FFY 2011 Figure 14.1, Arizona PSO Survey FFY 2011 Cohort, 2010–2011 School Year Exiters Part B Arizona Annual Performance Report for FFY 2011 124 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Proposed Targets for FFY 2012 Targets were set based on the FFY 2011 baseline data and input from stakeholders. FFY Measurable and Rigorous Targets 14A 14B 14C 26.6% 60.2% 74.1% 2012 Part B Arizona Annual Performance Report for FFY 2011 125 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Disaggregated Outcomes by Subgroups The ADE/ESS used the NPSO Data Display Templates to allow for a thorough understanding of the post school outcomes of Arizona’s youth and young adults. The outcomes were examined by each subgroup: gender, disability type, ethnicity, and exit type. Post School Outcomes by Gender As displayed in Figure 14.2, Indicator 14: Arizona’s Post School Outcomes by Gender, female leavers in Arizona had similar outcomes to males in the areas of some other employment and enrolled in other postsecondary education or training. Analysis of engagement in higher education and competitive employment reveals that females are slightly more likely (6%) to have enrolled in higher education (30% vs. 24%) and slightly less likely (7%) to be competitively employed than their male counterparts (29% vs. 36%). The ADE/ESS will share this information with PEAs and assist districts and charters in analyzing root causes for these gender discrepancies. Figure 14.2, Indicator 14: Arizona’s Post School Outcomes by Gender Part B Arizona Annual Performance Report for FFY 2011 126 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Post School Outcomes by Disability Category Figure 14.3, Indicator 14: Arizona’s Post School Outcomes Respondents by Type of Disability, shows that individuals with a disability category of mental retardation (MR) were twice as likely to be not engaged (53%) than those in the overall State rate (26%). (It should be noted that the “mental retardation” category was used for this analysis since the terminology update to “intellectually disabled” (ID) in this disability category has not yet been reflected in the SAIS system.) Further analysis of the outcomes of these individuals indicates that of those who responded to the PSO Survey, 30% were engaged in some other employment (13%) or enrolled in other postsecondary education or training (17%) compared to the overall State respondent rate of 14% engagement in some other employment or some other postsecondary education. Furthermore, engagement rates in competitive employment (14%) and higher education (3%) for individuals with a disability category of MR is 17% compared to the statewide engagement rate of 60% (competitive employment = 34% and enrolled in higher education = 26%). Based on this information, in collaboration with our local and State community of practice/community transition teams, ADE/ESS will continue to work on developing strategies and resources to assist local PEAs to improve overall engagement for individuals with a disability category of intellectual disability. Outcomes for individuals with a disability category of emotional disturbance (ED) were also less positive than the overall statewide average. With the exception of individuals with a disability category of MR, those with a disability category of ED were more likely to be not engaged (34%) than their peers with a specific learning disability (SLD) (20%) or individuals with all other categories of disability (30%). They were less likely to be enrolled in higher education (18%) or be competitively employed (29%) compared to their peers with a SLD (enrolled in higher education = 28% and competitively employed = 42%) or with individuals in all other disability categories who enrolled in higher education (33%). Figure 14.3, Indicator 14: Arizona’s Post School Outcomes Respondents by Type of Disability Part B Arizona Annual Performance Report for FFY 2011 127 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Post School Outcomes by Ethnicity As displayed in Figure 14.4, Indicator 14: Arizona’s Post School Outcomes by Ethnicity, Arizona youth enrolled in higher education at a rate ranging from 48% for Asians to 18% for American Indian/Alaska Natives. Black or African Americans (38%) and Whites (29%) had enrollment rates higher than the average statewide rate of 26%, while Hispanic/Latinos had a rate of 21%, which is lower than the statewide rate. The ethnic group with the largest percentage of youth who were competitively employed was Hispanic/Latino (38%), as compared with peers who are White (32%), and Black/African American and American Indian/Alaska Native, both at 27%. The percentage of youth enrolled in other postsecondary education or some other employment was consistent across all groups, with Arizona reporting an engagement rate of less than 15% for all ethnic groups. Engaging youth who are identified as American Indian/Alaska Native continues to be a concern for ADE/ESS since those individuals experience the least successful outcomes compared to their peers in terms of engagement in higher education and competitive employment. Figure 14.4, Indicator 14: Arizona’s Post School Outcomes by Ethnicity Part B Arizona Annual Performance Report for FFY 2011 128 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Post School Outcomes by Type of Exit Figure 14.5, Indicator 14: Arizona’s Post School Outcomes by Type of Exit, discloses that individuals who earned a high school diploma had better outcomes, especially in the areas of higher education and competitive employment. Only 1% of dropouts enrolled in higher education compared to 29% of graduates; similarly, 27% of dropouts were engaged in competitive employment as compared to 34% of graduates. Dropouts had a lower rate of engagement (45%) compared to graduates (77%). Figure 14.5, Indicator 14: Arizona’s Post School Outcomes by Type of Exit Characteristics of Nonresponders A review of the 609 leavers who did not respond to the survey by demographic subgroup indicates that 66% were male, 43% identified themselves as Hispanic/Latino, 74% exited with a high school diploma, and 65% were individuals identified with the disability category of specific learning disability. The ADE/ESS will continue to share this information with PEAs and encourage schools to target these subgroups for greater participation in the PSO survey. Trend Data Since ADE/ESS is resetting the baseline using corrected data for FFY 2011, trend data will not be available until FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 129 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Dissemination of FFY 2011 Data To ensure broad dissemination of Indicator 14 results, ESS will post results on the ADE/ESS Post School Outcomes Web site at http://www.azed.gov/special-education/special-projects/secondary-transition /postschool-outcomes-survey by May 2013. The PEAs that participated in the 2012 PSO Survey will be invited to attend a 2012 PSO Results webinar, which will include State results and information on how to access and use local PSO results. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Develop, implement, and evaluate procedures and trainings needed to assure participation in Post School Outcomes (PSO) Survey by identified PEAs Sub-Activities (Objectives or Action Steps) a) Revise PSO application and survey questions to align with new Indicator 14 Table, requirements, and definitions b) Provide training to PEAs on Indicator 14 changes and the ADE/ESS PSO Survey Application Timeline Complete Activity completed 7/1/11 to 6/30/12. Enhancements to the online PSO Survey application were completed and the training site was updated in June 2012. Enhancements included a data collection tracker (PSO response rate) by school and increased verification procedures to use before categorizing survey as “unable to contact.” Activity completed 7/1/11 to 6/30/12. Projected 7/1/10– 6/30/13 Resources (Planned) ADE/ESS Transition Specialists ADE/IT Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists PEAs in the FFY 2011 PSO Survey participation cohort were identified and targeted for training. Two trainings on Indicator 14 and the enhanced online PSO Survey application were provided at the annual ADE/ESS Directors Institute in August 2011. An additional two trainings on Indicator 14 and the enhanced online PSO Survey application were provided at Arizona’s Eleventh Annual Transition Conference in October 2011. Over 75 people Part B Arizona Annual Performance Report for FFY 2011 130 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 attended at least one of these sessions. c) Analyze PSO training evaluations and survey results to determine effectiveness of trainings d) Create PSO data reports for participating PEAs to use as a measure for analyzing and improving transition practices 2) Develop, implement, and a) Provide training to STMP teams on Eleven webinar trainings on Indicator 14 and the enhanced online PSO Survey application were offered to all PEAs, including the FFY 2010 and FFY 2011 reporting cohort throughout FFY 2011. Activity completed from 7/1/11 through 6/30/12. 7/1/10– 6/30/13 ADE/ESS Transition Specialists Review and analysis of PSO training participant evaluations was completed after each training and also at the conclusion of FFY 2011. Participant evaluation forms showed significant increases on self-rating measures of knowledge from pre- to post- training. Using a scale of 1 to 5 (1 = low and 5 = high), participants reported an average growth of 1.5 points. Activity completed 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Transition Specialists One webinar training that highlighted and discussed FFY 2010 State PSO results and included a demonstration of how to access PEA reports was conducted. PSO data reports for participating PEAs were created and made available to PEAs at both the district and school levels in the online PSO Survey application to enable PEAs to obtain response rates and results by subcategories. Activity completed 7/1/11 to 6/30/12. ADE/IT Specialists 7/1/10– 6/30/13 ADE/ESS Transition Part B Arizona Annual Performance Report for FFY 2011 131 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 sustain local community transition teams during Year 2 of the STMP capacity building team training grant evidence-based practices in developing local community transition teams b) Participate in PSO survey and share results with local community transition teams Specialists This activity was completed and discontinued because it is integrated into the STMP training program. ADE/ESS, in collaboration with STMP grant coaches from the University of Kansas/Transition Coalition, provided training and created instructional materials designed to facilitate the development of local Community Transition Teams (CTTs) for Year 2 STMP grant participants. The goals included: developing interagency CTTs; working across stakeholders to identify and prioritize community transition needs and challenges to attaining successful post school outcomes; developing protocols for working across stakeholders to increase employability and postsecondary participation of students as they leave high school; and improving post school outcome data. An overview of CTTs was provided to Year 1 STMP teams. Activity completed 7/1/11 7/1/10– to 6/30/12. 6/30/13 This activity was completed and discontinued because it is integrated into the STMP training program. ADE/ESS Transition Specialists STMP Grant Year 2 PEAs STMP teams were provided training, reports, and materials for use with local CTT teams on State and local PSO response rates, representativeness, and outcome data. Part B Arizona Annual Performance Report for FFY 2011 132 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 3) Provide technical assistance to PEAs on strategies to reach exiters to increase response rate, especially targeting dropouts and individuals from minority groups a) Develop and disseminate flyers and printed materials for use by PEAs to inform students and families and encourage participation in the PSO survey b) Encourage use of the Parent Advocacy Coalition for Educational Rights (PACER)/NPSO– created technical assistance video “Be a Superstar—Take the Survey” YouTube video and provide a link to the video on the ADE/ESS Web site Activity completed 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Transition Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists This activity was completed and discontinued. The ESS Web site is updated at least annually. A one-page summary of Arizona’s PSO Survey, an Arizona PSO glossary, and Arizona PSO parent and student (English and Spanish versions) announcement flyers were adapted from NPSO, disseminated via e-mail, and posted on the ADE/ESS Secondary Transition Web site: http://www.azed.gov /special-education /special-projects /secondary-transition /post-school-outcomessurvey/. These support documents were also included on the homepage of the online PSO Survey application. Activity completed 7/1/11 to 6/30/12. This activity was completed and discontinued. The ESS Web site is updated at least annually. ADE/ESS transition specialists and STMP trainers highlighted and encouraged the use of the PACER/NPSO–created technical assistance video “Be a Superstar—Take the Survey” YouTube video. A link to the video and the flyers was posted on the ADE/ESS Secondary Transition Web site: http://www.azed.gov /special-education Part B Arizona Annual Performance Report for FFY 2011 133 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 c) Provide session(s) at Arizona’s Annual Transition Conference devoted to increasing participation in the PSO Survey d) Survey PEAs to determine use of strategies 4) Work with the National PostSchool Outcomes (NPSO) Technical Assistance Center as an “intensive state” a) Implement technical assistance received from NPSO 5) Revise Arizona’s online PSO data collection system to include missing data and enable future trend analysis a) Revise PSO online data collection system to include reason for PEA failure to collect survey information /special-projects /secondary-transition/postschool-outcomes-survey/. Activity completed 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Transition Specialists Two sessions were offered during Arizona’s Eleventh Annual Transition Conference, and two sessions were planned for Arizona’s Twelfth Annual Transition Conference. Activity not completed during FFY 2011. 7/1/10– 6/30/13 ADE/ESS Transition Specialists This activity will be implemented during FFY 2012. Activity completed 7/1/11 to 6/30/12. 5/1/11– 6/30/13 ADE/ESS Transition Specialists This activity was completed and discontinued. PSO sessions are included in the conference schedule each year at Arizona’s Annual Transition Conference. ADE/ESS has taken steps to implement technical assistance received from NPSO as evidenced by enhancements made to the online PSO Survey application, updated trainings provided at conferences and via webinars to local PEAs, and incorporation of NPSO data collection and analysis tools. Activity completed 7/1/11 to 6/30/12. A review of the enhancements made to the PSO online data collection system showed that no additional changes were required related to PEA failure to collect NPSO Technical Assistance Center 7/1/10– 6/30/13 ADE/ESS Transition Specialists ADE/IT Specialists Part B Arizona Annual Performance Report for FFY 2011 134 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) Revise PSO online data collection system to allow for the exploration of additional data related to nonengaged youth 6) Increase the number of youth who respond to the post school outcomes survey from 61% (FFY 2010) to 67.5% a) Design and implement a marketing plan to target increased participation by male and ethnic minority youth and young adults b) Enhance the online PSO Survey application to filter and group data by size of PEA and county c) Use the NPSO response calculator to track responses during the data collection to monitor response rates 7) Increase the rate of engagement for students with a disability category of mental retardation from 47% (FFY 2011 baseline) to 48% a) Offer 3 regional trainings a year specifically designed to increase awareness of available options for postsecondary education, training, and employment for students with intellectual disabilities survey information at this time. This activity is discontinued. Activities completed 7/1/11 to 6/30/12. In FFY 2011, a plan for enhancing the online PSO Survey application to include data regarding nonengaged youth was researched and discussed. Further planning and implementation of this enhancement will begin in FFY 2012. Activities completed 1/1/12 to 6/30/12. This activity was completed and the goal was exceeded, as the response rate for FFY 2011 was 70%, an increase of 2.5 percentage points above the target of 67.5%. This activity will be completed during FFY 2012. 7/1/10– 6/30/13 ADE/IT Specialists 1/1/12– 6/30/12 11/1/11– 6/30/12 7/1/12– 9/30/12 The Arizona PSO Online application was enhanced to enable it to track response rates. Activities completed 1/1/12 to 6/30/12. 1/1/12– 6/30/13 Three regional trainings cofacilitated by ADE/ESS and the Division of Developmental Disabilities (DDD) were held highlighting local options for postsecondary engagement ADE/ESS Transition Specialists Parent Information Network Specialists Activity completed 7/1/11 to 6/30/12. This improvement goal is revised to reflect the new FFY 2011 baseline data. ADE/ESS Transition Specialists ADE/ESS Transition Specialists ADE/IT Specialists ADE/ESS Transition Specialists ADE/ESS Transition Specialists AZ Community of Practice on Transition (AZCoPT) Part B Arizona Annual Performance Report for FFY 2011 135 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 for students with intellectual disabilities. Review and analysis of evaluations from these trainings indicated an increase in knowledge pre/post-training regarding post school options for students with intellectual disabilities served through DDD. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 The targets for FFY 2012 were revised based on the new baseline data (see above). Part B Arizona Annual Performance Report for FFY 2011 136 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 15: Effective General Supervision General supervision system (including monitoring, complaints, hearings, etc.) identifies and corrects noncompliance as soon as possible but in no case later than one year from identification. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent of noncompliance corrected within one year of identification: a. # of findings of noncompliance. b. # of corrections completed as soon as possible but in no case later than one year from identification. Percent = [(b) divided by (a)] times 100. States are required to use the “Indicator 15 Worksheet” to report data for this indicator. Target Data for FFY 2011 FFY Measurable and Rigorous Target FFY 2011 100% Actual Target Data for FFY 2011 88% Arizona did not meet the target. Process to Select PEAs for Monitoring The ADE/ESS conducts compliance monitoring for IDEA procedural requirements. Arizona uses a sixyear cycle for monitoring with assigned activities always occurring in Year 4 of the cycle. The ADE/ESS directors and program specialists review each PEA’s data annually. The ADE/ESS can adjust a PEA’s monitoring schedule, and Year 4 monitoring activities can occur any time systemic issues arise related to the review of data. The data for FFY 2011 included the PEAs in Year 5 of the cycle; that is, data included those PEAs that were on a Corrective Action Plan to correct findings of noncompliance identified as a result of their monitoring activities during FFY 2010. Part B Arizona Annual Performance Report for FFY 2011 137 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The PEAs monitored each year represent a regional balance across the State. The monitoring cycle year has a mix of elementary, unified, and union high school districts, charter schools, and other public agencies such as secure care and accommodation districts. Definition of Finding for Monitoring for FFY 2010 During FFY 2010, a finding was issued when any line item was less than 100% compliant based upon the review of the components within the line item. The finding was a written notification to the PEA by the State that the line item(s) was noncompliant, and each finding included a description of a Federal or State statute or regulation. Explanation of Slippage That Occurred for FFY 2011 The slippage of 10.7 percentage points that occurred from FFY 2010 (98.7%) to FFY 2011 (88%) can be attributed to a number of factors. Five of the nine PEAs that extended past the one-year corrective action timeline were rural districts and charter schools with ongoing staff turnover throughout the corrective action year and beyond. ADE/ESS program specialists provided extensive support and technical assistance to each of the PEAs. The need to educate new staff members continually on the required corrective action was challenging and labor intensive for the specialists. The remaining four PEAs were charter schools located in the Phoenix area. The same company operated two of the four schools, and also shared staff. The ESS specialists spent significant time and effort to support the PEAs while completing the required corrective action. In addition, two of these charter schools experienced staff turnover during the course of the corrective action year that resulted in delayed corrective action. Although Arizona experienced slippage, the ADE/ESS continued to focus on the correction of noncompliance within one year. The ESS monitoring/program support director maintained communication with the executive director of the Arizona State Board for Charter Schools and the ADE deputy associate superintendent for School Improvement and Intervention. This collaboration helped to facilitate the timely correction of noncompliance when the CAP closeout for a charter school was not proceeding at the expected rate and helped to implement necessary enforcement when the CAP closeout went beyond the required timeline. The ADE/ESS provided staff development for ESS program specialists to increase understanding of the validity of compliance calls and to ensure interrater reliability. ESS mandates attendance by the specialists at a three-day summer monitoring training (six days for newly employed specialists). Follow-up meetings and workshops are scheduled throughout the school year. This staff development was planned and presented by the Monitoring Team, a group composed of the monitoring director and veteran specialists. The ADE/ESS mentoring program is another type of staff development for new ESS program specialists. Mentors were members of the Monitoring Team. New specialists shadowed the mentors on monitorings, CAP follow-up visits, technical assistance visits, and periodic meetings with PEAs. The specialists and mentors also communicated about issues that arose from regular interactions with the assigned PEA staff. The mentoring was maintained for up to one year. The director reviewed the CAPs on a weekly basis to check each PEA’s progress and scheduled followup visits and desk audits by the specialist. The director communicated with the assigned specialist for detailed updates if the CAP closeout was not progressing at a reasonable pace. Part B Arizona Annual Performance Report for FFY 2011 138 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Additionally, the monitoring director sent a monthly CAP Progression Report to the specialists and ESS directors that identified timelines toward the one-year closeout for each open monitoring. This alerted specialists to the remaining days for one-year closeouts. The director asked specialists to respond if difficulties existed that were impeding timely closeout. Strategies were then identified to assist the PEAs to close out the monitoring within the one-year timeline. The Monitoring Team played a crucial role in making progress with this Indicator. This established group, guided by the monitoring director, is a team of experienced ESS specialists that met monthly, at a minimum, to evaluate the monitoring process and system. In addition, the Monitoring Team members carried out their regular duties throughout the year. That is, they reviewed and revised the monitoring manual, which included forms and guide steps (an annual project); planned continuing support for program specialists; and designed resources and support materials for PEA administrators, evaluators, and teachers. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Develop and conduct webinars pertaining to the requirements for compliant evaluations and IEPs Sub-Activities (Objectives or Action Steps) a) Develop webinar trainings for evaluation and IEP requirements b) Conduct statewide webinars for evaluation and IEP requirements c) Collect and analyze training feedback from participants 2) Improve the general supervision system of PEAs by enhancing internal staff development d) Collect corrective action close-out (timeline) data for evaluation and IEP monitoring line items a) Review and revise, if necessary, the ADE/ESS mentoring system for ESS monitoring specialists Timeline Complete Activity completed 6/30/11. Monitoring director and monitoring team created webinars to train PEAs about evaluation process and timelines. Webinars were reviewed and revised and will be presented statewide beginning in November 2012. An update on this improvement activity will be given in the FFY 2012 APR. This activity will be done beginning in December 2012 and an update given in the FFY 2012 APR. This activity will be done beginning in June 2013 and an update given in the FFY 2012 APR. Activity completed from 7/1/11 to 6/30/12. The manual for the ESS mentoring system was updated. Projected Resources (Planned) 12/1/10– 6/30/11 ADE/ESS Directors and Specialists 7/1/11– 12/31/11 ADE/ESS Directors and Specialists 1/1/12– 4/30/12 ADE/ESS Directors and Specialists ADE/ESS Directors and Specialists 5/1/12– 6/30/12 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team This activity is completed. Part B Arizona Annual Performance Report for FFY 2011 139 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) Implement the ESS mentoring system for the monitoring specialists, based on demand and need c) Develop three-day summer monitoring training each year for ESS monitoring specialists d) Implement three-day summer monitoring training each year for ESS monitoring specialists e) Provide follow-up staff development for ESS monitoring specialists periodically throughout the year 3) Conduct Dispute Resolution presentations for PEAs and parent groups a) Review and revise presentation, support materials, and resources b) Conduct statewide presentations at various regional and statewide venues Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team This activity is completed. Activity completed as of July 2011. 7/1/11– 6/30/13 Materials on dispute resolution were reviewed, revised, and widely disseminated statewide. Activity completed 7/1/11 to 6/30/12. 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Dispute Resolution Coordinator ADE/ESS Dispute Resolution Director and Complaint Investigator The revised ESS mentoring system was implemented for all new ESS specialists who monitor. This activity is completed. Activity completed by 6/30/12. The director of program support and the monitoring team (M Team) developed the three-day summer monitoring training. This activity is completed. Activity completed by 6/30/12. Monitoring training was provided in July 2011 to all ESS specialists who monitor. This activity is completed. Activities completed from 7/1/11 to 6/30/12. Follow-up trainings and electronic communications were provided to all ESS specialists who monitor. Dispute resolution director and staff trained statewide at various conferences and workshops, including Directors Institute and Principals Institute. Part B Arizona Annual Performance Report for FFY 2011 140 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 4) Develop dispute resolution database to improve collection, maintenance, and reporting of data a) Review dispute resolution database Activity completed 7/1/11 through 6/30/12. 7/1/11– 6/30/12 b) Reconstruct dispute resolution database Corrective Action Compliance Monitor assessed the efficacy of the previously utilized database and targeted its deficiencies. Activity completed as of 9/30/12. ADE/ESS Dispute Resolution Director and Corrective Action Compliance Monitor 7/1/12– 12/31/12 ADE/ESS Dispute Resolution Director and Corrective Action Compliance Monitor 1/1/13– 6/30/13 Dispute Resolution Team 7/1/13 Dispute Resolution Team Corrective Action Compliance Monitor created a new database utilizing Microsoft Access to track and aggregate all corrective action data. c) Test phase  alpha-test to debug  beta-test to ensure ease of use d) Fully implement reconstructed dispute resolution database Timely Correction of FFY 2010 Findings of Noncompliance (corrected within one year from identification of the noncompliance) 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) (Sum of Column a on the Indicator B15 Worksheet) 725 2. Number of findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) (Sum of Column b on the Indicator B15 Worksheet) 635 3. Number of findings not verified as corrected within one year [(1) minus (2)] 90 FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance and/or Not Corrected) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 90 5. Number of FFY 2010 findings the State has verified as corrected beyond the one-year timeline (“subsequent correction”) 90 6. Number of FFY 2010 findings not yet verified as corrected [(4) minus (5)] 0 Part B Arizona Annual Performance Report for FFY 2011 141 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Verification of Correction for All Findings of Noncompliance Identified in FFY 2010 (either timely or subsequent) As specified in OSEP’s June 2011 Arizona Part B FFY 2009 SPP/APR Response Table, Arizona verified that each PEA with findings of noncompliance: 1) is correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through on-site monitoring; and 2) has corrected each individual case of noncompliance, unless the child is no longer within the jurisdiction of the PEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 (including any revisions to general supervision procedures, technical assistance provided, and/or any enforcement actions taken) Specific Actions for Correction of FFY 2010 Findings of Noncompliance for Indicator 11 Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 97% 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) 25 2. Number of FFY 2010 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 23 3. Number of FFY 2010 findings not verified as corrected within one year [(1) minus (2)] 2 Correction of FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 2 5. Number of FFY 2010 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 2 6. Number of FFY 2010 findings not verified as corrected [(4) minus (5)] 0 All findings of noncompliance from FFY 2010 related to Indicator 11 have been corrected and verified. The ADE/ESS specialists reviewed the child specific files from the monitorings to determine whether the PEAs completed the evaluation for any child whose initial evaluation was not timely, unless the child was no longer within the PEA’s jurisdiction. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine that the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to the evaluation process in conformity with 34 CFR § 300.301 (c) (1). There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Part B Arizona Annual Performance Report for FFY 2011 142 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Specific Actions for Correction of FFY 2010 Findings of Noncompliance for Indicator 12 Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 99% 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) 8 2. Number of FFY 2010 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 8 3. Number of FFY 2010 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2010 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2010 findings not verified as corrected [(4) minus (5)] 0 The PEAs submitted to ECSE the policies and procedures for early intervention transitions that were mutually agreed upon with the AzEIP service coordinators. The ADE/ECSE specialists reviewed the child specific files from the PEAs to determine whether the IEPs were developed and implemented, although late, unless the child was no longer within the PEA’s jurisdiction. The ADE/ESS specialists conducted follow-up on-site visits and/or desk audits and reviewed updated data based on subsequent student files to verify that each PEA was correctly implementing the regulatory requirements (i.e., achieved 100% compliance) in conformity with 34 CFR § 300.124 (b). There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Specific Actions for Correction of FFY 2010 Findings of Noncompliance for Indicator 13 Level of compliance (actual target data) State reported for FFY 2010 for this indicator: 89.2% 1. Number of findings of noncompliance the State made during FFY 2010 (the period from July 1, 2010, through June 30, 2011) 163 2. Number of FFY 2010 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 137 3. Number of FFY 2010 findings not verified as corrected within one year [(1) minus (2)] 26 Part B Arizona Annual Performance Report for FFY 2011 143 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Correction of FFY 2010 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2010 findings not timely corrected (same as the number from (3) above) 26 5. Number of FFY 2010 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 26 6. Number of FFY 2010 findings not verified as corrected [(4) minus (5)] 0 All findings of noncompliance from FFY 2010 related to Indicator 13 have been corrected and verified. The ADE/ESS specialists reviewed the child specific files from the monitorings to determine whether the PEAs developed and implemented IEPs that included the secondary transition regulatory requirements for any child aged 16 and above, unless the child was no longer within the PEA’s jurisdiction. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine whether the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to secondary transition in conformity with IDEA requirements. There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Specific Actions for Additional Related Requirements Identified Through the Monitoring System (Indicators 4, 9, 10, 11, and 13) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2010, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data were as follows. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  ADE/ESS specialists reviewed student files during follow-up on-site visits and/or desk audits to verify correction of all instances of noncompliance, including child specific, and to ensure that regulatory requirements were being implemented.  ADE/ESS specialists reviewed updated data from subsequent files and conducted interviews with special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including child specific, were corrected and to ensure ongoing sustainability with the implementation of the regulatory requirements.  ADE/ESS specialists reviewed the drill down analyses completed by the PEAs in targeted areas of both compliance and results SPP/APR Indicators to determine if PEAs had conducted genuine and thorough examinations of root causes. The analyses resulted in action plans to address systemic issues and to ensure sustainability of compliance. Specific Actions for Dispute Resolution (Indicators 16, 17, 18, and 19) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2010, including child specific noncompliance, and were correctly implementing the regulatory requirements were as follows. The review of data did not identify systemic noncompliance; therefore, the correction was at the student level. There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Part B Arizona Annual Performance Report for FFY 2011 144 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011  The Corrective Action Compliance Monitor (CACM) maintained a database of all corrective actions and tracked timelines to ensure timely correction of noncompliance.  As direct follow-up to a child specific finding of noncompliance, the CACM reviewed the student file, generally via desk audit but occasionally via site visit, to verify correction of any instance of noncompliance, including child specific, and to ensure that regulatory requirements were being implemented.  The CACM reviewed the corrective action plan documentation via desk audit to ensure that all instances of noncompliance were corrected. The corrective action plan documentation may include such actions as a written action plan, professional development, and/or a letter of assurance. Each PEA submitted documentation evidencing correction of all the noncompliance.  The CACM approved all corrective action plan documentation via desk audit to verify that the PEAs corrected all instances of noncompliance, including child specific, and were adhering to the regulatory requirements. Specific Actions for Additional Related Requirements for Early Childhood Transitions (Indicator 12) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2010, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data, were as follows. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  The ADE/ECSE specialists reviewed the written process and procedures for the PEAs’ early intervention transitions.  The ADE/ESS specialists and/or ADE/ECSE specialists reviewed student files during subsequent on-site visits and/or desk audits of updated data to determine if the PEAs corrected all instances of noncompliance, including child specific, and to ensure ongoing sustainability with the implementation of the regulatory requirements. Specific Actions for Other Related Requirements Pertaining to Graduation, Dropout, Assessment, School Age and Preschool LRE, Preschool Outcomes, Parent Involvement, and Post School Outcomes (Indicators 1, 2, 3, 5, 6, 7, 8, and 14) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2010, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data were as follows. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  ADE/ESS specialists reviewed student files during follow-up on-site visits and/or desk audits to verify correction of all instances of noncompliance, including child specific, and to ensure that regulatory requirements were being implemented.  ADE/ESS specialists reviewed updated data from subsequent files and conducted interviews with the special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including child specific, were corrected and to ensure ongoing sustainability with the implementation of the regulatory requirements. Part B Arizona Annual Performance Report for FFY 2011 145 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011  ADE/ESS specialists reviewed the drill down analyses completed by the PEAs in targeted areas of both compliance and results SPP/APR Indicators to determine if the PEAs had conducted genuine and thorough examinations of root causes. The analyses resulted in action plans to address systemic issues and to ensure sustainability of compliance. Actions Taken if Noncompliance Not Corrected When findings of noncompliance are made, Arizona works collaboratively with the PEAs as the agencies identify the root causes of their continuing noncompliance through drill downs in the specific focus area that affects this Indicator. However, when noncompliance is not corrected within one year, the ADE/ESS uses a variety of methods to ensure that all public agencies meet the requirements of State and Federal statutes related to special education. The enforcement actions taken by ESS for the PEAs that are unable to demonstrate compliance within one year from the date of written notification are as follows:  Enforcement of corrective action plan (CAP) activities as outlined in the current agency CAP, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Review and revision of the current CAP to develop targeted activities that address the continued noncompliance, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Assignment of a special monitor.  Interruption of IDEA payments until adequate compliance is achieved. For charter schools not receiving IDEA funds, a request to begin withholding 10% of State funds.  For charter schools, a request to the appropriate board for a notice of intent to revoke the charter.  With Arizona State Board of Education approval, interruption of Group B weighted State aid or redirection of funds pursuant to 34 CFR § 300.227 (a).  Referral to the Arizona Attorney General for legal action. Although these enforcement actions are in place, the ESS monitoring system is designed to work with the PEAs to correct the findings of noncompliance as soon as possible but no later than one year. ESS directors and specialists meet periodically throughout the year to discuss and better understand the PEAs’ data pertaining to the APR indicators, student population, and other data elements. This information is used to make decisions about the type of monitoring and need for technical assistance for each PEA. The ESS specialists use the same data during their on-site visits to the education agencies to address concerns and offer resources. Another form of technical assistance offered by ESS is the annual site visit by the ESS specialist. These site visits are designed to assist PEAs with understanding IDEA regulatory requirements through technical assistance provided as a result of student file reviews. Information from these visits is logged in a database to track the need for additional technical assistance. The ESS secondary transition specialists use the file review results related to the components of Indicator 13 to target their trainings and grants. The monitoring director, facilitator, and Monitoring Team develop the monitoring manual and train the specialists throughout the year to ensure interrater reliability for compliance calls according to regulatory requirements. The ADE/ESS staff conducts on-site and regional trainings for the PEAs to enhance understanding of compliance and the monitoring process. A major component of this is the Corrective Action Plan follow-up, which includes a strict schedule after a monitoring to ensure timely correction of noncompliance. The Early Childhood Special Education specialists also offer targeted technical assistance to districts that are not in compliance through individual trainings, monthly audits, and consultations. In addition, ESS Part B Arizona Annual Performance Report for FFY 2011 146 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 specialists review files of preschool students during the annual site visits to provide assistance and resources when needed. Correction of Remaining FFY 2009 Findings of Noncompliance Not applicable. Although Arizona reported < 100% for this Indicator in the FFY 2010 APR, all remaining FFY 2009 findings were subsequently corrected. 1. Number of remaining FFY 2009 findings noted in OSEP’s FFY 2010 APR response table for this indicator 0 2. Number of remaining FFY 2009 findings the State has verified as corrected 0 3. Number of remaining FFY 2009 findings the State has NOT verified as corrected [(1) minus (2)] 0 Correction of Any Remaining Findings of Noncompliance Identified FFY 2008 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response In reporting on correction of findings of noncompliance in the FFY 2011 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2010 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2011 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2011 APR for Indicators 11, 12, 13, and 15, Arizona reported that it verified each PEA with noncompliance (FFY 2010 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. In addition, in reporting on Indicator 15 in the FFY 2011 APR, the State must use the Indicator 15 Worksheet. In the FFY 2011 APR, Arizona uses the Indicator 15 Worksheet to report on Indicator 15. In addition, in responding to Indicators 11, 12, and 13, in the FFY 2011 APR, the State must report on correction of the noncompliance described in this table under those indicators. In the FFY 2011 APR, Arizona reports on the correction of noncompliance for Indicator 11 within the Indicator 11 section and within Indicator 15, reports on the correction of noncompliance for Indicator 12 within the Indicator 12 section and within Indicator 15, and reports on the correction of noncompliance for Indicator 13 within the Indicator 13 section and within Indicator 15. Part B Arizona Annual Performance Report for FFY 2011 147 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 148 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 INDICATOR 15 WORKSHEET Indicator/Indicator Clusters 1. Percent of youth with IEPs graduating from high school with a regular diploma. 2. Percent of youth with IEPs dropping out of high school. 14. Percent of youth who had IEPs, are no longer in secondary school and who have been competitively employed, enrolled in some type of postsecondary school, or both, within one year of leaving high school. 3. Participation and performance of children with disabilities on statewide assessments. 7. Percent of preschool children with IEPs who demonstrated improved # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring System: 28 Monitoring System: 44 Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other General Supervision System Components (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Monitoring System: 37 # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 2 corrected and verified within 13 months from identification 4 corrected and verified within 17 months from identification 1 corrected and verified within 20 months from identification Complaints: 0 Complaints: 0 Complaints: 0 Due process: 0 Due process: 0 Due process: 0 Monitoring System: 47 Monitoring System: 47 Monitoring System: 38 3 corrected and verified within 13 months from identification 1 corrected and verified within 14 months from identification 2 corrected and verified within 16 months from identification 3 corrected and verified within 20 months from identification Part B Arizona Annual Performance Report for FFY 2011 149 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters outcomes. 4A. Percent of districts identified as having a significant discrepancy in the rates of suspensions and expulsions of children with disabilities for greater than 10 days in a school year. General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 7 Monitoring System: 7 Monitoring System: 6 APR: 0 APR: 0 APR: 0 # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 1 corrected and verified within 18 months from identification Part B Arizona Annual Performance Report for FFY 2011 150 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters 4B. Percent of districts that have: (a) a significant discrepancy, by race or ethnicity, in the rate of suspensions and expulsions of greater than 10 days in a school year for children with IEPs; and (b) policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards. 5. Percent of children with IEPs aged 6 through 21 educational placements. 6. Percent of preschool children aged 3 through 5 – early childhood placement. # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Dispute Resolution: Complaints, Hearings Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Monitoring System: 45 Monitoring System: 81 Monitoring System: 70 General Supervision System Components # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 2 corrected and verified within 13 months from identification 4 corrected and verified within 14 months from identification 1 corrected and verified within 16 months from identification 4 corrected and verified within 19 months from identification Part B Arizona Annual Performance Report for FFY 2011 151 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters 8. Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities. General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Complaints: 35 Complaints: 69 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 69 Due Process: 1 Due Process: 1 Due Process: 1 Monitoring System: 37 Monitoring System: 105 Monitoring System: 93 # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 1 corrected and verified within 13 months from identification 2 corrected and verified within 14 months from identification 9 corrected and verified within 20 months from identification Complaints: 16 Complaints: 24 Complaints: 24 Due Process: 0 Due Process: 0 Due Process: 0 Part B Arizona Annual Performance Report for FFY 2011 152 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters 9. Percent of districts with disproportionate representation of racial and ethnic groups in special education that is the result of inappropriate identification. 10. Percent of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification. General Supervision System Components Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Monitoring System: 41 Monitoring System: 41 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Monitoring System: 32 APR: 0 APR: 0 APR: 0 # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 3 corrected and verified within 13 months from identification 2 corrected and verified within 14 months from identification 4 corrected and verified within 20 months from identification Part B Arizona Annual Performance Report for FFY 2011 153 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters 11. Percent of children who were evaluated within 60 days of receiving parental consent for initial evaluation or, if the State establishes a timeframe within which the evaluation must be conducted, within that timeframe. 12. Percent of children referred by Part C prior to age 3, who are found eligible for Part B, and who have an IEP developed and implemented by their third birthdays. General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring Activities: Data Collected by Census, SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 38 Monitoring System: 111 Monitoring System: 96 # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 1 corrected and verified within 13 months from identification 3 corrected and verified within 14 months from identification 11 corrected and verified within 20 months from identification Complaints: 9 Complaints: 13 Complaints: 13 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 0 Monitoring System: 0 Monitoring System: 0 Indicator 12 Census (early childhood transition): 15 Indicator 12 Census (early childhood transition): 15 Indicator 12 Census (early childhood transition): 15 Part B Arizona Annual Performance Report for FFY 2011 154 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters 13. Percent of youth aged 16 and above with IEP that includes coordinated, measurable, annual IEP goals and transition services that will reasonably enable student to meet the post-secondary goals. Other areas of noncompliance: General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 24 Monitoring System: 163 Monitoring System: 137 # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 7 corrected and verified within 14 months from identification 14 corrected and verified within 17 months from identification 5 corrected and verified within 18 months from identification Complaints: 4 Complaints: 4 Complaints: 4 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 0 Monitoring System: 0 Monitoring System: 0 Complaints: 0 Complaints: 0 Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Part B Arizona Annual Performance Report for FFY 2011 155 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Indicator/Indicator Clusters General Supervision System Components # of PEAs Issued findings in FFY 2010 (7/1/10 to 6/30/11) Sum the numbers down Column a and Column b Percent of noncompliance corrected within one year of identification = (column (b) sum divided by column (a) sum) times 100. (b) / (a) X 100 = % (a) # of findings (individual student files) of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification # of findings of noncompliance identified in FFY 2010 (7/1/10 to 6/30/11) for which correction was verified later than one year from identification 725 635 90 635 / 725 = 0.8758 X 100 = 88% Part B Arizona Annual Performance Report for FFY 2011 156 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 18: Resolution Session Effectiveness Percent of hearing requests that went to resolution sessions that were resolved through resolution session settlement agreements. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent = (3.1(a) divided by 3.1) times 100. Target Data for FFY 2011 FFY Measurable and Rigorous Target 2011 75.5% Actual Target Data for FFY 2011 number of hearing requests that went to resolution sessions that were resolved through resolution session settlement agreements number of hearing requests that went to resolution sessions 13 29 Actual Target Data for FFY 2011 45% (3.1 (a)  3.1)  100 = X 13  29 = 0.4482  100 = 45% Arizona did not meet the target. Data Data Source Data are the same as the data submitted under section 618, Table 7, Report of Dispute Resolution, under Part B of the Individuals with Disabilities Education Act. Valid and Reliable Data The ADE/ESS collects and maintains the dispute resolution data in its internal database and assures the accuracy, reliability, and validity of the data. The dispute resolution data are the same as the data reported under section 618, Dispute Resolution, Table 7. Arizona uses a Resolution Session Tracking Form, which is sent to each PEA, along with the Notice of Hearing, when a due process complaint is filed. Part B Arizona Annual Performance Report for FFY 2011 157 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 This comprehensive form, which must be submitted to the ADE/ESS and the assigned administrative law judge, provides, among other things, information about the date and outcome of each resolution session held. Explanation of Slippage That Occurred for FFY 2011 During FFY 2011, 29 resolution sessions were held pursuant to due process hearing requests, with 13 matters (45%) resulting in resolution agreements. This result reflects slippage from FFY 2010 (56%). Of the 29 resolution sessions held: 13 resulted in resolution agreements; one matter resulted in a fully adjudicated hearing; two matters were pending a due process hearing as of June 30, 2012; two matters were withdrawn by the complaining party; and two matters resulted in settlement agreements outside of the resolution session process. Slippage, however, is attributed to a single factor: of the 29 resolution sessions held, nine were between the same complaining party and that party’s school district. These parties have a very contentious relationship and a resolution agreement was not reached in any of the nine resolution sessions. It is noteworthy that of the 58 due process complaints filed:  6 resulted in a fully adjudicated hearing;  15 were pending as of June 30, 2012;  13 were resolved through a formal resolution agreement;  9 were resolved via mediation agreement;  8 were resolved via private settlement; and  7 were dismissed or withdrawn. Thus, although Arizona did not meet its target of 75.5% on this Indicator, of the 37 matters that were resolved without a hearing by or before June 30, 2012, 30 or 81% were resolved by resolution agreement, mediation agreement, or a private settlement. This indicates that, overall, the State’s due process system is successful in resolving due process complaints without the need for a fully adjudicated due process hearing. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Train PEAs on resolution session effectiveness Sub-Activities (Objectives or Action Steps) a) Identify qualified trainer Timeline Complete Activity completed as of April 2011. 10/1/10– 5/1/11 b) Provide training at a statewide conference A qualified trainer was identified to present at the next Directors Institute. Activity completed as of 6/30/12. ADE/ESS Dispute Resolution Director and Coordinator 5/1/11– 6/30/12 ADE/ESS Dispute Resolution Director and Coordinator Two sessions on leading an effective resolution session were offered at the 2011 Directors Institute. Projected Resources (Planned) Part B Arizona Annual Performance Report for FFY 2011 158 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 2) Conduct dispute resolution presentations for PEAs and parent groups a) Review and revise presentation, support materials, and resources b) Conduct statewide presentations at various regional and statewide venues 3) Review and revise dispute resolution brochure a) Revise and update brochure b) Disseminate brochure statewide and post on ADE/ESS Web site Activity completed as of July 2011. 7/1/11– 6/30/13 Materials on dispute resolution were reviewed, revised, and widely disseminated statewide. Activity completed 7/1/11 to 6/30/12. ADE/ESS Dispute Resolution Director and Coordinator 7/1/11– 6/30/13 Dispute resolution director and staff trained statewide at various conferences and workshops, including Directors Institute and Principals Institute. Activity completed as of July 2011. ADE/ESS Dispute Resolution Director and Complaint Investigator 7/1/11– 6/30/13 Dispute resolution brochure was revised and updated and is currently in circulation. Activity completed 7/1/11 to 6/30/12. ADE/ESS Dispute Resolution Director and Coordinator 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Coordinator Revised and updated brochure was disseminated at local and statewide conferences and via the Parent Information Network specialists. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 159 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 19: Mediation Agreements Percent of mediations held that resulted in mediation agreements. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent = [(2.1(a)(i) + 2.1(b)(i)) divided by 2.1] times 100. Target Data for FFY 2011 FFY Measurable and Rigorous Target 2011 85% Actual Target Data for FFY 2011 number of mediations held that resulted in mediation agreements number of mediations 29 35 Actual Target Data for FFY 2011 83% (2.1 (a) (i) + 2.1 (b) (i))  (2.1)  100 = X 9 + 20  35 = 0.8286  100 = 83% Arizona did not meet the target. Data Data Source Data are the same as the data submitted under section 618, Table 7, Report of Dispute Resolution under Part B of the Individuals with Disabilities Education Act. Valid and Reliable Data The ADE/ESS collects and maintains the dispute resolution data in its internal database and assures the accuracy, reliability, and validity of the data. The dispute resolution data are the same as the data reported under section 618, Dispute Resolution, Table 7. Explanation of Progress That Occurred for FFY 2011 Part B Arizona Annual Performance Report for FFY 2011 160 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Arizona experienced progress as compared to FFY 2010 (69%), but did not meet its target of 85% for FFY 2011. The progress of 14 percentage points may be due to the cadre of mediators in Arizona and the training provided to them by the State. Arizona maintains a list of independent contractors to serve as mediators. The State has maintained the same mediators for many years and they are becoming more experienced in the area of special education mediation. The mediators are required to complete a 40-hour course in mediation, have 20 hours of hands-on mediation experience, and have a background in education. As part of their ongoing training, mediators continue to have the opportunity to participate in quarterly mediator conference calls through the Mountain Plains Regional Resource Center and attend the ADE’s annual Directors Institute, which includes a full-day private training specifically tailored for state administrative complaint investigators, administrative law judges, and mediators. Additionally, all mediators were given the opportunity to participate in a three-day training on Facilitated IEPs during FFY 2011. Information about Arizona’s mediation system is disseminated to PEAs through trainings and conferences and upon request. Additionally, the Dispute Resolution director works closely with ADE’s Parent Information Network specialists to ensure that information on mediation is widely disseminated to parents. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Train mediators on current developments in special education law 2) Conduct dispute resolution presentations for PEAs and parent groups Sub-Activities (Objectives or Action Steps) a) Invite mediators to attend the ADE/ESS Directors Institute Timeline Complete Activity completed as of July 2011. Projected 7/1/10– 6/30/13 Mediators were invited to attend the ADE’s annual Directors Institute. b) Invite mediators to participate in the Dispute Resolution in Special Education (DRSE) Consortium quarterly conference calls for mediators a) Review and revise presentation, support materials, and resources Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 The mediators participated in the quarterly conference calls. Activity completed as of July 2011. 7/1/11– 6/30/13 Materials on dispute resolution were reviewed, revised, and widely disseminated statewide. Resources (Planned) ADE/ESS Dispute Resolution Director and Coordinator Technical Assistance for Excellence in Special Education (TAESE) ADE/ESS Dispute Resolution Director and Coordinator TAESE ADE/ESS Dispute Resolution Director and Coordinator Part B Arizona Annual Performance Report for FFY 2011 161 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) Conduct statewide presentations at various regional and statewide venues 3) Review and revise dispute resolution brochure a) Revise and update brochure b) Disseminate brochure statewide and post on ADE/ESS Web site Activity completed 7/1/11 to 6/30/12. 7/1/11– 6/30/13 Dispute Resolution director and staff trained statewide at various conferences and workshops, including Directors Institute and Principals Institute. Activity completed as of July 2011. ADE/ESS Dispute Resolution Director and Complaint Investigator 7/1/11– 6/30/13 Dispute resolution brochure was revised and updated and is currently in circulation. Activity completed 7/1/11 to 6/30/12. ADE/ESS Dispute Resolution Director and Coordinator 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Coordinator Revised and updated brochure was disseminated at local and statewide conferences and via the Parent Information Network specialists. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 162 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 20: State Reported Data State reported data (618 and State Performance Plan and Annual Performance Report) are timely and accurate. (20 U.S.C. 1416 (a)(3)(B)) Measurement State reported data, including 618 data, State Performance Plan, and Annual Performance Reports, are: a. Submitted on or before due dates (first Wednesday in February for child count, including race and ethnicity; and educational environments; first Wednesday in November for exiting, discipline, personnel and dispute resolution; December 15 for assessment; May 1 for Maintenance of Effort & Coordinated Early Intervening Services; and February 1 for Annual Performance Reports). b. Accurate, including covering the correct year and following the correct measurement. Target Data for FFY 2011 FFY Measurable and Rigorous Target 2011 100% Actual Target Data for FFY 2011 100% Arizona met the target. Data Data Source Arizona collects the 618 data and the SPP/APR data through the following sources:  Student Accountability Information System (SAIS), a Web-based system for the collection of all student data from the PEAs;  Arizona’s Instrument to Measure Standards (AIMS) and Arizona’s Instrument to Measure Standards Alternate (AIMS A), the statewide student assessment system used by the Arizona Department of Education for AMO and AZ LEARNS determinations;  Arizona Safety Accountability for Education (AZ SAFE), a Web-based system for PEAs to submit data on the discipline elements; Part B Arizona Annual Performance Report for FFY 2011 163 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011  Annual Special Education Data Collection, a Web-based system for PEAs to submit data on the preschool transition, personnel, and exit elements;  Teaching Strategies Gold, a web-based data collection system for PEAs to submit preschool outcome data;  Arizona Parent Survey, a Web-based system for parents to submit survey responses;  Arizona Monitoring System, a Web-based system to collect monitoring data; and  Dispute Resolution spreadsheet to collect, maintain, and report all dispute resolution information. Data Description Based on the Part B Indicator 20 Self-Scoring Rubric, Arizona’s results for submission of timely and accurate data were 100% for FFY 2011.  Child Count and Educational Environment, due February 1, 2012, were submitted on time and were accurate. This data applied to Indicators 4, 5, 6, 9, and 10.  Personnel, due November 7, 2012, was submitted on time and was accurate.  Exit, due November 7, 2012, was submitted on time and was accurate. This data applied to Indicators 1 and 2.  Discipline, due November 7, 2012, was submitted on time and was accurate. This data applied to Indicator 4.  Dispute Resolution, due November 7, 2012, was submitted on time and was accurate. This data applied to Indicators 16, 17, 18, and 19.  Assessment, due December 19, 2012, was submitted on time and was accurate. This data applied to Indicator 3.  Maintenance of Effort/Coordinated Early Intervening Services, due May 1, 2012, was submitted on time and was accurate.  Annual Performance Report, due February 15, 2013, was submitted on time and was accurate. Discussion of Improvement Activities Completed for FFY 2011 Primary Activity (GOAL) 1) Provide SPP/APR Indicator data to each PEA in secure format Sub-Activities (Objectives or Action Steps) a) Develop Data Profiles each federal fiscal year Timeline Complete Activity completed from 7/1/11 to 6/30/12. ADE contracted with third party to develop PEA data profiles to include Report Card, Report Card Snapshot, and Trend Report for FFY 2010. Projected 7/1/10– 6/30/13 Resources (Planned) ADE/ESS Directors and Data Management Specialist Part B Arizona Annual Performance Report for FFY 2011 164 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) Disseminate Data Profiles each federal fiscal year 2) Review and revise the ADE Student Accountability Information System (SAIS) to improve timely and accurate special education data a) ADE/ESS will meet with Information Technology (IT) staff periodically to revise procedures as necessary and address problems b) ADE/ESS will write business rules for the SAIS revisions c) ADE/ESS will analyze SAIS operations for timely and accurate collection and reporting of special education data d) Investigate the creation of two FTE positions: 1) a PEA data support, and 2) an IT SAIS developer 3) Provide information to PEAs about data accuracy and timeliness a) Develop webinars and workshops for PEAs Activity completed from 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE contracted with third party to generate user names and passwords for special education administrators allowing each PEA to access PEA Data Profiles via a secure Web-based application. Activity completed from 7/1/11 to 6/30/12. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 Daily stand-up and weekly scrum meetings held to review current work items and backlog items. Activity completed from 7/1/11 to 6/30/12. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 Collaborated with School Finance business analyst to develop new and revised business rules. Activity completed from 7/1/11 to 6/30/12. 7/1/11– 6/30/13 ADE IT Staff ADE/ESS Directors and Data Management Specialist ADE IT Staff ADE/ESS Directors and Data Management Specialist Participated in IT meetings to establish SAIS processing schedule. Monitored schedule to ensure valid and timely collection and reporting of special education data. Activity completed from 7/1/11 to 6/30/12. 7/1/11– 6/30/13 Two contracted positions filled and funded: business analyst and quality assurance staff. Activity completed from 7/1/11 to 6/30/12. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 ADE/ESS Directors and Data Management Specialist ESS Data Management developed presentations for use at the annual Directors Institute, regional workshops, and webinars. ADE IT Staff Part B Arizona Annual Performance Report for FFY 2011 165 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 b) Conduct data workshops at annual Directors Institute c) Conduct webinars and workshops for PEAs Activity completed from 7/1/11 to 6/30/12. 7/1/11– 6/30/13 ESS Data Management presented five sessions on three different topics related to the collection and reporting of special education data at the Directors Institute. Activity completed from 7/1/11 to 6/30/12. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 ADE/ESS Directors and Data Management Specialist ESS Data Management conducted a total of twenty-seven regional workshops and webinars on the collection and reporting of special education data. Part B – Indicator 20 Self-Scoring Rubric Part B Indicator 20 – SPP/APR Data APR Indicator Valid and Reliable Correct Calculation Total 1 2 3A 3B 1 1 1 1 1 1 1 1 2 2 3C 4A 4B 5 6 7 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 8 9 10 11 12 13 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 14 15 18 19 1 1 1 1 1 1 1 1 2 2 2 2 Part B Arizona Annual Performance Report for FFY 2011 166 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 APR Score Calculation Subtotal Timely Submission Points – If the FFY 2011 APR was submitted on time, place the number 5 in the cell on the right. 38 5 Grand Total – (Sum of the subtotal and Timely Submission Points) = 43.00 Part B Indicator 20 – 618 Data Table Table 1 – Child Count Due Date: 2/1/12 Table 2 – Personnel Due Date: 11/7/12 Table 3 – Ed. Environments Due Date: 2/1/12 Table 4 – Exiting Due Date: 11/7/12 Table 5 – Discipline Due Date: 11/7/12 Table 6 – State Assessment Due Date: 12/19/12 Table 7 – Dispute Resolution Due Date: 11/7/12 Table 8 – MOE/CEIS Due Date: 5/1/12 Timely Complete Data Passed Edit Check Responded to Date Note Requests Total 1 1 1 1 4 1 1 1 N/A 3 1 1 1 1 4 1 1 1 N/A 3 1 1 1 N/A 3 1 N/A N/A N/A 1 1 1 1 N/A 3 1 1 N/A N/A 2 Subtotal 23 Grand Total (Subtotal X 1.8695) = 618 Score Calculation 43.00 Indicator 20 Calculation A. APR Grand Total B. 618 Grand Total C. APR Grand Total (A) + 618 Grand Total (B) = Total N/A in APR Total N/A in 618 Base D. Subtotal (C divided by Base*) = 43.00 43.00 86.00 0 0 86.00 1.00 Part B Arizona Annual Performance Report for FFY 2011 167 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 E. Indicator Score (Subtotal D x 100) = 100.00 *Note any cell marked as N/A will decrease the denominator by 1 for APR and 1.8695 for 618. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response In reporting on Indicator 20 in the FFY 2011 APR, the State must use the Indicator 20 Data Rubric. Arizona used the Indicator 20 Data Rubric in the FFY 2011 APR. If the State does not report 100% compliance in the FFY 2011 APR, the State must review its improvement activities and revise them, if necessary. Arizona reviewed its improvement activities for Indicator 20. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2012 There are no revisions for FFY 2012. Part B Arizona Annual Performance Report for FFY 2011 168 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Attachments The following are attachments to the FFY 2011 APR: Attachment 1  List of Acronyms and Terms Part B Arizona Annual Performance Report for FFY 2011 169 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 Attachment 1 List of Acronyms and Terms AAC Arizona Administrative Code ADE Arizona Department of Education AIMS Arizona’s Instrument to Measure Standards AIMS A Arizona’s Instrument to Measure Standards Alternate ALJ Administrative Law Judge AMO Annual Measurable Objective APR Annual Performance Report ARS Arizona Revised Statutes ASVL Annual Site Visit Log AYP Adequate Yearly Progress AZCoPT Arizona Community of Practice on Transition AzEIP Arizona Early Intervention Program for Infants and Toddlers AZHSRI Arizona High School Renewal and Improvement Initiative Az SAFE Arizona Safety Accountability for Education AZ TAS Arizona Technical Assistance System CACM Corrective Action Compliance Monitor CAP Corrective Action Plan CEIS Coordinated Early Intervening Services CFR Code of Federal Regulations CoP Community of Practice CSPD Comprehensive System of Personnel Development CTE Career and Technical Education CTT Community Transition Team DAC Data Accountability Center Part B Arizona Annual Performance Report for FFY 2011 170 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 DBHS Division of Behavioral Health Services DDD Division of Developmental Disabilities DI Directors Institute ECE Early Childhood Education ECQUIP Early Childhood Quality Improvement Practices Process ECSE Early Childhood Special Education ESEA Elementary and Secondary Education Act ESS Exceptional Student Services FAPE Free Appropriate Public Education FFY Federal Fiscal Year GOLD Teaching Strategies GOLD (early childhood assessment) Group B Arizona Funding Category for Significant Disabilities IDEA The Individuals with Disabilities Education Act IEP Individualized Education Program IGA Intergovernmental Agreement IT Information Technology LRE Least Restrictive Environment M Team Monitoring Team MOE Maintenance of Effort MPRRC Mountain Plains Regional Resource Center NASDSE National Association of State Directors of Special Education NCCRESt National Center for Culturally Responsive Educational Systems NDPC-SD National Dropout Prevention Center for Students with Disabilities NPSO National Post School Outcomes Center NSTTAC National Secondary Transition Technical Assistance Center OCSHCN Office for Children with Special Health Care Needs Part B Arizona Annual Performance Report for FFY 2011 171 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 OELAS Office of English Language Acquisition Services OSEP Office of Special Education Programs/U.S. Department of Education PBISAz Positive Behavioral Interventions and Supports of Arizona PEA Public Education Agency PINS Parent Information Network Specialist PSO Post School Outcome PTI Parent Training and Information Center R and E Research and Evaluation RSA/VR Rehabilitation Services of Arizona/Vocational Rehabilitation RTI Response to Intervention SAIS Student Accountability Information System SEAP Special Education Advisory Panel SEAS-Math Special Education Achieving Success in Mathematics SPP State Performance Plan STMP Secondary Transition Mentoring Project SUMS Special Education Using Mathematics for School Improvement TA Technical Assistance TAESE Technical Assistance for Excellence in Special Education WRR Weighted Risk Ratio Part B Arizona Annual Performance Report for FFY 2011 172 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2011 The contents of this publication were developed with funds allocated by the U.S. Department of Education under The Individuals with Disabilities Education Act. These contents do not necessarily represent the guideline of the agency, nor should endorsement by the federal government be assumed. The Arizona Department of Education of the State of Arizona does not discriminate on the basis of race, religion, color, national origin, sex, disability or age in its programs, activities or in its hiring and employment practices. For questions or concerns regarding this statement, please contact Administrative Services at 602-542-3186. Printed in Phoenix, Arizona, by the Arizona Department of Education Part B Arizona Annual Performance Report for FFY 2011 173